Stefano Marinelli on Nostr: Bots like Brew. Brew doesn't like bots. From the BSD Cafe's reverse proxy: if ...
Bots like Brew.
Brew doesn't like bots.
From the BSD Cafe's reverse proxy:
if ($http_user_agent ~* (bytespider)) {
return 403;
}
if ($http_user_agent ~* (AhrefsBot)) {
return 444;
}
if ($http_user_agent ~* (Amazonbot)) {
return 444;
}
#BSDCafeStatus #BSDCafeServices #StopScrapingTheBSDCafe
Published at
2024-07-03 19:12:02Event JSON
{
"id": "7f855f155d4b773bfb9ca81b3d672c438ce6c5995ae59a8c9974c7a29685d577",
"pubkey": "ae3bf73746446a5edd319cef72033b539598a21ad07c2c2c4a84fbeaf22b835a",
"created_at": 1720026722,
"kind": 1,
"tags": [
[
"t",
"bsdcafestatus"
],
[
"t",
"bsdcafeservices"
],
[
"t",
"stopscrapingthebsdcafe"
],
[
"proxy",
"https://mastodon.bsd.cafe/users/stefano/statuses/112723671290039761",
"activitypub"
]
],
"content": "Bots like Brew.\nBrew doesn't like bots.\n\nFrom the BSD Cafe's reverse proxy:\n\n if ($http_user_agent ~* (bytespider)) {\n return 403;\n }\n\n if ($http_user_agent ~* (AhrefsBot)) {\n return 444;\n}\n\n if ($http_user_agent ~* (Amazonbot)) {\n return 444;\n}\n\n#BSDCafeStatus #BSDCafeServices #StopScrapingTheBSDCafe",
"sig": "89e36679d00af2fada539edc24183cc8fdf8091576163adba89efbabf78753a99bbca17033ee5fb52176bf68d07dd1997833cfd595b91f3b8136c43203890dbd"
}