gyptazy on Nostr: Is the crawler from #META / #Facebook acting up again? My traffic monitoring system ...
Is the crawler from #META / #Facebook acting up again? My traffic monitoring system alerted me to some unusual activity on my CDN. Someone was repeatedly downloading the same files. It turns out the Facebook crawler was downloading every file in the directory listing every two hours.
Time to say good bye, crawler. (blocked at 19.30PM)
Published at
2024-06-07 21:32:04Event JSON
{
"id": "ffccce60ddac5dc2e398be2ff7906f48d3df80b9d9e8082d6214884ee0e23015",
"pubkey": "03bc75ee90d5b22f472da64e5491e88cfc17dfe8ed200eeef1326a71496bf71c",
"created_at": 1717788724,
"kind": 1,
"tags": [
[
"t",
"meta"
],
[
"t",
"facebook"
],
[
"proxy",
"https://gyptazy.ch/snac/gyptazy/p/1717788724.181831",
"activitypub"
]
],
"content": "Is the crawler from #META / #Facebook acting up again? My traffic monitoring system alerted me to some unusual activity on my CDN. Someone was repeatedly downloading the same files. It turns out the Facebook crawler was downloading every file in the directory listing every two hours.\n\nTime to say good bye, crawler. (blocked at 19.30PM)\n\n\nhttps://gyptazy.ch/snac/gyptazy/s/cbaf025ddc8e7442e1cf2767403e16d8.jpg",
"sig": "163ac6ffc4b45af773186cac1caeb487f63253e3cdb42af42818d53137714da7cf79db8799e01fbd9f4a2ddbe5904701b8ec863f01962a6db257eec84426b8b7"
}
