> (…) split the index, which is to say the part of Google Search that scrapes the web and makes that content searchable, from the search user interface, and manage that index as a public utility that different search services could rely on and pay for, an idea that was suggested in a recent paper.
That would be incredible. There’s also an efficient-resourcing argument to be made here, because many sites are hit hard by swarms of crawlers, all collecting the same data.
https://mastodon.social/@robin/112933151964297678
Erlend /
npub14c…z2e8a
2024-08-10 13:06:16
Author Public Key
npub14ch0zfh9cwswwgp65f7jxrk3yzu0z5xk6rueaf3cchrvreqnfzys6z2e8aSeen on
wss://relay.noswhere.comPublished at
2024-08-10 13:06:16Event JSON
{
"id": "21338349e530661e57a30a92b353b0aaf424f8d42c5cb2951279e3454f76fef6",
"pubkey": "ae2ef126e5c3a0e7203aa27d230ed120b8f150d6d0f99ea638c5c6c1e4134889",
"created_at": 1723287976,
"kind": 1,
"tags": [
[
"proxy",
"https://writing.exchange/users/erlend/statuses/112937400837773437",
"activitypub"
]
],
"content": "\u003e (…) split the index, which is to say the part of Google Search that scrapes the web and makes that content searchable, from the search user interface, and manage that index as a public utility that different search services could rely on and pay for, an idea that was suggested in a recent paper.\n\nThat would be incredible. There’s also an efficient-resourcing argument to be made here, because many sites are hit hard by swarms of crawlers, all collecting the same data.\nhttps://mastodon.social/@robin/112933151964297678",
"sig": "5bc2cb8ac8e638eb62511789f1ce7289f987b6e331e19deb15a25406ff639a67243aaee7ce8f29e3a2b966529e1e834dbfb033c50079f2b7a105067af8a09368"
}