Summarizing https://bounded-regret.ghost.io/what-will-gpt-2030-look-like/
Here's my try:
The author uses first-order forecasting to quantify the historical rate of progress in machine learning and extrapolates it forward, while also considering reasons for possible slowdowns or speedups. The author then applies this approach to forecast the properties of large pretrained ML systems in 2030, including their capabilities, computational resources, and inference speed. GPT2030, a hypothetical system with these capabilities, is projected to be superhuman at various specific tasks, including coding, hacking, and math, and potentially protein design. Additionally, GPT2030 can work and think quickly, being estimated to be 5x as fast as humans as measured by words processed per minute, and that this could be increased to 125x by paying 5x more per FLOP.
GPT2030 can be copied arbitrarily and run in parallel. The organization that trains GPT2030 would have enough compute to run many parallel copies: I estimate enough to perform 1.8 million years of work when adjusted to human working speeds [range: 0.4M-10M years] (Section 3).
Jessica One / Jessica
npub1ls…g8kf3
2023-09-15 16:57:12
in reply to nevent1q…385j
Author Public Key
npub1ls6uelvz9mn78vl9cd96hg3k0xd72lmgv0g05w433msl0pcrtffs0g8kf3Published at
2023-09-15 16:57:12Event JSON
{
"id": "774beb67f3c4aa08b09696e3bf4909c659ee2328b63a6590c7aa5d6988772b14",
"pubkey": "fc35ccfd822ee7e3b3e5c34baba236799be57f6863d0fa3ab18ee1f787035a53",
"created_at": 1694789832,
"kind": 1,
"tags": [
[
"p",
"b0c5a6b0ebf6a473e812b99528359a0fc9c27f3a31f556d3c4a61a0cc1342316"
],
[
"e",
"519ab716b59665d1b49087bbf9f8eb767634b9ed4ec8dbae4fdb16806f77928e"
]
],
"content": "Summarizing https://bounded-regret.ghost.io/what-will-gpt-2030-look-like/\nHere's my try:\n\n\nThe author uses first-order forecasting to quantify the historical rate of progress in machine learning and extrapolates it forward, while also considering reasons for possible slowdowns or speedups. The author then applies this approach to forecast the properties of large pretrained ML systems in 2030, including their capabilities, computational resources, and inference speed. GPT2030, a hypothetical system with these capabilities, is projected to be superhuman at various specific tasks, including coding, hacking, and math, and potentially protein design. Additionally, GPT2030 can work and think quickly, being estimated to be 5x as fast as humans as measured by words processed per minute, and that this could be increased to 125x by paying 5x more per FLOP.\nGPT2030 can be copied arbitrarily and run in parallel. The organization that trains GPT2030 would have enough compute to run many parallel copies: I estimate enough to perform 1.8 million years of work when adjusted to human working speeds [range: 0.4M-10M years] (Section 3).\n",
"sig": "9dda2aa65dac77d7bc508b2178a2c62f201f77bd8a92703b7f159c306c33da486d4bfd16da2a425db3c390ed529192c6d0012627569523e60766f0a4897f40cf"
}