Andrej Karpathy / @karpathy (RSS Feed) / is on Nostr

Twitter feed for: @karpathy. Generated by nitter.moomoo.me

https://nitter.moomoo.me/karpathy

Last Notes

2023-05-02 18:38:39

Excellent TED talk from Sal Khan:
\- many inspiring examples of GPTs finetuned into socratic tutors, assisting without giving away answers.
\- none of it "out of the box", requires prompt engineering, finetuning, data collection, iteration.
\- sense of barely scratching the surface.
nitter.moomoo.me/salkhanacademy/status/1653070197056585732#m (https://nitter.moomoo.me/salkhanacademy/status/1653070197056585732#m)
https://nitter.moomoo.me/karpathy/status/1653438865880023041#m

2023-04-27 20:48:31

FaceTime with ChatGPT callannie.ai (https://callannie.ai)
Fun, qualitatively different experience over "texting"
Ty @CallAnnieAI (https://nitter.moomoo.me/CallAnnieAI) for shoutout to my tweet a ~year ago pitching this as an idea nitter.moomoo.me/CallAnnieAI/stat… (https://nitter.moomoo.me/CallAnnieAI/status/1651379993413558272)
With improvements to personality, latency, ASR/TTS could be magical ✨
nitter.moomoo.me/frantzfries/status/1651316031762071553#m (https://nitter.moomoo.me/frantzfries/status/1651316031762071553#m)
https://nitter.moomoo.me/karpathy/status/1651659606844928000#m

2023-04-02 21:30:16

**Response to @karpathy:**
All of that is just one agent/thread. People coalesce into organizations so they can specialize and parallelize work towards shared goals. Imo this is likely to happen to AutoGPTs and for the same reasons, strung into AutoOrgs, with AutoCEO, AutoCFO, AutoICs, etc.
https://nitter.moomoo.me/karpathy/status/1642610417779490816#m

2023-04-02 21:19:09

**Response to @karpathy:**
1 GPT call is a bit like 1 thought. Stringing them together in loops creates agents that can perceive, think, and act, their goals defined in English in prompts.
For feedback / learning, one path is to have a "reflect" phase that evaluates outcomes, saves rollouts to memory,… nitter.moomoo.me/i/web/status/164… (https://nitter.moomoo.me/i/web/status/1642607620673634304)
https://nitter.moomoo.me/karpathy/status/1642607620673634304#m

2023-03-26 19:26:45

Good example of us not seeing max GPT-4 capability yet, imo. Prompt design, tool use, meta cognition strategies (eg idea of attempt, critique, retry, capabilities model, etc) are very likely to go a long way.
nitter.moomoo.me/ericjang11/status/1639882111338573824#m (https://nitter.moomoo.me/ericjang11/status/1639882111338573824#m)
https://nitter.moomoo.me/karpathy/status/1640042620666920960#m

2023-03-24 01:45:22

"How to chat with a 56-page PDF"
Good developer-focused YouTube explainer: piped.moomoo.me/watch?v=ih9PBGVV… (https://piped.moomoo.me/watch?v=ih9PBGVVOO4)
Very excited about the growing layer of software infrastructure on top of GPT APIs, and all of the possible extensions here.
nitter.moomoo.me/mayowaoshin/status/1636165883876745217#m (https://nitter.moomoo.me/mayowaoshin/status/1636165883876745217#m)
https://nitter.moomoo.me/pic/card_img%2F1636565967445581824%2FJoiWgKGG%3Fformat%3Djpg%26name%3D800x320_1
https://nitter.moomoo.me/karpathy/status/1639065836815273984#m

2023-03-23 20:16:20

GPT is a new kind of computer architecture that runs on text. Yes it can talk to us, but also to much of our existing software infrastructure. First via apps on top of APIs, now inside ChatGPT via plugins.
What a time right now...
openai.com/blog/chatgpt-plug… (https://openai.com/blog/chatgpt-plugins)
https://nitter.moomoo.me/karpathy/status/1638983034522460162#m

2023-03-16 21:07:42

Less publicized but highly awesome aspect of GPT-4 launch was that OpenAI open sourced an evals framework, allowing us to crowdsource model evaluations at scale 📈. The repo is getting some very high quality PRs (rewarded with GPT-4 access).
I <3 evals; \`pip install evals\`
nitter.moomoo.me/kondrich2/status/1635687409668227072#m (https://nitter.moomoo.me/kondrich2/status/1635687409668227072#m)
https://nitter.moomoo.me/karpathy/status/1636459245184106497#m

2023-03-14 22:05:51

The GPT-4 developer livestream (piped.moomoo.me/watch?v=outcGtbn… (https://piped.moomoo.me/watch?v=outcGtbnMuQ)) was a great preview of new capability.
Not sure I can think of a time where there was this much unexplored territory with this much new capability in the hands of this many users/developers.
nitter.moomoo.me/bentossell/status/1635737961747677189#m (https://nitter.moomoo.me/bentossell/status/1635737961747677189#m)
https://nitter.moomoo.me/pic/card_img%2F1635688966144811008%2FF34E_VAj%3Fformat%3Djpg%26name%3D800x320_1
https://nitter.moomoo.me/karpathy/status/1635749104059056128#m

2023-03-12 23:46:03

Dropout layers in a Transformer leak the phase bit (train/eval) - small example. So an LLM may be able to determine if it is being trained and if backward pass follows. Clear intuitively but good to see, and interesting to think through repercussions of
colab.research.google.com/dr… (https://colab.research.google.com/drive/1286r553N8drh6-VeZjZA1vbUBY9Z1fps?usp=sharing)
https://nitter.moomoo.me/pic/card_img%2F1634876168636874752%2FCaT1snPL%3Fformat%3Dpng%26name%3D280x280_2
https://nitter.moomoo.me/karpathy/status/1635049541534879745#m

2023-03-12 17:31:08

File reading under the "horror" genre.
reality vs expectation
nitter.moomoo.me/bilalmahmood/status/1634595490968006661#m (https://nitter.moomoo.me/bilalmahmood/status/1634595490968006661#m)
https://nitter.moomoo.me/pic/media%2FFrCGsSMaYAIEH2l.jpg
https://nitter.moomoo.me/karpathy/status/1634955190964219905#m

2023-03-09 17:55:16

"The hot mess theory of AI misalignment"
a favorite talk from a recent alignment workshop turned article; offers a unique and imo fairly realistic framework for superintelligent system futures that departs from your stock paperclip maximizers.
nitter.moomoo.me/jaschasd/status/1633864184697081858#m (https://nitter.moomoo.me/jaschasd/status/1633864184697081858#m)
https://nitter.moomoo.me/karpathy/status/1633874103672406017#m

2023-03-01 19:00:29

ControlNet is 🔥 github.com/lllyasviel/Contro…
Allows for very fine control over stable diffusion process, has taken over r/stablediffusion and friends
nitter.moomoo.me/bhutanisanyam1/status/1630980866775330819#m
https://nitter.moomoo.me/karpathy/status/1630991410387378176#m