Dropout layers in a Transformer leak the phase bit (train/eval) - small example. So an LLM may be able to determine if it is being trained and if backward pass follows. Clear intuitively but good to see, and interesting to think through repercussions of
colab.research.google.com/dr… (https://colab.research.google.com/drive/1286r553N8drh6-VeZjZA1vbUBY9Z1fps?usp=sharing)
https://nitter.moomoo.me/pic/card_img%2F1634876168636874752%2FCaT1snPL%3Fformat%3Dpng%26name%3D280x280_2
https://nitter.moomoo.me/karpathy/status/1635049541534879745#m
Andrej Karpathy / @karpathy (RSS Feed) /
npub1rj…2ffmk
2023-03-12 23:46:03
Author Public Key
npub1rj7u39tvjdgfpzg3c3xfym6vzalt34p7t5uvdsqhzgst9jtl7dgqs2ffmkPublished at
2023-03-12 23:46:03Event JSON
{
"id": "d42c11356e2095f9c2e6609e24a565567836fd715fd7950715bfd6f2829a06ed",
"pubkey": "1cbdc8956c9350908911c44c926f4c177eb8d43e5d38c6c0171220b2c97ff350",
"created_at": 1678661163,
"kind": 1,
"tags": [],
"content": "Dropout layers in a Transformer leak the phase bit (train/eval) - small example. So an LLM may be able to determine if it is being trained and if backward pass follows. Clear intuitively but good to see, and interesting to think through repercussions of\n\ncolab.research.google.com/dr… (https://colab.research.google.com/drive/1286r553N8drh6-VeZjZA1vbUBY9Z1fps?usp=sharing)\n\nhttps://nitter.moomoo.me/pic/card_img%2F1634876168636874752%2FCaT1snPL%3Fformat%3Dpng%26name%3D280x280_2\n\nhttps://nitter.moomoo.me/karpathy/status/1635049541534879745#m",
"sig": "0caa25420bfce6a054297866e69de6ac809527d052a0cfcfdf4d675de728092716c2401aa18a9edffd2fba2b7eaf2b65fd277863f97c0c221a1d0c116c4c4e97"
}