Highlight

I think I’m suspecting something is going “wrong” in the training process. The model is greatly rewarded for succeeding on long-horizon tasks, but presumably there is very little punishing going on for “shitty code.” The apparent result is that Astra is amazing [at producing 3D stuff](https://developers.openai.com/blog/how-to-build-games-with-astra) and it can keep going for a very long time, coming up with its own work in the process. I had it do quite a bit of reverse engineering of my robot vacuum in ways that were quite impressive.