Program like a team member

With some huge repositor(y|ies), train a model using the sequence of commits to learn "how we got to now", "what mistakes we had to fix and how we fixed them", "how we are incrementally closing in on our bigger goal", ... that kind of stuff.

The more resolution the better (i.e., ideally we would also commit our mistakes).

In any case, use a repo's "time" dimension to do more stuff.

Write an implementation of a language

Given, for a single programming language (say "C")...

1. Zillions of small, self-contained projects
2. Their resulting binaries, output by the usual compiler
3. An executable environment that the model has "control" of

It should be possible (with enough of the above) to get an AI to write a compiler for us.

But I bet it could go even farther than that. Since the model also has access to endless bugs, by necessity, and since it will find these bugs through its prodding, I bet that it could add language features like, "this pattern results in a null pointer", "this pattern will access beyond the end of the list"... that kind of thing.

So maybe it could also extend the language with syntax sugar for clearly and explicitly reproducing these bugs. (and the like)

Which may point to improvements humans could make to the language/compiler.