A plan to train a tiny language model on free Colab
The goal is to build a for limited tasks such as grammar, , and math. The model is meant to stay small, using nanoGPT_75M as the starting point. After that, it would be trained further on personal project data.
The planned setup is the free version of . There are no results, cost numbers, or working agent examples yet; the concrete need is advice on and choices.
Key points
- The target tasks are grammar, , and math.
- nanoGPT_75M is the proposed starting point.
- The model would later be trained on personal project data.
- The available compute is the free version of .
- No , cost saving, or agent result is provided yet.