Experimental torrents for downloading open AI models

is a repository and website for sharing torrent files for popular open models. People can download model files through BitTorrent, and when no peers are available, is used as a fallback web seed. A small backend service routes BitTorrent client requests to the right file location.

This extra routing is needed because some files are stored with LFS and some are not. The system is still . CDN errors can happen for some files, but downloads usually work after a few retries.

The next goal is to automate torrent creation for new models and publishing through . GitHub’s free runners only provide about 100GB of disk space, so models larger than 100GB need another approach.

Key points

  • shares torrent files for popular open models.
  • can act as a fallback web seed when no torrent peers are available.
  • A backend service redirects torrent client requests to the correct file endpoint.
  • The project is , and some file downloads may need retries.
  • free runners are too small for easy automation of models over 100GB.
Read original