Old enterprise archives may be useful training data for LLMs

Public internet data for LLM training is becoming harder to use as a fresh source. Many teams rely on the same public sources, so their training data overlaps heavily. Private domain-specific datasets may matter more in fields such as financial services, healthcare, and oil and gas, where proprietary data carries important expert knowledge.

Large amounts of enterprise data from the 1980s through the 2000s may still exist on physical tape and may never have been digitized. Because this material is less likely to overlap with scraped web data, it could become useful for building specialized models or agents for specific industries.