Choosing a local Hermes Agent model for an RTX 4070 PC
The planned PC has a Ryzen 7 5700X processor, 64GB of , an Nvidia RTX 4070 with 12GB of , and a 2TB SSD. It can currently run models with up to 12 billion in LM Studio using default settings.
The goal is to adjust the settings so can still produce long outputs, but no tested or benchmark is available. The required work includes reasoning, image recognition, , difficult problem solving, deep web research, organizing research data, and occasional local coding.
The quality target is roughly 80–85% of Sonnet. A specific model and suitable LM Studio settings for this hardware have not yet been identified.
Key points
- The PC has a Ryzen 7 5700X, 64GB of , an Nvidia RTX 4070 with 12GB, and a 2TB SSD.
- LM Studio currently runs models containing up to 12 billion with default settings.
- The desired setup would run a larger model while continuing to produce long outputs.
- Required abilities include image and speech handling, web research, data organization, reasoning, and complex problem solving.
- The target is local coding support and results around 80–85% as capable as Sonnet.