Local model choice: bigger Q4 or smaller Q8

On a 32 GB machine, -A3B at Q4 runs at about 15 . is being considered for work against a codebase.

Because is smaller, it should fit comfortably at Q8, and BF16 might even be possible. The practical question is whether a larger, more model or a smaller, less model gives the better mix of quality, speed, and memory use.

Key points

  • -A3B at Q4 reaches about 15 on 32 GB .
  • is small enough to fit comfortably at Q8 on the same setup.
  • BF16 may also be possible for because of its smaller size.
  • The main tradeoff is larger and more versus smaller and less .
Read original