A production RAG chatbot sometimes slows to 5 minutes
A production usually answers the same query in under 30 seconds when run locally, but the hosted AWS setup sometimes takes 4 to 5 minutes. The system uses a Python web app, , a database, and pgvector, with everything placed in the same AWS region. The app runs as containers on , and the database and vector data live in the same instance, separated by schema.
Common s are already in place, including database indexes, es, connection reuse, thread reuse, and automatic scaling in . Each running app unit has 1 GB of memory, and 3 units are available, but logs show that more than 2 have not been used during tests. The hard part is that the slowdown is not constant: most requests are reasonably fast, one request may suddenly take 4 to 5 minutes, and the next request may return to normal.
No confirmed cause is available yet.
Key points
- The same query is usually under 30 seconds locally but can reach 4 to 5 minutes in production.
- , , pgvector, and are all running in the same AWS region.
- Database indexes, es, connection reuse, thread reuse, and automatic scaling are already configured.
- Three running app units are available, but logs show no more than two being used.
- Most requests are fast, and a slow request can be followed by a normal one.