Vector databases and inference platforms
The retrieval layer for RAG and the execution layer that actually runs the model. What separates these is rarely performance — it is what the free tier lets you prove, whether it can run on your own infrastructure, and what the meter counts.
Published · updated
How we ranked these
Stated before the list, so you can disagree with the weighting rather than the result.
The ranking
- 01
Pinecone4.5$20/mo
The default managed vector database for RAG, with serverless indexes plus hosted embedding and reranking.
The fastest route to production RAG when self-hosting is not a requirement.
- 02
Qdrant4.5Free
An open-source vector search engine you can run managed, in your own cloud, or entirely self-hosted.
The right default where data residency matters or where lock-in is the thing you are avoiding.
- 03
Weaviate4.4From $45/mo
An open-source vector database offered as a managed service, with embedding generation and a query agent on the same platform.
Choose it when you want retrieval as one system rather than three. Pinecone is simpler if you only need vectors.
- 04
Chroma4.4Free
Open-source search infrastructure for AI, covering vector, full-text and metadata search in one engine.
The right first step for any retrieval project. Plan the migration once the corpus is large.
- 05
Modal4.6従量課金
Run Python serverlessly with per-second GPU billing and no charge for idle time.
Excellent for bursty GPU and batch work in Python. Measure cold starts before putting it behind a user.
- 06
Baseten4.4従量課金
Production inference for custom, fine-tuned and open-source models, billed by compute time rather than uptime.
The right call when you have a model and no infrastructure team. Revisit at sustained high volume.
- 07
Together AI4.4従量課金
Inference and GPU clusters for open-source models, available serverless or dedicated.
A good home for open-model workloads at scale. Start serverless, move to clusters when the maths flips.
- 08
Fireworks AI4.4従量課金
Fast inference and fine-tuning for open models, with dedicated GPUs rented by the hour.
Good for fine-tuning experiments. Compare GPU-hour rates against Together before committing to dedicated.
- 09
Groq4.4従量課金
An inference cloud that competes on speed: the same models, answered faster.
The right choice when a human is waiting for the response. Not a general-purpose replacement.
- 10
Replicate4.5従量課金
Run open-source models through an API, billed by run time or hardware seconds, and deploy your own with Cog.
Excellent for getting an open model into production quickly. Revisit the maths once volume is steady.
- 11
OpenRouter4.6従量課金
One OpenAI-compatible API in front of hundreds of models, with no markup on the provider's own price.
The default way to start when you do not yet know which model you need. Move to a direct provider integration once you do.
Questions we get
- Which vector database should we pick?
- The technical gap has narrowed, so choose on deployment shape. Pinecone to be running fastest, Qdrant if self-hosting is a possibility, Chroma if you want full-text search in the same engine.
- Rented GPUs or an API?
- It inverts with volume. At low volume the API is dramatically cheaper; under sustained heavy load dedicated GPUs win. Together AI lists preemptible H100s at $1.99 per GPU hour, which matters for interruptible batch work.
- Is a router like OpenRouter worth the hop?
- It makes switching models easy and gives you one key across vendors. It states no markup over provider pricing, so there is no cost penalty.