Blog · page 15 of 17
Writing
203 posts RSS
No sponsorships, no affiliate links, no paid placements. Editorial policy.
- Best Semantic Caching Tools for LLM Apps Semantic caching trades correctness for cost, and the failure is a false hit rather than a miss. The failure classes, the tenancy leak, and seven tools compared.
- Best LLM Evaluation Tools Offline gates and online scoring answer different questions, and LLM-as-judge is unreliable until you calibrate it. Ten evaluation tools, and how to measure the judge.
- Best Prompt Management Platforms Moving prompts out of the repo turns them into a runtime dependency that can change under a shipped binary. The trade-offs, the fetch design, and eight tools.
- Best LLM Cost Tracking and FinOps Tools A provider invoice tells you the total and nothing else. Why per-customer LLM attribution is a call-site tagging problem, and the eight tools that can and cannot solve it.
- Best LLM Observability Tools An LLM trace carries the full prompt, the completion and every tool call, which makes it your most sensitive store. Ten tools compared on capture, redaction and retention.
- LiteLLM Alternatives Four reasons teams move off a self-hosted LiteLLM proxy, each pointing at a different shortlist — plus what LiteLLM still does better than every alternative here.
- Portkey Alternatives Teams leave Portkey for four different reasons, and each one points at a different shortlist. Segment by your reason before you evaluate a single replacement.
- Best Self-Hosted LLM Gateways Self-hosting an LLM gateway means a proxy tier, Redis, and a Postgres full of prompt payloads — plus being in the request path for every AI feature you ship.
- LiteLLM vs Portkey vs Kong AI Gateway Three different architectural bets, not three feature sets: a library that became a proxy, a managed gateway inside a security platform, and LLM traffic as another protocol.
- Best AI Gateways for Enterprise The controls procurement and security block on — redaction, audit, hard budgets, SSO, model allow-lists, deployment mode — and the gateways that satisfy them.
- Best Open Source AI Gateways How to read an open-core boundary before you commit, why licence shape matters more than the word open, and which gateways survive both tests.
- Best AI Gateways for Startups How fast you get to a first routed request, whether swapping a base URL is genuinely enough, and the honest case for not adopting a gateway yet at all.