LLM API costs have a way of surprising you at scale. A feature that costs pennies in development can run up thousands in production if you're not thinking about model selection, token efficiency, caching, and gateway architecture from the start. These skills cover the operational side of running LLM features: cost modeling, observability, streaming UX, and the infrastructure decisions that keep your AI features economically viable.
Key takeaway
ClaudeVault's LLM operations and cost skills turn the 'why is our API bill so high' meeting into a structured optimization pass — prompt caching that cuts input tokens up to 90%, model routing that picks the cheapest model that meets the quality bar, streaming UX that makes responses feel 40% faster, and observability that catches hallucinations before they become tickets. Strategic optimization routinely cuts LLM costs 60-80% without quality loss.
When an API bill grew 5x after launch and the team cannot tell which endpoint, model, or prompt is driving the cost
When a streaming response feels slow even though the total latency matches a competitor's
When the team is choosing between Claude, GPT, and Gemini for a new feature and wants a defensible decision
When a single failing model call takes down a user session and there is no fallback path to a cheaper or different provider
A full Claude Code LLM ops pass starts with measurement, picks the right model for each call, cuts the tokens the feature does not need, routes traffic through a gateway, and then watches everything in production.
Start with the AI cost calculator. Claude reads the current usage, breaks cost down per endpoint, per model, and per token type, and surfaces the specific calls driving the bill. Optimization without a baseline usually cuts the wrong thing and leaves the 400ms endpoint untouched.
The model selector runs the target workload against candidate models — Claude 4.5, GPT-5, Gemini 3 Pro, Haiku, Flash — and reports quality versus cost per call. Claude picks based on measured trade-offs, not the marketing benchmark the vendor runs on its favorite eval.
Hand the prompts to the token optimizer. Claude turns on prompt caching, trims system prompts, constrains output length, and moves few-shot examples into cached prefixes. On long workloads, these four levers typically cut input token cost by 60-80% without touching quality.
The LLM gateway architect configures LiteLLM or Portkey with routing rules, failover policies, and usage quotas so a single model outage does not kill the feature. Claude writes the routing table as a rule set the team can read and argue about, not as hidden config buried inside the gateway UI.
Finally, the AI observability designer wires up Langfuse, Helicone, or Datadog LLM Observability with dashboards for TTFT, token counts, cost per request, and hallucination rate. Claude sets alert thresholds tied to the cost baseline so a creeping regression pages the right person before the invoice does.
Outcome
A measured cost baseline, a model chosen on quality-versus-cost data, trimmed prompts with caching turned on, routing and fallback through an LLM gateway, and production observability with alerts tied to real thresholds.
| Skill | Best for | Complexity | Primary use case |
|---|---|---|---|
| AI Cost Calculator | Teams without a clear cost baseline | Beginner | Per-endpoint, per-model cost breakdown |
| Model Selector | Feature launches with multiple candidate models | Intermediate | Quality vs cost benchmarking on real workload |
| Token Optimizer | Prompts with high input token cost | Intermediate | Caching, prompt trimming, and output limits |
| LLM Gateway Architect | Production systems with more than one provider | Advanced | LiteLLM and Portkey routing and failover design |
| AI Observability Designer | Production LLM systems without dashboards | Intermediate | TTFT, token, cost, and hallucination metrics |
| Streaming UX Designer | Chat and completion interfaces feeling slow | Intermediate | SSE streaming and perceived-latency patterns |
Designs monitoring and observability for LLM-powered systems — request logging, quality metrics, cost dashboards, latency tracking, and alerting with anti-noise rules. Use when building AI monitoring, tracking prompt version impact, detecting quality degradation, or designing AI-specific dashboards. AI monitoring, LLM observability, AI ops.
Reduces LLM token usage and API costs through prompt compression, context management, caching strategies, and model routing. Use when API costs are too high, context limits are being hit, or token budgets need optimization. Token reduction, cost optimization, prompt compression.
Designs LLM API gateway infrastructure — provider abstraction, failover chains, rate limit management, response caching, and request routing. Use when building multi-provider resilience, managing API key pools, or abstracting LLM provider differences. Gateway, failover, load balancing, API proxy.
Estimates and optimizes LLM API costs with real-world multipliers — system prompt overhead, retry rates, caching ROI, model tier allocation, and scaling projections. Use when budgeting AI features, justifying costs to leadership, or diagnosing unexpected API bills. Cost estimation, token budget, AI spend, pricing.
Recommends the right LLM model for a task based on a 5-dimension capability scoring matrix, with multi-model architecture patterns (router, cascade, draft+refine). Use when choosing between model tiers, designing model routing, or optimizing cost/quality tradeoffs. Model selection, model comparison, which model.
Designs streaming AI interfaces — progressive rendering, markdown buffering, code block detection, cancellation handling, tool call indicators, and error recovery mid-stream. Use when building token-by-token chat UIs, streaming LLM responses, or handling partial content rendering. Streaming, SSE, real-time AI, chat UI.
Anthropic's explicit prompt caching can cut input token costs up to 90% and latency up to 85% on long prompts. Real cases in the community report API spend dropping from roughly $720 per month to $72 per month after the token optimizer walked through the caching setup. The lever sits on input tokens — output tokens are untouched.
Yes for any multi-model production system. A gateway like LiteLLM or Portkey centralizes routing, fallback, usage quotas, and observability. Without one, every provider outage becomes a feature outage, and cost attribution per team or feature becomes guesswork the finance team cannot trust at renewal time.
It depends on the workload. Claude 4.5 leads on long coding sessions and tool use reliability. Gemini 3 Pro leads on multimodal and very long context. GPT-5 is the balanced default when no single requirement dominates. The model selector skill runs the target workload through all three and reports quality-per-dollar so the decision has data behind it.
Users perceive streaming as roughly 40% faster even when total latency is identical, because the first few tokens arrive in 200-400 milliseconds instead of after the full 5-second generation completes. The streaming UX designer skill handles the SSE wiring and the skeleton-state UX that makes the perceived-latency win stick.
Track time-to-first-token, input and output token counts per request, cost per request, quality scores from an LLM-as-judge eval, and hallucination rate sampled against a golden set. The AI observability designer wires these five metrics into Langfuse or Datadog with alerts tied to the cost baseline so regressions page someone.
Four levers usually get there: turn on prompt caching for repeatable prefixes, route cheaper models for low-stakes calls, trim output length constraints, and cut the system prompt of filler. Output tokens cost 3-5x input tokens, so aggressive output limits are usually the highest-leverage lever. The token optimizer skill walks all four in sequence.