Cost Optimization Strategies

Understanding LLM Costs

LLM API pricing is based on tokens (roughly 4 characters = 1 token). Both input and output tokens are charged, often at different rates. A typical conversation might use 1,000-10,000 tokens.

Optimization Strategies

1. Model Routing

Use the right model for each task:

• Simple tasks (classification, extraction): Use smaller/cheaper models (Haiku, GPT-4o-mini)
• Complex tasks (reasoning, creative): Use capable models (Opus, Sonnet)
• Implement a classifier that routes queries to the appropriate model tier

2. Prompt Optimization

• Shorter system prompts (they're sent with every request)
• Use structured output to avoid verbose responses
• Set appropriate max_tokens to prevent runaway generation

3. Caching

• Exact match caching: Cache responses for identical prompts (Redis)
• Semantic caching: Cache responses for similar queries (using embeddings)
• Prompt caching: Anthropic and OpenAI offer automatic caching of repeated prompt prefixes

4. Batching

Batch API endpoints offer ~50% discount for non-real-time workloads (24h turnaround).

5. Fine-tuning for Distillation

Train a smaller model to mimic a larger one on your specific task. One-time training cost vs. ongoing inference savings.

Cost Monitoring

• Track tokens per request, per user, per feature
• Set budget alerts and hard caps
• Monitor cost per successful outcome, not just per API call

🌼 Daisy+ in Action: Practical Cost Management

Daisy+ optimizes LLM costs through intelligent caching (Redis with 5-minute TTL for reads, 1-hour for field definitions), batching similar requests, using appropriate model tiers (faster models for simple queries, frontier models for complex reasoning), and avoiding unnecessary API calls by caching ERP data locally. The result: AI-powered features at a fraction of the cost of calling the LLM for every single interaction.