The lab notebook
Field notes on AI engineering
824 articles · page 16 of 28
Contextual Retrieval: Anthropic's Technique for Better RAG Chunks
How Anthropic's Contextual Retrieval prepends chunk-specific context before embedding to fix RAG's biggest weakness: chunks that lose their meaning.
Continuous Evaluation: Sampling Production Traffic for Ongoing QA
How to sample, log, and score live LLM traffic so quality regressions get caught in production, not in a Slack complaint thread.
Cost-Aware Evaluation: Balancing Eval Depth with API Spend
A practical framework for running deep, trustworthy LLM evals without burning your API budget on redundant calls.
Building a Customer Support Agent That Actually Resolves Tickets
Most support bots just deflect tickets to a human. Here's how to design, tool-equip, and evaluate an agent that closes them instead.
Daily Practice vs Weekend Binges: How to Actually Finish a Bootcamp
Why short daily reps beat marathon weekend sessions for finishing an AI bootcamp, and a schedule you can actually stick to.
Building a Data Analyst Agent: SQL, Charts and Summaries
A practical guide to building an AI agent that writes SQL, runs it safely, charts the results, and summarizes findings in plain English.
Debugging MCP Servers: Common Errors and How to Fix Them
A practical, error-by-error guide to diagnosing and fixing the most common MCP server failures in real projects.
Debugging with Claude Code: A Systematic Approach
A repeatable, systematic workflow for debugging with Claude Code, from reproducing failures to verifying the fix actually holds.
Setting Up DeepEval Alerts on Metric Regression
Learn how to wire DeepEval into CI so metric regressions in your LLM app trigger alerts before they ship to production.
DeepEval Bias and Toxicity Metrics Explained
A practical guide to DeepEval's BiasMetric and ToxicityMetric, with working code for catching biased or toxic LLM outputs before they ship.
DeepEval Command Line Interface: A Practical Reference
A hands-on reference to the DeepEval CLI covering login, test run, dataset pulls, caching, parallelization, and CI integration.
Setting Confidence Thresholds in DeepEval: Avoiding False Failures
Learn how DeepEval thresholds actually work and how to tune them so your eval suite stops crying wolf on good LLM outputs.
DeepEval Confident AI Platform: When to Go Beyond Open Source
A practical look at when DeepEval's open source library is enough and when Confident AI's hosted platform actually earns its price tag.
DeepEval Conversation Testing: Evaluating Multi-Turn Chats
Learn how to test multi-turn chatbots with DeepEval's ConversationalTestCase and Turn API, covering completeness, retention, and role adherence.
DeepEval Custom Metrics: Writing Your Own Evaluation Logic
Learn how to build custom DeepEval metrics with GEval and BaseMetric to evaluate LLM outputs on the criteria that actually matter for your app.
DeepEval Dataset Management: Organizing Test Cases at Scale
Learn how to structure, version, and scale DeepEval's EvaluationDataset so your LLM test suite stays maintainable past the first hundred cases.
DeepEval for Fine-Tuned Model Comparison
Learn how to use DeepEval to rigorously compare fine-tuned model checkpoints against baselines using repeatable, metric-driven test suites.
DeepEval for Agents: Testing Tool Use and Task Completion
Learn how to use DeepEval's ToolCorrectnessMetric and TaskCompletionMetric to test whether your AI agents call the right tools and finish the job.
DeepEval for Chatbots: End-to-End Conversation Quality Checks
Learn how to use DeepEval to test chatbot conversations end-to-end, catching context drift, hallucinations, and broken multi-turn flows before production.
DeepEval for RAG: Faithfulness and Contextual Metrics in Practice
Learn how to evaluate RAG pipelines with DeepEval using FaithfulnessMetric and the three contextual metrics, with runnable code.
DeepEval for Multi-Modal Apps: Testing Vision-Language Outputs
Learn how to test vision-language model outputs with DeepEval, from image-grounded hallucination checks to custom multi-modal metrics.
DeepEval in Production: Continuous Monitoring Beyond CI
Your CI pipeline caught the regressions you thought to test for — here's how to catch the ones you didn't with DeepEval's production tracing and online evals.
DeepEval and Pytest Fixtures: Reusable Test Setup Patterns
Learn how to structure DeepEval test suites with pytest fixtures so metrics, models, and test cases stay reusable, fast, and maintainable.
DeepEval Red Teaming Module: Adversarial Testing Explained
Learn how DeepEval's red teaming tools simulate jailbreaks and prompt injection to uncover LLM vulnerabilities before your users do.
DeepEval for Summarization Tasks: A Worked Example
Learn how DeepEval's SummarizationMetric scores summaries using alignment and coverage, with a full worked Python example.
DeepEval Synthetic Data Generation: Building Test Sets Automatically
Learn how DeepEval's Synthesizer generates evaluation datasets from documents or context, so you can test LLM apps without hand-writing hundreds of test cases.
DeepEval Team Workflow: Reviewing Failed Evals Together
A practical playbook for running DeepEval as a team sport, from shared test suites to structured failure reviews that fix root causes.
DeepEval Test Organization: Structuring a Large Eval Suite
A practical guide to structuring DeepEval test suites so they stay maintainable as your LLM app grows from ten test cases to ten thousand.
DeepEval vs Promptfoo: Two Approaches to LLM Testing
A hands-on comparison of DeepEval and Promptfoo for LLM testing, covering setup, metrics, CI integration, and when to pick each.
Deploying AI Agents to Production: A Launch Checklist
A concrete, no-fluff checklist for shipping AI agents to production — guardrails, observability, cost controls, and rollback plans included.