The lab notebook
Field notes on AI engineering
824 articles · page 21 of 28
Ragas Cost Considerations: Managing LLM Calls During Evaluation
Ragas evaluations quietly multiply LLM calls per sample — here's how to estimate, control, and cut evaluation costs without losing signal.
Ragas Custom Metrics: Extending the Framework for Your Use Case
Learn how to build custom Ragas metrics with LLM-based scoring, rubrics, and Python functions when the built-in evaluators don't fit your RAG pipeline.
Building a Ragas Dashboard for Ongoing RAG Quality Tracking
Learn how to turn one-off Ragas evaluations into a persistent dashboard that tracks RAG quality over time, across releases, and across data drifts.
Ragas Integration with LangChain: A Practical Guide
Learn how to wire Ragas into a LangChain RAG pipeline to score faithfulness, relevance, and recall with real, working code.
Ragas Integration with LlamaIndex: Evaluating LlamaIndex Pipelines
Learn how to wire Ragas into a LlamaIndex RAG pipeline to score faithfulness, relevancy, and context quality with real, runnable code.
Ragas for Multi-Hop Question Answering Evaluation
Learn how to evaluate multi-hop RAG pipelines with Ragas, from custom multi-hop test sets to metrics that catch broken reasoning chains.
Ragas for Multi-Language RAG Systems
Learn how to evaluate multi-language RAG pipelines with Ragas, from translation-aware metrics to cross-lingual retrieval pitfalls and custom judges.
Ragas vs Manual Spot-Checking: Why Automate RAG Evaluation
Manual spot-checking feels rigorous but silently rots as your RAG pipeline grows — here's why Ragas-based automated evaluation actually catches what eyeballing misses.
Rate Limiting and Backpressure for AI Agent Systems
How to design rate limiting and backpressure for AI agent systems so they degrade gracefully instead of falling over under load.
Real-Time Agents: Handling Streaming Input and Low-Latency Response
A practical guide to building real-time AI agents that handle streaming input and deliver low-latency responses without sacrificing reliability.
Building a Recruiting Agent That Screens Resumes Fairly
A practical guide to building a resume-screening agent with concrete bias-mitigation techniques, structured scoring, and audit trails.
Red-Teaming Your LLM App: Adversarial Evaluation Basics
A practical guide to adversarially testing your LLM application before users find the cracks for you.
Regression Testing for Prompts: Catching Silent Quality Drops
Learn how to build a regression testing pipeline for prompts so a "small tweak" never silently tanks output quality in production.
Reranking Models Compared: Cohere vs BGE vs Jina for RAG
A practitioner's comparison of Cohere, BGE, and Jina rerankers for RAG pipelines, with code, tradeoffs, and when to self-host.
Building a Research Agent That Cites Its Sources
Learn how to build a research agent that grounds every claim in a real, checkable source instead of confidently making things up.
Building a Sales Agent That Qualifies Leads Automatically
Learn how to design, build, and ship a sales AI agent that scores and qualifies inbound leads using tool calls, structured output, and a real CRM.
Building a Scheduling Agent That Coordinates Calendars
Learn how to design a scheduling AI agent that reads calendars, resolves conflicts, and negotiates meeting times across time zones.
Self-RAG and Corrective RAG: Agents That Check Their Own Retrieval
Self-RAG and corrective RAG teach retrieval pipelines to grade their own evidence and recover when the first search comes back wrong.
Semantic Caching for RAG: Reducing Redundant LLM Calls
Learn how semantic caching cuts redundant LLM calls in RAG pipelines by matching meaning, not exact text, to slash latency and cost.
Setting Up Claude Code in a New Project: A Checklist
A practical checklist for installing, configuring, and securing Claude Code the moment you start a new codebase.
Solo Learning vs Cohort-Based Bootcamps for AI Agents
A practical comparison of solo learning and cohort-based AI bootcamps, with honest tradeoffs on cost, speed, accountability, and code quality.
Sparse vs Dense Retrieval: BM25 and Its Place in Modern RAG
A practical comparison of BM25 sparse retrieval and embedding-based dense retrieval, with code, tradeoffs, and hybrid strategies for RAG.
Speeding Up DeepEval Test Runs: Caching and Parallelization
Cut DeepEval suite runtime with built-in caching, pytest-xdist parallelization, and smarter metric batching without sacrificing eval quality.
Statistical Significance in LLM Evals: Avoiding False Confidence
Most LLM eval "improvements" are noise, not signal — here's how to tell the difference with real statistical tests.
Testing MCP Servers: Unit Tests for Tool Definitions
Learn how to unit test MCP tool definitions, schemas, and handlers so broken tools fail in CI instead of in production.
Testing RAG Systems Like Software: A QA Engineer's Playbook
A practical QA framework for testing RAG pipelines, covering retrieval metrics, generation checks, regression suites, and LLM-as-a-Judge.
Tool Design for AI Agents: Writing Schemas Agents Actually Use Well
Learn why most agent tool schemas fail in production and how to design tool interfaces that models call correctly, every time.
Building a Travel Planning Agent with Multi-Step Reasoning
Learn how to design a travel planning AI agent that reasons across multiple steps, calls tools, and recovers from its own mistakes.
From Tutorial Hell to Production: Why Bootcamps Beat Blog Posts
Tutorial hell isn't a discipline problem — it's a design problem, and it explains why structured bootcamps produce production-ready developers faster.
Vector Database Comparison: Pinecone vs Weaviate vs Qdrant vs pgvector
A practitioner's vector database comparison of Pinecone, Weaviate, Qdrant, and pgvector to help you pick the right store for your RAG stack.