The lab notebook
Field notes on AI engineering
824 articles · page 6 of 28
The Cost vs Quality Tradeoff in LLM Evaluation
A working engineer's guide to controlling llm eval cost without letting your evaluation quality collapse. Sampling, judge-model choice, caching, and the math to decide when a cheaper eval is still trustworthy.
Continuous LLM Evaluation in Production
A working engineer's guide to LLM continuous evaluation: how to score live traffic, catch silent regressions, and wire scores into CI without shipping blind.
A/B Testing LLM Changes in Production
A working engineer's guide to LLM A/B testing in production: how to route traffic, pick metrics that survive contact with users, log the right data, and read results without fooling yourself.
LLM-Assisted Data Labeling: A Practical Guide for Engineering Teams
A hands-on walkthrough of building an LLM data labeling pipeline that actually holds up in production, from prompt design to human review to catching silent quality drift.
Batch Inference for LLMs
A working engineer's guide to LLM batch inference: when to use provider batch APIs versus self-hosted continuous batching, with runnable code for Anthropic, OpenAI, and vLLM.
A/B Testing LLM Models in Production
A hands-on guide to LLM model A/B testing in production: bucket users deterministically, log the right metrics, run the stats, and ship the winner without breaking latency or cost.
LangGraph State Reducers Explained
LangGraph state reducers control how node outputs merge into shared state, and getting them wrong silently drops data in parallel branches. Here's how they actually work, with runnable code.
Collecting Human Feedback in LangGraph
A practical guide to pausing a LangGraph run, collecting real human feedback with interrupt(), and resuming the graph exactly where it left off, with working code for approval gates, edits, and multi-turn review loops.
LangGraph Checkpointing with Redis
A practical walkthrough of wiring RedisSaver into a LangGraph agent so runs survive restarts, support human-in-the-loop pauses, and scale past a single process.
Building Custom LangFlow Components
Learn how to write, test, and register langflow custom components in Python so you can drop your own logic straight into the visual canvas instead of fighting the built-in node set.
Building Custom Retrievers in LangChain
Learn how to build a LangChain custom retriever from scratch, wire in hybrid search and reranking, and avoid the pitfalls that break retrieval quality in production RAG apps.
LangChain Memory Types Compared
A working engineer's comparison of every LangChain memory type, from buffer memory to vector store recall, with runnable code so you can pick the right one instead of guessing.
LangChain Callbacks and Tracing: A Practical Guide for Debugging LLM Chains
LangChain callbacks let you hook into every step of a chain or agent run, from token streaming to tool calls, so you can log, debug, and trace exactly what your LLM app is doing.
Flowise vs n8n for AI Workflows
Flowise and n8n solve different problems even though both use drag-and-drop canvases. Here is how to pick the right one for your AI workflow, with real setup steps for both.
Fine-Tuning vs RAG in 2026: Which to Choose
A practical, code-first guide to fine-tuning vs RAG: what each one actually changes, when to reach for which, how to combine them, and the runnable commands to ship either.
Fine-Tuning Small Language Models
A hands-on guide to fine-tuning small models with LoRA and QLoRA, including data prep, runnable training code, evaluation, and deployment on a single GPU.
Evaluating a Fine-Tuned Model
A fine-tuning evaluation is how you prove your tuned model is actually better than the base, and not just memorizing. Here is a runnable workflow with held-out sets, regression checks, and LLM-judge scoring.
Preparing a Fine-Tuning Dataset
A practical, code-first guide to building a fine-tuning dataset that actually moves your model: schema, cleaning, dedup, splits, and validation you can run today.
Fine-Tuning Embedding Models for Your Domain
Off-the-shelf embedding models miss the vocabulary that makes your domain hard. Here is how to build a training set, fine-tune with sentence-transformers, and prove the lift before you ship it.
Monitoring Embedding Drift in Production
Embedding drift silently degrades retrieval quality long before anyone notices. Here is how to detect it, measure it, and set up alerts before your RAG system quietly gets worse.
Embedding Dimensionality Reduction with Matryoshka Representation Learning
Cut your vector database storage and search latency by 4x or more without retraining a model, using Matryoshka embeddings that let you truncate dimensions on the fly.
Document Extraction with Vision-Language Models: A Practical Guide
Learn how document extraction with VLMs replaces brittle OCR pipelines, with runnable code for PDFs, tables, and forms plus a field-level evaluation harness.
Codex vs Cursor Agent: Which AI Coding Tool Fits Your Workflow
A hands-on comparison of OpenAI Codex and Cursor Agent for real engineering work: setup, autonomy, editor integration, and which one to reach for on your next task.
Reviewing Pull Requests with OpenAI Codex
A practical walkthrough of using Codex code review to catch bugs, enforce conventions, and speed up pull request review, from the CLI to a fully automated GitHub workflow.
An OpenAI Codex Python Workflow
A practical setup for using OpenAI Codex on Python projects, from AGENTS.md to test-driven loops and CI, with the commands you actually need.
Prompting OpenAI Codex Effectively
Codex prompting is not the same as chatting with a general assistant. Here is how to scope tasks, write an AGENTS.md, and structure prompts so Codex ships working code on the first pass.
OpenAI Codex Cloud Tasks: Running Agentic Coding Jobs Without a Local Machine
Codex cloud tasks let you hand off a coding job to a sandboxed cloud agent and get a reviewable diff back, no local shell required. Here is how the workflow actually works, end to end.
OpenAI Codex Best Practices: A Practical Guide for Engineering Teams
A working engineer's guide to using OpenAI Codex without letting it wreck your codebase: setup, prompting, sandboxing, review discipline, and CI integration that actually holds up.
Using Claude Code on TypeScript Projects
A practical walkthrough of using Claude Code on TypeScript codebases: project setup, self-verifying type checks, monorepo edits, refactors, and the mistakes that waste your tokens.
Running Claude Code Subagents in Parallel
Learn how to dispatch multiple Claude Code subagents at once so research, code review, and test runs finish in minutes instead of one after another.