Codex

OpenAI Codex for Data Science Workflows

Ira Menon · May 18, 2026 · 15 min read

Why Data Scientists Are Adding Codex to Their Toolbox

Most data science work is not modeling. It is wrangling. It is staring at a CSV with inconsistent date formats, chasing down a SettingWithCopyWarning, rewriting the same groupby-aggregate pattern for the fifth dataset this month, and writing boilerplate matplotlib code that looks almost identical every time. The actual "science" part — picking a model, tuning it, interpreting results — is often the smallest slice of the day.

OpenAI Codex, accessed through the Codex CLI, is a terminal-based coding agent that can read your project files, run code, see the output, and iterate — all without you copy-pasting snippets into a chat window. For data science specifically, that loop matters more than it does for typical web development. A pandas bug rarely announces itself clearly in the traceback; you usually need to print a dataframe's shape, dtypes, and a .head() before you understand what went wrong. An agent that can execute code and read the actual output closes that loop far faster than a chatbot that only sees text you paste in.

This article is a practical walkthrough of using Codex CLI for real data science tasks: cleaning messy data, exploratory analysis, feature engineering, debugging notebooks, and even light modeling work. It assumes you already know pandas reasonably well — the goal is to show where an agent like Codex earns its keep versus where you are better off just writing the code yourself.

Setting Up Codex CLI for a Data Science Project

Codex CLI runs from your terminal and operates on the working directory you launch it from. For a typical data science project, that means starting it from the root of your analysis repo — the folder that has your notebooks, a data/ directory, and probably a requirements.txt or pyproject.toml.

A sensible starting structure looks like this:

project/
  data/
    raw/
      sales_2023.csv
      sales_2024.csv
    processed/
  notebooks/
    01_explore.ipynb
  src/
    cleaning.py
    features.py
  requirements.txt

Before asking Codex to do anything, it helps to give it context about the project. Many teams keep a short AGENTS.md (or similar) file at the repo root describing conventions: which columns are IDs, what date format the raw data uses, which columns should never be dropped, and how missing values are typically encoded (empty string, NA, -1, etc.). Codex CLI reads this kind of file automatically if present, and it saves you from re-explaining the same context in every session.

A minimal example:

# Project conventions

- Raw CSVs live in data/raw/, never modify them in place.
- Processed output goes to data/processed/ as parquet, not CSV.
- Missing values in raw data are encoded as empty string or "N/A".
- Date columns are stored as strings in "DD-MM-YYYY" format in raw files.
- Always use snake_case column names in processed data.

With that in place, Codex stops guessing about date formats and column naming — it just follows the file.

Cleaning Messy Data with a Conversational Loop

The clearest win with Codex on data science work is data cleaning, because cleaning is inherently iterative: you inspect, you write a transform, you inspect again, you find an edge case you missed, you adjust. That is exactly the loop an agent with code execution handles well.

Say you have a raw sales CSV with mixed date formats, inconsistent casing in a region column, and some numeric columns stored as strings with currency symbols. Instead of writing the whole cleaning function yourself, you can describe the problem and let Codex inspect the file first.

A prompt like "look at data/raw/sales_2024.csv, tell me what's wrong with it, then write a cleaning function in src/cleaning.py" causes Codex to actually load the file, run df.info() and df.head(), and reason from real output rather than assumptions. That matters because a model working blind will often assume a "clean" schema that does not match your actual file.

The resulting function tends to look like straightforward, defensive pandas code:

import pandas as pd
import numpy as np

def clean_sales_data(path: str) -> pd.DataFrame:
    df = pd.read_csv(path, dtype=str)

    # Normalize column names
    df.columns = [c.strip().lower().replace(" ", "_") for c in df.columns]

    # Region casing is inconsistent: "north", "North", "NORTH "
    df["region"] = df["region"].str.strip().str.title()

    # Revenue column has "$" and commas, and blanks for missing values
    df["revenue"] = (
        df["revenue"]
        .str.replace("$", "", regex=False)
        .str.replace(",", "", regex=False)
        .replace("", np.nan)
        .astype(float)
    )

    # Two date formats show up: DD-MM-YYYY and DD/MM/YYYY
    df["order_date"] = pd.to_datetime(
        df["order_date"].str.replace("/", "-", regex=False),
        format="%d-%m-%Y",
        errors="coerce",
    )

    # Drop rows where we couldn't parse a date at all
    before = len(df)
    df = df.dropna(subset=["order_date"])
    dropped = before - len(df)
    if dropped:
        print(f"Dropped {dropped} rows with unparseable dates")

    return df.reset_index(drop=True)

Notice the print statement reporting dropped rows. That is a detail an agent that actually runs the code tends to add on its own after seeing that some dates fail to parse — it is reacting to the real failure mode in your data, not a generic template. When you ask a plain chat model for a "data cleaning function," you often get something generic that silently drops or mishandles edge cases because it never saw your file.

The practical workflow is: ask Codex to inspect first, propose the function, run it against the real file, and show you the before/after shape and a sample of rows that got dropped or coerced. That last part is easy to forget when writing cleaning code by hand under time pressure, but it is the difference between quietly losing 200 rows and knowing exactly why they were dropped.

Exploratory Data Analysis Without the Boilerplate

EDA involves a lot of repetitive but necessary steps: checking for nulls, looking at distributions, checking cardinality of categorical columns, spotting outliers, checking correlations. None of this is intellectually hard, but writing it fresh every time is tedious, and skipping it is how bugs get into a model three weeks later.

You can ask Codex to run a full first-pass EDA on a dataframe and report back in plain language, not just code. For example, asking it to "load the processed sales data and give me a written summary of data quality issues, unusual distributions, and anything that looks like it needs more digging" gets you something closer to a junior analyst's findings than a dump of code.

Under the hood, it is running things like:

import pandas as pd

df = pd.read_parquet("data/processed/sales_2024.parquet")

print(df.shape)
print(df.dtypes)
print(df.isna().sum().sort_values(ascending=False).head(10))

for col in df.select_dtypes(include="object").columns:
    n_unique = df[col].nunique()
    print(f"{col}: {n_unique} unique values")
    if n_unique < 20:
        print(df[col].value_counts())

numeric_cols = df.select_dtypes(include="number").columns
print(df[numeric_cols].describe().T)

# Flag columns with extreme skew, a common sign of outliers
skew = df[numeric_cols].skew().sort_values(key=abs, ascending=False)
print(skew.head(10))

The value is not the code itself — any data scientist can write this in five minutes. The value is that Codex runs it, reads the actual numbers, and then tells you something like "the discount_pct column has a skew of 4.2 and 90th percentile of 0.05 but a max of 0.97, which looks like a handful of erroneous entries rather than real discounts — worth checking against the source system." That is a judgment call grounded in your actual data, not a hallucinated observation.

A good habit here is to ask it to save the EDA as a reusable script (src/eda_report.py) rather than leaving it scattered across a notebook, so the same checks can be rerun automatically the next time a new data drop arrives.

Feature Engineering with Immediate Verification

Feature engineering is another area where the "write code, run it, check the result" loop pays off, because subtle bugs in feature engineering are dangerous precisely because they do not crash — they just quietly corrupt your model's training signal.

A common example: creating a lagged feature for time-series or panel data. It is very easy to get the groupby-shift order wrong and leak future information into a feature.

import pandas as pd

def add_lag_features(df: pd.DataFrame, group_col: str, target_col: str, lags: list[int]) -> pd.DataFrame:
    df = df.sort_values([group_col, "order_date"]).copy()
    for lag in lags:
        df[f"{target_col}_lag_{lag}"] = (
            df.groupby(group_col)[target_col].shift(lag)
        )
    return df

This looks correct, but it is worth having Codex actually verify it rather than trust it on sight. A good follow-up prompt is to ask it to write a small test that checks no lagged value equals a value from a later date within the same group — essentially a leakage test:

def test_no_leakage(df: pd.DataFrame, group_col: str, date_col: str, lag_col: str, target_col: str):
    for _, group in df.groupby(group_col):
        group = group.sort_values(date_col).reset_index(drop=True)
        for i in range(1, len(group)):
            expected = group.loc[i - 1, target_col]
            actual = group.loc[i, lag_col]
            if pd.notna(actual) and actual != expected:
                raise AssertionError(
                    f"Leakage or misalignment at index {i}: expected {expected}, got {actual}"
                )
    print("No leakage detected.")

Having Codex generate and *run* this kind of check against your actual feature dataframe is far more useful than asking it to "review this code for bugs" in the abstract. Static review catches obvious mistakes; running a targeted test against real data catches the subtle off-by-one and grouping errors that are the actual source of most feature engineering bugs.

This same pattern extends to encoding categorical variables, building rolling-window aggregates, and one-hot encoding with unseen categories at inference time — all places where "it ran without error" and "it is correct" are different claims, and only one of them is verified by execution.

Debugging Notebooks and Long Tracebacks

Notebooks accumulate state. A variable defined three cells up, redefined two cells later, and referenced out of order is one of the most common sources of "works on my machine, breaks when I restart the kernel" bugs. When a notebook throws a confusing traceback — say, a KeyError deep inside a groupby chain — pasting just the error into a chat model loses the surrounding context: what the dataframe actually looked like at that point, what ran before it, what other cells might have mutated shared state.

Codex CLI, run against the project directory, can open the notebook file directly (.ipynb is just JSON under the hood), trace through the cells in order, and re-execute the relevant parts to reproduce the error rather than guessing at it from a traceback alone. Concretely, a prompt like "cell 14 in notebooks/01_explore.ipynb throws a KeyError: 'customer_segment', figure out why and fix it" leads Codex to:

Check whether the column was created in an earlier cell versus assumed to already exist in the raw data
Check for a merge or join earlier in the notebook that might have silently dropped the column due to a suffix collision (_x/_y)
Re-run the relevant cells in sequence rather than only look at cell 14 in isolation

This class of bug — "the column existed when I ran this two hours ago, but not now" — is extremely common in exploratory notebooks and is genuinely hard to debug from a static read of the code, because the bug depends on execution order and mutated state. An agent that can actually execute the notebook cells is diagnosing the real runtime state rather than pattern-matching the traceback text.

One practical tip: ask Codex to convert a debugged, finalized notebook section into a plain .py module once it is stable. Notebooks are great for exploration but bad for reuse and testing — moving logic into src/ once it works removes an entire category of "it worked in the notebook but the pipeline job fails" bugs.

Automating Repetitive Reporting and Visualization Code

A large share of data science time goes into visualization code that is mechanically similar across projects: a bar chart of top categories, a time-series line plot with a rolling average overlay, a correlation heatmap. This is exactly the kind of code where hand-writing matplotlib boilerplate from scratch every time is a poor use of a data scientist's attention.

Asking Codex to generate a small internal plotting library, tailored to your project's actual column names and conventions, tends to work better than reusing generic snippets found online, because it can look at your actual dataframe first.

import matplotlib.pyplot as plt
import pandas as pd

def plot_monthly_trend(df: pd.DataFrame, date_col: str, value_col: str, window: int = 3):
    monthly = (
        df.set_index(date_col)[value_col]
        .resample("MS")
        .sum()
    )
    rolling = monthly.rolling(window=window).mean()

    fig, ax = plt.subplots(figsize=(10, 5))
    ax.plot(monthly.index, monthly.values, label="Monthly total", alpha=0.5)
    ax.plot(rolling.index, rolling.values, label=f"{window}-month rolling avg", linewidth=2)
    ax.set_title(f"{value_col.replace('_', ' ').title()} Over Time")
    ax.legend()
    ax.grid(alpha=0.3)
    return fig

The more useful part of this workflow is not the plotting code itself but asking Codex to run it against your real data and flag anything visually odd before you look at it — a sudden discontinuity that lines up suspiciously with a known data migration date, or a seasonal pattern that looks too clean and might indicate duplicated rows. Since Codex can execute the plotting code and, in supported setups, describe what shows up, it functions as a first-pass reviewer that catches obviously broken charts (empty axes, a single outlier crushing the scale) before you spend time interpreting a chart that was never valid in the first place.

Where Codex Fits Alongside Jupyter and Traditional Tools

It is worth being honest about the boundaries here. Codex is not a replacement for Jupyter, and it is not a replacement for understanding your data. A few practical guidelines:

Use Codex for the mechanical parts: cleaning functions, boilerplate EDA, repetitive plotting code, writing tests for feature logic, debugging why a notebook cell fails. These are places where execution-and-inspection genuinely beats writing from memory.
Keep modeling decisions in your own hands. Which features to include, which model family fits the problem, how to interpret a confusion matrix in the context of business cost — these require domain judgment that should stay with you. Codex can implement a train/test split and run cross_val_score, but the decision about which metric matters for your problem is yours.
Review generated code before it touches anything important. This is true of any AI-assisted coding, but it is especially true when a "cleaning" step could silently drop or alter rows. Ask Codex to report exactly how many rows changed, and spot-check a sample rather than trusting a summary alone.
Treat it as a pair programmer with a terminal, not an oracle. The strongest results come from short, verifiable loops — ask it to inspect, propose, run, and show output — rather than one large prompt asking for an entire pipeline end to end.

A reasonable rule of thumb: if a task involves running code and looking at the result to decide the next step, Codex's execution loop is a genuine speedup. If a task is really a judgment call about the business problem, treat any code Codex writes as a first draft for your review, not a final answer.

Guardrails for Running an Agent Against Real Data

Because Codex can execute shell commands and write files, it deserves the same caution you would give any tool with write access to a dataset you cannot easily reconstruct. A few habits make this safe rather than risky:

Keep raw data read-only at the filesystem level where possible. If data/raw/ is chmod'd to read-only during a session, an accidental to_csv() targeting the wrong path fails loudly instead of silently overwriting your source data.
Use the CLI's approval mode for anything that isn't a pure read. Reading a CSV and printing statistics is low risk. Running a script that writes files, installs packages, or calls an external API deserves a manual approval step, at least until you trust the specific workflow.
Work inside a virtual environment or container, never against a system Python install. Codex may suggest installing a package to resolve an import error, and you want that contained.
Version your data, not just your code. A tool like DVC, or even a simple dated-folder convention such as data/processed/2026-07-03/, means that if an agent-driven cleaning step goes wrong, you are rolling back a folder rather than reconstructing lost work from memory.
Ask to see the diff before the run, not after. A simple habit — "show me the code you're about to execute before running it" — costs a few seconds per iteration and catches an entire class of mistakes before they touch disk.

None of this is unique to Codex; it is the same discipline you would want around any automated system with write access to a data pipeline. The difference is that an agentic coding tool executes many small actions per session, so the guardrails need to be structural — permissions, sandboxing, read-only mounts — rather than something you remember to check manually every time.

Getting Started This Week

You do not need a large project to try this. Pick one recurring annoyance from your current work — a cleaning script that breaks every time a vendor changes their CSV export, a notebook that throws a stale KeyError after every kernel restart, a plotting function you rewrite from scratch each sprint — and point Codex CLI at just that file. Ask it to inspect the actual data first, propose a fix, run it, and show you the before-and-after output. That single loop, repeated across a few real annoyances, teaches you faster than any amount of reading about the tool in the abstract where it earns its keep on your specific data and where it does not.

Once the workflow clicks, it tends to spread naturally: the cleaning function you built together becomes a shared utility, the EDA report becomes a script your team runs on every new data drop, and the debugging session becomes muscle memory for the next stale-notebook bug. None of this replaces the actual thinking in data science — it just clears out the mechanical work standing between you and the parts that do require thinking.

If you want a structured, hands-on path through this rather than piecing it together from blog posts, our OpenAI Codex CLI Tutorial course on TeachYou.ai walks through setup, real pandas and notebook workflows, and debugging patterns step by step, using the same kind of messy, realistic datasets covered in this article.