A Practical EDA Workflow (Before You Touch a Model)

The fastest way to waste a week is to jump straight to modeling on data you haven't looked at. The model trains, the score looks fine, and three days later you discover a column was 40% nulls, or a "price" field had negative values, or two features were the same thing measured twice.

Exploratory data analysis (EDA) is the cheap insurance against that. It's not glamorous, but it's the same handful of questions every time, so I've turned it into a checklist.

Step 1 — Shape and types

Before anything, know what you're holding. How many rows, how many columns, what type is each?

import pandas as pd
 
df = pd.read_csv("data.csv")
df.shape            # (rows, columns)
df.info()           # dtypes + non-null counts
df.head(10)         # actually look at the data

df.info() alone catches half the problems: a numeric column stored as text, a date parsed as a string, a column that's almost entirely null.

Step 2 — Missing data, honestly

Nulls aren't just noise; where they're missing tells you something. A sensor that fails at night leaves a pattern, not random gaps.

df.isna().mean().sort_values(ascending=False)   # fraction missing per column

Then decide per column, not globally: drop it if it's mostly empty, impute if the gaps are small and random, or treat "missing" as its own category if the absence is meaningful.

Step 3 — Distributions

For each numeric column, look at the spread. The summary stats catch the obvious; a histogram catches the rest.

df.describe()                    # count, mean, std, min, quartiles, max
df["price"].plot(kind="hist", bins=50)

This is where impossible values surface — a negative age, a price of zero, an outlier three orders of magnitude off. describe() showing a min of -1 on a count column has saved me more than once.

Step 4 — Categories and cardinality

For text and categorical columns, count the values. High cardinality (thousands of unique values) changes how you'll encode a feature later.

df["region"].value_counts()
df["user_id"].nunique()          # is this an ID? then it's not a feature

A column with as many unique values as rows is almost always an identifier, not something to model on.

Step 5 — Relationships

Now look at how features move together — with each other and with your target.

df.corr(numeric_only=True)["target"].sort_values()

Two things to watch for. First, features strongly correlated with the target are your promising signal. Second, features strongly correlated with each other are redundant — keeping both adds noise and, for some models, instability.

Step 6 — Write down what you found

The output of EDA isn't the charts. It's a short list of decisions: these columns get dropped, this one gets imputed, this one is a leak, the target is imbalanced 90/10 so accuracy is the wrong metric. That list is what actually shapes the model.

Why the discipline pays off

Every EDA step is a question that, unanswered, becomes a bug later. Skipping it doesn't save time — it moves the debugging to a worse place, after you've built on top of the flaw. Twenty minutes with info(), describe(), and value_counts() is the highest-leverage part of the whole project.