A dataset rarely looks messy at first glance. It loads, the columns have names, and the first twenty rows seem sensible. Then you group by region and a category you have never seen appears. Or a join quietly drops a tenth of your rows because one side stores ZIP codes as text and the other as numbers.

Cleaning is not one heroic afternoon. It is a sequence of small, boring decisions, each easier to defend when it is written down. This 30-day checklist breaks that work into daily tasks you can finish in under an hour, covering missing values, duplicates, outliers, data types, documentation, and validation.

How to Use This Checklist

Treat each day as a short, focused session, not a deadline. If a task takes ten minutes, spend the rest of the session looking at rows by hand. Four rules apply throughout:

1. Never edit the raw file. Copy it and keep the original read-only.
2. Log decisions as you make them. Memory is not documentation.
3. Prefer a script over manual edits, even for a one-off fix.
4. When two reasonable options exist, pick one, note why, and move on.

Days 1–5: Understand the Data Before Changing It

Day 1. Write down, in one sentence, the question this dataset is supposed to answer. Every cleaning decision should serve that sentence.

Day 2. Profile the file: row count, column count, data types, memory use, and null counts per column. In pandas, df.info(), df.describe(), and df.isna().sum() cover most of it.

Day 3. Find the data dictionary or ask whoever exported the file what each column means, its units, and when it was collected. "Revenue" in euros and "revenue" in cents will wreck an analysis that never checks.

Day 4. Establish the grain: one row equals one what? Test a candidate key for uniqueness before you trust it.

Day 5. Freeze a copy of the raw data, record where it came from and the date you pulled it, and note its row count. That count is your reference for every later reconciliation.

Days 6–12: Missing Values and Duplicates

Days 6–7. Count missingness per column, then look at the pattern. Missing values scattered evenly are often safe to impute. Missing values concentrated in one region, one month, or one upload batch usually signal a collection problem, and imputing them hides it.

Day 8. Decide per column: drop the row, impute with a documented method (median, group median, forward fill), or keep the null and handle it downstream. A null is sometimes information.

Not every empty cell is missing data

A blank "date of last purchase" for a customer who never bought anything is not a gap to fill; it is a true zero. Encoding it with the median purchase date corrupts both the timeline and the segment. Create a separate flag column for "not applicable" versus "unknown".

Days 9–11. Handle duplicates in two passes. First exact duplicates on the full row. Then near-duplicates: names or emails that differ only by case, surrounding whitespace, punctuation, or a trailing middle initial. Standardise text (trim, lowercase, strip punctuation) before you compare, or you will miss most of them.

Day 12. Write down every decision from this week: column, rule, rows affected, and the reason. Keep it in a single file, not spread across chat messages.

Days 13–18: Outliers and Data Types

Days 13–15. Investigate outliers instead of deleting them. Plot each numeric column and sort by extremes. An age of 200 is a typo. A weight that jumps by a factor of 2.2 is a units problem. A single enormous order value may be your most important customer. Cap or winsorise only when you can explain why in one sentence.

Days 16–17. Fix data types properly:

• Dates stored as strings, often in two or three formats in the same column.
• Numbers carrying currency symbols, thousand separators, or stray text like "approx".
• Booleans written as Y/N, 1/0, true/false, and blank.
• Categories with typos and spacing variants — "New York", "new york ", "NY", "N.Y."

Day 18. Run range and domain checks: percentages between 0 and 100, no negative quantities where they are impossible, dates that are not in the future, and codes that actually exist in your reference list.

Days 19–24: Document Every Change

Keep a changelog with four columns: date, field, rule applied, and rows affected. Write entries while you work, not at the end. A useful test: if you cannot explain a change to a colleague in one sentence, you probably should not make it silently.

Cleaning should be reproducible from the raw file. If you have to remember what you clicked, the work is not finished.

Move manual edits into a script so the same transformation can run next month. Version your outputs in stages — raw, interim, clean — and update the data dictionary with any column you renamed, split, or created. Then note what you deliberately chose not to fix, and why. Those notes prevent the next person from "fixing" it differently.

Days 25–30: Validation Checks Before Anyone Uses It

Run these checks on the cleaned output and record the result of each:

• Row count reconciles with the source, or the difference is explained.
• Primary key is unique; no nulls in required fields.
• Referential integrity holds across joins — every foreign key matches a record.
• Range, domain, and cross-field logic pass, such as end date on or after start date.
• Totals add up: parts sum to the whole in any grouped report.
• Distributions are compared before and after cleaning, so you can see whether you shifted a mean or lost a segment.

Then run the whole pipeline from the raw file on a fresh copy, start to finish. If it succeeds, hand it to someone else and watch them run it without your help. Their questions are your missing documentation.

After Day 30: Make It a Habit

Convert the checks that mattered most into automated tests that run whenever new data arrives. Keep a small set of hand-verified rows as a reference you can compare against after every refresh. Review the rules once a quarter and delete the ones that no longer apply — column names change, sources get replaced, and stale cleaning logic can quietly damage new data. Thirty days gets one dataset into shape; a short recurring routine keeps it there.

Photo: Anna Tarazevich / Pexels