From Clay Tablets to AI: The Complete Field Guide to Data Analysis

Every job posting wants someone "data-driven." Almost none of them explain what that means on a Tuesday afternoon, in front of a spreadsheet, when the numbers don't add up.

This is the guide I wanted when I started. Not a tool tutorial — those are everywhere, and they go stale. This is the thing underneath the tools: where this discipline came from, what the words actually mean, which skills compound and which ones don't, and what an analyst is still for in a world where a machine can write a working SQL query in four seconds.

I'll be straight about where I'm standing. I'm early in this career, not twenty years into it. What I have is an accounting background, a Google Data Analytics certificate, and a portfolio of projects where I've done the unglamorous work end to end — including one where the most valuable thing I produced was the sentence "this number is wrong." Where I'm drawing on experience, I'll say so. Where I'm drawing on the accumulated wisdom of the field, I'll say that too.

Part 01Five thousand years of counting things

Data analysis feels like a modern job. It isn't. It's one of the oldest jobs there is — arguably the oldest white-collar job — and knowing its history tells you something useful about where it's going.

The earliest known writing isn't poetry or scripture. It's accounting. The clay tablets recovered from Mesopotamia, dating back to roughly 3000 BCE, are overwhelmingly administrative: tallies of grain, headcounts of livestock, records of who owed what to whom. There's a serious argument among historians that writing was invented because someone needed to keep track of inventory. The first data problem was a supply chain problem.

From there, the through-line is remarkably consistent. Every few centuries, someone figures out how to turn a pile of records into an argument.

Part 02The vocabulary, defined properly

A lot of confusion in this field is vocabulary confusion. People use "data," "metric," and "insight" interchangeably, and then wonder why meetings go in circles. Here's the vocabulary, defined in a way you can actually use.

The ladder: data → information → insight → decision

This is the most important distinction in the whole discipline, and most people stop climbing one rung too early.

Junior analysts deliver information and think they've delivered insight. The test is simple: if your output doesn't change what anybody does, it wasn't insight. A dashboard nobody acts on is an expensive decoration.

Types of data

By structure:

• Structured — fits neatly in rows and columns. Sales tables, transaction logs, spreadsheets. This is where you'll spend most of your career.
• Semi-structured — has organizational markers but no rigid schema. JSON, XML, email.
• Unstructured — no predefined model. Free-text reviews, images, audio, video. Historically hard to analyze; increasingly tractable with modern tooling.

By measurement type (this determines which statistics and charts are even legal):

Metrics, dimensions, and KPIs

• Metric — something you measure. Revenue, clicks, conversion rate.
• Dimension — something you slice a metric by. Channel, region, month, device.
• KPI — a metric someone is actually accountable for. Every KPI is a metric; very few metrics deserve to be KPIs. If everything is a KPI, nothing is.
• Benchmark — the expected value you compare against. Without one, a number is just a number. "ROAS was 3.64x" is information. "ROAS was 3.64x against a 4.0x target" is the start of an insight.
• Variance — the gap between actual and expected. This is where nearly all real analysis lives.

The four types of analytics

This is the standard maturity model, and it's genuinely useful for locating yourself and your organization.

Most job postings that say "data analyst" mean descriptive and diagnostic, with the occasional forecast. That's not a lesser job — diagnostic work is the hardest thinking in the stack, and it's what actually moves companies.

Statistical concepts you cannot skip

• Population vs. sample — the whole group vs. the subset you measured. Nearly everything you analyze is a sample, and samples have error bars even when your spreadsheet doesn't show them.
• Mean, median, mode — the mean is the most-used and least-robust. One billionaire in a room of ten people makes the mean income absurd; the median doesn't budge. When a distribution is skewed, report the median.
• Distribution — the shape of your data. Look at it before you summarize it. A histogram takes thirty seconds and prevents entire categories of embarrassment.
• Variance and standard deviation — how spread out values are. Two datasets with identical means can behave completely differently.
• Outlier — a value far from the rest. Critically: an outlier is a question, not a nuisance. Sometimes it's a typo. Sometimes it's fraud. Sometimes it's your most important customer.
• Correlation vs. causation — the one everyone can recite and few genuinely internalize. Correlation means two things move together. Causation means one drives the other. Establishing causation almost always requires an experiment or a very careful design, not a bigger dataset.
• Statistical significance — a measure of how likely your result is to be noise. Useful, routinely abused, and not the same as practical significance. A statistically significant 0.1% improvement may be worth exactly nothing.
• Simpson's paradox — a trend that appears in several groups can reverse when the groups are combined. It's the reason "always check the segments" is a rule and not a suggestion.

Part 03Analyst, scientist, engineer: who does what

Job titles in this field are notoriously inconsistent — the same title means different things at different companies. But the underlying functions are real, and understanding them tells you what to learn and what to apply for.

Part 04The lifecycle: how a real analysis actually runs

There are several standard frameworks for this. The one I learned through the Google Data Analytics certificate uses six phases, and I'll use it here because it maps well onto how the work genuinely feels. What matters isn't the labels — it's knowing what goes wrong at each stage.

1. Ask — define the question

Someone comes to you with a request. It is almost never the real question. "Can you pull last month's sales numbers?" usually means "I think something is wrong and I'm trying to work out what." Your first job is to find the decision hiding behind the request.

Questions to ask before touching any data:

• What decision will this inform? Who makes it?
• What would you do differently if the answer were X versus Y? (If the answer is "nothing," stop — you've found a curiosity, not a project.)
• What's the deadline, and what precision does the decision actually require?
• Is there a number you're expecting? What would surprise you?

Where it goes wrong: you answer the question as asked, spend two days on it, and deliver something technically correct and completely useless.

2. Prepare — find and collect the data

Identify what data exists, where it lives, who owns it, and whether you're allowed to use it. Understand how it was collected, because collection method determines what conclusions are legitimate. Survey data has selection bias. Log data has instrumentation gaps. Nothing is neutral.

Where it goes wrong: you use a table without understanding what a row represents. Getting the grain of a table wrong — is one row a transaction, a customer, a customer-month? — poisons everything downstream.

3. Process — clean and validate

This is where the majority of your time goes. It's also where most of your value gets created, which is the opposite of what beginners expect. Part 6 covers this properly because it deserves its own section.

The core moves: deduplicate, standardize text and formats, handle missing values deliberately, fix data types, investigate outliers, and document every decision.

Where it goes wrong: silent cleaning. You quietly drop 40 rows, never mention it, and nobody — including future you — can reproduce the number.

4. Analyze — find the pattern

Aggregate, segment, compare against benchmarks, look at trends over time, test the obvious explanations before the exotic ones. Good analysis is mostly disciplined comparison: this segment vs. that one, this month vs. last, actual vs. target.

Where it goes wrong: confirmation bias. You find the answer someone wanted and stop looking. The discipline is to spend equal energy trying to break your own finding.

5. Share — communicate the finding

Lead with the answer, not the methodology. Your stakeholder doesn't care how many joins you wrote. Structure it as: here's what I found, here's what it means, here's what I'd do, here's what I'm uncertain about. That last part builds more trust than false confidence ever will.

Choose charts that match the question: line for trends over time, bar for comparing categories, scatter for relationships between two variables. And label your axes — an unlabelled chart is a rumour.

Where it goes wrong: burying the finding on slide 14 after thirteen slides of methodology.

6. Act — and then measure whether it worked

The phase everyone forgets. If a recommendation is taken, the analysis isn't finished — you now have a natural experiment. Did the change do what you predicted? Analysts who close this loop become trusted very quickly, because they're the only ones keeping score.

Part 05Zero to mastery: the four levels

Here's the honest version of the skill ladder — what each level looks like, what to build, and how to know you've actually reached it rather than just watched a video about it.

One piece of advice before the levels, and it's the most practically useful thing in this section: learn in this order — Excel, then SQL, then a BI tool, then statistics, then Python. Not because Python is unimportant, but because beginners who start with Python spend six months learning to program and zero months learning to analyze. Excel teaches you to think in tables. SQL teaches you to think in sets. Both transfer. Start where the feedback loop is fastest.

Part 06The part nobody teaches you

Courses teach tools. What follows is the craft — the things that matter far more than knowing one more function, and which almost no curriculum covers properly.

Cleaning is the job, not the obstacle

The often-repeated figure is that analysts spend around 80% of their time preparing data rather than analyzing it. Treat the exact number with suspicion — it's an estimate that gets quoted far more confidently than it deserves — but the shape of it matches every real project I've worked on.

Beginners experience this as a tax on the interesting part. It isn't. Cleaning is where you learn what the data actually is, and it's where the highest-value findings tend to hide. In my own marketing campaign project, the cleaning stage was the analysis: 798 raw rows contained 18 duplicate campaign IDs, one channel name spelled four different ways, and 103 blank cells. Anyone skipping straight to the pivot table would have produced a confident, professional-looking report built on double-counted campaigns and a channel split across four spellings.

The skeptic's reflex: "too good to be true" is a hypothesis

This is the single habit I'd most want to pass on.

In that same project, one channel showed a return on ad spend of 89.7x. Every other channel sat between 2.5x and 4.7x. It accounted for 45% of all revenue on 3% of all spend.

The tempting move is to celebrate it. It's the best number in the dataset — it makes the whole portfolio look extraordinary, and the blended return came out at 6.4x on the back of it. The correct move is to distrust it. A result that far outside the range of everything comparable is far more likely to be a defect than a discovery. It looked like a units error in the source data — spend recorded at the wrong scale. Excluding it, the real blended return was 3.7x, not 6.4x. The headline number had been inflated by roughly 73%.

Notice what the actual deliverable was there. It wasn't a chart. It was preventing a company from believing a false number and setting budgets by it. Sometimes the most valuable output of an analysis is "don't trust this."

Cross-check every important number two ways

If a figure is going in front of a decision-maker, calculate it by a second, independent method and confirm the two agree. Build the summary as a PivotTable, then verify it with SUMIFS. Write the SQL aggregate, then check the row count separately. It costs a few minutes, and it's the difference between an analyst who is trusted and one who is checked.

Document your decisions, not just your steps

"Imputed 61 missing spend values using the channel average; flagged 12 rows where region was unrecoverable as Unknown" is worth more than any amount of tidy code. Cleaning involves dozens of judgment calls, and an undocumented judgment call is indistinguishable from a mistake six weeks later.

Learn the business, not just the tables

The analyst who understands why a metric matters will beat the more technical analyst who doesn't, every time. If you can't explain how your company makes money and what would make that number go up, your SQL skill has a hard ceiling. This is where an accounting background turns out to be an unfair advantage — you already think in terms of what reconciles and what doesn't.

Deliver the decision, not the dashboard

Nobody wants your analysis. They want to know what to do. Every deliverable should answer: what did you find, what does it mean, what should we do, and how confident are you. If you're explaining methodology before your finding, you've inverted it.

Part 07AI vs. the data analyst: an honest assessment

This is the question underneath every "should I still learn data analysis" post, so let's take it seriously rather than reassuringly. I'll give both arguments their strongest form before drawing a conclusion, because the honest answer is genuinely mixed.

The strongest case that AI displaces analysts

It goes like this, and it isn't stupid:

A large share of what junior analysts do all day is mechanical translation — turning a question into a query, a query result into a chart, and a chart into three bullet points. Language models are now genuinely good at all three. They write competent SQL from a plain-English description, generate spreadsheet formulas instantly, explain unfamiliar code, and produce a reasonable first-pass summary of a dataset. The tasks that used to constitute a junior analyst's first year are precisely the tasks that have become cheapest.

Meanwhile, BI tools are embedding natural-language querying directly into the product, so the stakeholder who used to file a ticket with the analytics team can increasingly just ask the dashboard. If the analyst's role was to be a human interface between a business question and a database, that interface is being automated. And organizations that need fewer juniors hire fewer juniors — which breaks the pipeline that produces seniors.

The strongest case that it doesn't

Also not stupid:

Every capability just listed operates on the mechanical layer — the layer that has been automated continuously since Hollerith's punch cards in 1890. Look back at Part 1: the spreadsheet automated the calculation, the warehouse automated the consolidation, self-service BI automated the chart-building. Each wave was predicted to reduce the need for analysts. Each one increased it, because cheaper analysis produced more analysis, and more analysis produced more demand for someone who could tell which of it was trustworthy.

And trust is exactly where these systems remain weakest. An AI can write the query that produces the 89.7x return. It has no particular reason to doubt the result — the query is correct, the arithmetic is correct, the data says what it says. What it lacks is the context to know that 89.7x is implausible for that channel, the instinct to suspect a units error, and the standing to walk into a meeting and tell a marketing director that the number they've been reporting to the board is wrong. The characteristic failure mode is confident, fluent, well-formatted output that's wrong in a way requiring domain knowledge to detect.

Where I actually land

The honest synthesis, I think, is this: AI raises the floor and raises the bar at the same time.

It raises the floor because a beginner can now do things that used to require a year of training. Writing a window function is no longer a meaningful barrier to entry. That's genuinely good, and it's a large part of why someone can self-teach this field from anywhere in the world — including from Somalia, which is not a place with an abundance of senior analysts to learn from.

It raises the bar because that same beginner is now competing against everyone else holding the same tool. If your entire value proposition was "I can write SQL," that proposition has depreciated sharply. What appreciates is everything the model can't supply: business context, skepticism, accountability, and the ability to persuade.

The risk I take most seriously isn't job displacement, though — it's skill hollowing. If you let AI write every query, you may never develop the intuition that tells you a result is wrong. That intuition gets built by struggling through the mechanical work, not by skipping it. There's a real trade-off between learning fast and learning deeply, and beginners in particular should be deliberate about which one they're optimizing for on any given day.

Part 08A twelve-month roadmap

Concrete, ordered, and realistic for someone studying part-time alongside other commitments. Adjust the pace; don't reorder the sequence.

On portfolios

Three deep projects beat ten shallow ones. Recruiters aren't counting; they're looking for evidence you can think. For each project the README matters as much as the code — state the question, the cleaning decisions and why you made them, the findings with numbers, and the recommendation.

Above all: use messy data. Anyone can produce a chart from a clean tutorial dataset. The demonstration that gets you hired is showing what you did when the data was broken — because that's what the job is, every day.

Part 09What it comes down to

Five thousand years ago someone in Mesopotamia pressed marks into wet clay because they needed to know how much grain was in the store. In 1854 John Snow drew dots on a map because he suspected the experts were wrong about cholera. Today an analyst opens a spreadsheet because a number in a board deck doesn't look right.

It's the same job. The tools have changed beyond recognition — clay to punch cards to spreadsheets to warehouses to models that write their own queries — and every single change made the mechanical part cheaper. Not one of them replaced the person willing to look at a confident number and ask whether it's true.

That's the whole discipline, really. Learn the tools, because you can't earn the right to be skeptical without them. But understand that the tools are the entry fee, not the job. The job is judgment: knowing what to ask, noticing what doesn't fit, and being willing to say so.

If you're starting out — start. Get a messy dataset, clean it badly, then clean it better. It's an unusually learnable field, and one of the few where an outsider with persistence and an internet connection can build genuine, demonstrable competence from anywhere.