What Data Analyst Interviews Actually Score (It's Not Just Whether Your SQL Runs)

The SQL is right. The rejection comes anyway.

This isn't a rare, unlucky outcome — it's close to the default outcome for candidates who treat a data analyst interview as a technical-correctness test. The format usually isn't a timed, leetcode-style quiz. It's a business case: an ambiguous prompt, a dataset, and an open-ended 'what would you tell the VP of Marketing' at the end — closer to a consulting case round than a coding exam.

It's the data-analyst version of a mismatch covered elsewhere on this blog: technical interviews are scored on a rubric where code correctness is one row among several — data analyst case interviews run on the same principle. Different rows, same underlying rule: a right answer with no visible reasoning around it is an incomplete answer on the actual scorecard.

“A right number with no reasoning around it isn't a complete answer. It's a blank scorecard with one row filled in.”

What's actually on the scorecard

Aced (formerly Exponent), a widely used interview-prep platform built by former hiring managers from companies including Google and Facebook, publishes a take-home case-study rubric for data analyst interviews that breaks evaluation into seven separate dimensions, not one.

• Problem framing — did you correctly interpret the prompt and set the right scope before touching any data
• Framework and approach — is there a clear structure guiding the analysis, not just a sequence of queries
• Assumptions and reasoning — are the assumptions stated explicitly, and are they reasonable
• Analytical rigor — is the analysis thorough, accurate, and documented well enough for someone else to follow
• Insight quality — are the findings tied back to an actual business need, not just reported as numbers
• Visualization and storytelling — do the visuals and the narrative around them hold together
• Communication and iteration readiness — does the answer anticipate the stakeholder questions that follow

SQL correctness and the accuracy of the underlying numbers live almost entirely inside two of those seven rows — Analytical rigor, and, partially, Framework and approach. The other five are judged on reasoning and framing that a query by itself can't show.

“A correct query touches two rows out of seven. The other five are judged on how you think, not what you ran.”

Why interviews get scored on a rubric at all

A rubric this specific isn't a quirk of one company's hiring process. It's downstream of one of the most replicated findings in hiring research. Frank Schmidt and John Hunter's 1998 meta-analysis in Psychological Bulletin, covering 85 years of personnel-selection research, found that structured interviews — scored against a fixed, defined rubric — predict actual job performance meaningfully better than unstructured ones: a validity coefficient of .51 for structured interviews versus .38 for unstructured ones.

That gap shows up outside data roles too — the same structured-versus-unstructured split is exactly what makes HR and final rounds the least predictive part of most hiring processes, for the same underlying reason: no shared rubric means the evaluation leans on gut feel instead of defined criteria.

A data-analyst case-study rubric is that research applied directly to an analytics role: instead of one interviewer deciding afterward whether the candidate 'seemed sharp,' there are seven specific, named things being checked off in real time.

The actual gap: business framing, not technical skill

Here's where most candidates lose the most points without realizing it. Aced's own guidance on the rubric is explicit that the evaluation focuses on synthesis and business impact rather than isolated technical metrics — meaning a technically flawless query that never gets tied back to a business decision scores worse than a simpler query that does.

Take a typical prompt: weekly active users dropped 12% last month, what happened? A candidate scoring only on the technical rows writes a query segmenting the drop by cohort, finds that new-user activation fell hardest, reports the number, and stops.

A candidate scoring well across all seven rows runs the same query — then states the assumption they're making about what counts as 'active,' explains why they chose to segment by signup cohort before geography or platform, ties the finding to a plausible business cause such as a recent onboarding change or a marketing-channel shift, and closes with what they'd check next and what they'd tell a stakeholder in one sentence.

The SQL in both cases can be identical. The second answer fills in the other five rows the first one leaves blank.

“The query can be identical in both answers. Only one of them fills in the other five rows.”

Where candidates lose points without noticing

A handful of specific habits account for most of the gap between a technically correct answer and a fully scored one:

• Writing the first query before stating how you're interpreting the question — framing happens after the fact instead of before
• Never naming an assumption out loud, even an obvious one, which leaves Assumptions and reasoning with nothing to score
• Reporting a number with no recommendation attached — 'activation fell 12%' instead of 'activation fell 12%, which is why I'd look at the onboarding flow next'
• A wall of metrics with no narrative thread connecting them, which scores poorly on Visualization and storytelling even when every chart is individually correct
• Treating the answer as finished the moment the main number is found, instead of anticipating the follow-up question a stakeholder would actually ask next

How to practice the rows a SQL course never tests

Most SQL courses, bootcamps and certification tests are built to grade exactly one of these seven rows — correctness — because that's the only one a script can check automatically. Practicing with real case prompts, including the open-ended 'what would you tell the VP' part, is the only way to get reps on the other six, and it's a trained skill like any other interview skill: it gets more natural with repetition, not with another SQL refresher.

The underlying pattern repeats across role-specific interviews on this blog — product manager interviews run on a near-identical rubric, scored on judgment and framing as much as any single right answer, and the fix is the same in both cases: rehearse the reasoning out loud, not just the computation.

And the behavioral half of a data analyst loop deserves the same rigor — a memorized STAR answer falls apart under a real follow-up question the same way an un-narrated SQL query does.

Reps matter more than most candidates expect going in — how many mock interviews it actually takes before this stops feeling rehearsed is worth answering with real numbers rather than a guess.

Key Takeaways

If you remember only a few things, remember these:

• A data analyst case interview is scored on a rubric, not pass or fail on output — Aced's published rubric breaks it into seven dimensions: problem framing, approach, assumptions, rigor, insight quality, visualization, and communication.
• SQL correctness and the accuracy of the numbers mostly live in one or two of those seven rows. The other five are judged on reasoning you have to say out loud.
• Structured, rubric-based interviews predict job performance meaningfully better than unstructured ones — Schmidt and Hunter's 1998 meta-analysis found a validity coefficient of .51 versus .38 — which is part of why this format exists at all.
• The biggest point loss usually isn't a wrong number. It's a right number with no stated assumption, no business framing, and no recommendation attached to it.
• The same query can score very differently depending on whether it's narrated with reasoning or handed over silently.
• This is a rehearsable skill, not a fixed trait — practicing full case prompts out loud is the only way to train the six rows a SQL test can't check.