Assess the work
Data engineer interview scorecard: late events, replay and corrections
Trace a subscription event into an analytical result, then introduce duplication, late corrections and a schema change. Assess how a candidate explains correctness and recovery without hiding missing data.
Run a bounded 60-minute data discussion
This discussion assesses event-contract, data-correction and governance reasoning. It is not a SQL coding test, a finance-policy exam or proof of production pipeline implementation. Share the format and prompt in advance, offer an adjustment path, and score independently before debrief.
| Minutes | Stage | Fixed prompt | Stop rule |
|---|---|---|---|
| 0–6 | Orient | Read the scenario. “Which definition or owner would you need before claiming this metric is correct?” | Answer only clarifying questions; do not provide an expected design. |
| 6–17 | Event contract | “How would you represent event identity, event time and arrival time so repeated delivery can be recognized?” | Capture the candidate's key and assumptions; move on at minute 17. |
| 17–29 | Duplicate and late event | Reveal the repeated event and a late correction. “What should change in the canonical history and the published monthly metric?” | Ask the fixed follow-up: “Who decides the monetary meaning?” Do not invent policy for the candidate. |
| 29–41 | Schema change | Reveal a producer adding a new event version and renaming a field. “What do you check before accepting it, and who might be affected?” | Stop after producer/consumer compatibility and one validation check are named. |
| 41–52 | Replay and correction | “Finance reports a prior month is wrong. How would you preview, bound, run and verify a correction?” | Ask the fixed follow-up: “What must remain available to explain the before/after change?” No live data modification. |
| 52–57 | Lineage and access | “A downstream analyst requests raw customer identifiers to debug a dashboard. What do you offer and who approves access?” | Score data minimization and traceability, not a named catalog product. |
| 57–60 | Close | “What unknown would you resolve first before changing the published number?” | Close; do not introduce another scored competency. |
Global stop rule: stop each stage at its minute and do not borrow time from later dimensions. Candidate may ask for clarification, but interviewers should only clarify facts printed in the packet. Optional SQL, orchestration or statistical-analysis probes are unscored extensions for a separate, consistently offered round.
Beam's documentation describes a watermark as the system's estimate of input completeness and makes late-data handling dependent on trigger and allowed-lateness choices. The exercise therefore asks for an explicit policy rather than one assumed streaming answer: Beam programming guide.
Give the candidate the supplied facts
The candidate-facing print target contains only the problem, instructions and data facts. It excludes scoring anchors and interviewer guidance.
Only the packet below is intended for candidates.
Scenario: subscription events arrive late and twice
A subscription service emits events to a data pipeline. Each event includes a subscription identifier, event identifier, event type, event timestamp, received timestamp, amount and currency. No authoritative rule has been supplied for whether a dashboard's “monthly revenue” means cash collected, invoiced amount or recognized revenue.
Two records arrive with the same event identifier but different received times. One event for the prior month arrives after that month has been reported. Later, the producer announces a new schema version and renames the amount field. Finance says a prior month's published number needs correction but has not yet approved the adjustment rule or replay scope.
Talk through what you would clarify, preserve, validate and change. State who needs to decide business definitions and who should approve a correction. You may ask the interviewer to repeat a supplied fact. You do not need to choose a warehouse, stream processor or transformation tool.
This is a discussion only. Do not use live systems, customer records or credentials. There is no required architecture or financial policy answer.
Score the event contract and consumer boundary
Fixed prompt, 6–17: “How would you represent event identity, event time and arrival time so repeated delivery can be recognized?”
Look for a stable source event key, an explicit distinction between source event time and ingestion time, and a policy for conflicting records sharing an identity. A storage key alone does not answer whether a later row is a correction, duplicate or invalid conflict.
Fixed follow-up: “Which downstream consumers or metric owner would you identify before changing the canonical output?” Seek an awareness that the same row can feed reports with different freshness or restatement requirements.
Do not require Kafka, Beam, a warehouse brand or a particular dimensional model. Credit batch, streaming or hybrid proposals when their latency, completeness, cost and correction behavior match the supplied need.
Probe access to identifiers and lineage
This prompt checks the data access boundary, not application tenant authorization.
Fixed prompt, 52–57: “A downstream analyst requests raw customer identifiers to debug a dashboard. What do you offer and who approves access?”
Look for the least data needed to diagnose the issue, a safe aggregate or pseudonymous path where it meets the need, a named data owner or privacy/security approver, and a way to trace the metric to its input fields. If raw access is necessary, the candidate should describe purpose and scope rather than treat read-only as risk-free.
Fixed follow-up: “How would you show which dashboards use the renamed field?” Credit a catalog, query lineage, explicit consumer registry or another maintainable method. Tool brand is not the competency.
NIST's least-privilege definition applies to access decisions: grant only what the task requires. It does not replace the organization's data classification and approval rules: NIST least privilege.
Stage correction, schema change and replay
Stage 1, minute 17: reveal that two records share an event ID with different receive times and a prior-month event arrives after reporting. Ask: “What should change in the canonical history and the published monthly metric?” Then ask: “Who decides the monetary meaning?” A correct response may defer publication until the policy owner settles the rule.
Stage 2, minute 29: reveal that the producer adds a schema version and renames the amount field. Ask: “What do you check before accepting it, and who might be affected?” Listen for producer/consumer contracts, compatibility, data validation and downstream metric impact.
Stage 3, minute 41: “Finance reports a prior month is wrong. How would you preview, bound, run and verify a correction?” Then: “What must remain available to explain the before/after change?” Seek authority, scoped inputs, repeat-safe execution, reconciliation and retained evidence. No one should change published data during the interview.
Stopping rules: do not ask the candidate to invent cash-vs-recognition policy; stop after a reviewable correction plan and its owner are identified; do not add a second dataset or live SQL task; stop each stage on time. Optional discussion of partitioning, streaming framework details or statistical inference is unscored.
PostgreSQL documents constraints and `ON CONFLICT` behavior as implementation tools. The candidate is not required to name them; assess the data identity and transaction reasoning instead: PostgreSQL constraints and PostgreSQL INSERT.
Rate the evidence and name what remains unknown
Write the candidate's actual decision, assumption or proposed check before assigning a level. No one should be penalized for refusing to invent an unprovided monetary rule.
Worked calibration: borderline and strong
Hypothetical borderline evidence: “The later row has the same ID, so upsert it and rerun the month.” When asked who owns the monetary definition and what the rerun touches, the candidate says they would decide based on the field name and rebuild the whole table. For metric definition and replay safety, 1 fits this evidence: the response overrides missing business authority and proposes broad unreviewed change. Do not infer that every upsert or full rebuild is wrong; the missing policy, scope and validation are the issue.
Hypothetical strong evidence: The candidate preserves both source and received timestamps, quarantines a conflicting duplicate until the source correction rule is known, asks finance to define the measure and restatement policy, checks consumers before accepting the renamed field, previews a bounded non-production replay, compares affected aggregates and records the approved definition and before/after result. This may support 4 across event, schema and correction dimensions if the fixed prompts elicited each point.
NO example: If the interviewer never reveals the late event because the schedule was interrupted, mark late-data reasoning NO with “inject skipped.” Do not use another dimension as a proxy.
Scale: 1 = material gap remains after the fixed follow-up; 2 = partial evidence with a meaningful gap; 3 = workable evidence at this role's stated scope; 4 = level 3 plus independent handling of an important interaction; NO = not enough fair evidence. Record why NO applies. Do not convert it to 1 or zero.
| Competency | 1 · Gap | 3 · Meets this role | 4 · Strong evidence |
|---|---|---|---|
| Event identity and late data | Uses arrival time as event truth or assumes duplicate delivery can be removed without an identity/correction rule. | Separates event and arrival time, names a source identity or conflict policy, and explains how late events reach an approved metric result. | Explains a concrete collision/correction edge, bounds lateness behavior and makes the affected output traceable for consumers. |
| Metric and monetary definition | Invents subscription/revenue policy from field names or treats a successful aggregate as proof the number is correct. | Identifies the metric owner and required policy inputs such as effective time, currency and reversals before changing a published value. | Separates technical transformation from policy approval and shows how definition version and correction meaning travel to reports. |
| Schema and consumer evolution | Accepts a renamed field without checking producers, downstream consumers or compatibility. | Checks version/compatibility expectations, identifies affected consumers and proposes a validation before switching output. | Describes a staged migration or explicit compatibility window with lineage and a consumer sign-off that limits breakage. |
| Replay and correction safety | Overwrites/rebuilds an unbounded range without preview, approval or verification evidence. | Bounds the source range, previews impact, gets the proper owner approval and compares expected before/after output. | Explains idempotent rerun or recovery boundary, retained audit evidence and how partial failure avoids publishing a mixed result. |
| Lineage, access and communication | Grants raw identifiers broadly or cannot trace a reported number to its source and owner. | Offers a minimum-necessary debugging path, names an approver and can trace the relevant fields and downstream consumer. | Connects access review, lineage and consumer notice to an actionable correction record while protecting unrelated data. |
For every dimension, 2 means the answer contains a useful part but leaves a material gap or needs substantial prompting. NO means the interviewer did not elicit enough fair evidence because time ended, a prompt was skipped or unclear, or an agreed adjustment changed the evidence available. Record the reason and use the same follow-up for candidates at that stage.
Interviewers score independently, then discuss evidence and the job's actual boundaries. If an owner or policy was absent from the prompt, credit the candidate for naming the gap and do not grade their invented answer as business truth.
Print a self-contained evidence packet
Record job-related evidence only. Keep any notes within the organization's candidate-data access and retention rules.
The interviewer packet repeats the agenda and rating rubric for standalone use.
Data engineer interview: fixed 60-minute agenda
| Minutes | Stage | Fixed prompt | Stop rule |
|---|---|---|---|
| 0–6 | Orient | Read the scenario. “Which definition or owner would you need before claiming this metric is correct?” | Answer only clarifying questions; do not provide an expected design. |
| 6–17 | Event contract | “How would you represent event identity, event time and arrival time so repeated delivery can be recognized?” | Capture the candidate's key and assumptions; move on at minute 17. |
| 17–29 | Duplicate and late event | Reveal the repeated event and a late correction. “What should change in the canonical history and the published monthly metric?” | Ask the fixed follow-up: “Who decides the monetary meaning?” Do not invent policy for the candidate. |
| 29–41 | Schema change | Reveal a producer adding a new event version and renaming a field. “What do you check before accepting it, and who might be affected?” | Stop after producer/consumer compatibility and one validation check are named. |
| 41–52 | Replay and correction | “Finance reports a prior month is wrong. How would you preview, bound, run and verify a correction?” | Ask the fixed follow-up: “What must remain available to explain the before/after change?” No live data modification. |
| 52–57 | Lineage and access | “A downstream analyst requests raw customer identifiers to debug a dashboard. What do you offer and who approves access?” | Score data minimization and traceability, not a named catalog product. |
| 57–60 | Close | “What unknown would you resolve first before changing the published number?” | Close; do not introduce another scored competency. |
Fixed prompts
- Event contract: “How would you represent event identity, event time and arrival time so repeated delivery can be recognized?” Follow-up: “Which downstream consumers or metric owner would you identify before changing the canonical output?”
- Late/duplicate inject: “What should change in the canonical history and the published monthly metric?” Follow-up: “Who decides the monetary meaning?”
- Schema inject: “What do you check before accepting it, and who might be affected?”
- Correction: “How would you preview, bound, run and verify a correction?” Follow-up: “What must remain available to explain the before/after change?”
- Lineage/access: “A downstream analyst requests raw customer identifiers to debug a dashboard. What do you offer and who approves access?” Follow-up: “How would you show which dashboards use the renamed field?”
Scale: 1 = material gap remains after the fixed follow-up; 2 = partial evidence with a meaningful gap; 3 = workable evidence at this role's stated scope; 4 = level 3 plus independent handling of an important interaction; NO = not enough fair evidence. Record why NO applies. Do not convert it to 1 or zero.
| Competency | 1 · Gap | 3 · Meets this role | 4 · Strong evidence |
|---|---|---|---|
| Event identity and late data | Uses arrival time as event truth or assumes duplicate delivery can be removed without an identity/correction rule. | Separates event and arrival time, names a source identity or conflict policy, and explains how late events reach an approved metric result. | Explains a concrete collision/correction edge, bounds lateness behavior and makes the affected output traceable for consumers. |
| Metric and monetary definition | Invents subscription/revenue policy from field names or treats a successful aggregate as proof the number is correct. | Identifies the metric owner and required policy inputs such as effective time, currency and reversals before changing a published value. | Separates technical transformation from policy approval and shows how definition version and correction meaning travel to reports. |
| Schema and consumer evolution | Accepts a renamed field without checking producers, downstream consumers or compatibility. | Checks version/compatibility expectations, identifies affected consumers and proposes a validation before switching output. | Describes a staged migration or explicit compatibility window with lineage and a consumer sign-off that limits breakage. |
| Replay and correction safety | Overwrites/rebuilds an unbounded range without preview, approval or verification evidence. | Bounds the source range, previews impact, gets the proper owner approval and compares expected before/after output. | Explains idempotent rerun or recovery boundary, retained audit evidence and how partial failure avoids publishing a mixed result. |
| Lineage, access and communication | Grants raw identifiers broadly or cannot trace a reported number to its source and owner. | Offers a minimum-necessary debugging path, names an approver and can trace the relevant fields and downstream consumer. | Connects access review, lineage and consumer notice to an actionable correction record while protecting unrelated data. |
For every dimension, 2 means the answer contains a useful part but leaves a material gap or needs substantial prompting. NO means the interviewer did not elicit enough fair evidence because time ended, a prompt was skipped or unclear, or an agreed adjustment changed the evidence available. Record the reason and use the same follow-up for candidates at that stage.
Optional unscored extensions: a separate SQL exercise, orchestration implementation or statistical analysis. Do not add these to the same hour or use them for these ratings.
Role / level: ____________________________________ Date: ______________
Candidate identifier: __________________________ Interviewer: __________________
Evidence sheet
| Competency | Observed words, decisions and assumptions | Rating | Missing evidence / consistent follow-up |
|---|---|---|---|
| Event identity and late data | 1 / 2 / 3 / 4 / NO | ||
| Metric and monetary definition | 1 / 2 / 3 / 4 / NO | ||
| Schema and consumer evolution | 1 / 2 / 3 / 4 / NO | ||
| Replay and correction safety | 1 / 2 / 3 / 4 / NO | ||
| Lineage, access and communication | 1 / 2 / 3 / 4 / NO |
Independent debrief notes
Evidence tied to the role outcome:
______________________________________________________________
Different readings and relevant anchor:
______________________________________________________________
Unknown policy or evidence to follow up consistently:
______________________________________________________________
Support, access or ownership gap in the job brief:
______________________________________________________________
Before using it
- Confirm data, metric and correction owners are available to the hire.
- Have a data engineer and the relevant metric owner review scenario assumptions.
- Dry-run wording and timing; no human pilot is claimed here.
- Share format in advance and offer a reasonable adjustment path.
- Use a separate exercise if production implementation or SQL execution is required for the role.
This is a discussion aid, not a validated hiring test, finance/accounting policy or automated recommendation. People make the decision and remain accountable for consistent administration.