Python backend interview scorecard and worked scenario

Run a consistent 60-minute interview

This is a structured discussion. It does not demonstrate implementation, debugging or testing ability. If those are essential, schedule a separate bounded work sample with the same instructions and support for each candidate.

This original practice scorecard assesses how a Python backend engineer reasons about a customer job-processing service. It is a structured discussion, not a coding test, company question bank or validated predictor of job performance. Adapt it with people who know the role before using it with candidates.

Dry-run checklist before the first round

• Start a timer and stop each stage at its scheduled boundary: 3, 5, 10, 5, 10, 11, 7, 6 and 3 minutes.
• Confirm the candidate prompt prints by itself, without interviewer clarifications, sample answers or rating anchors.
• Check that the interviewer can capture one concrete observation per competency before the six-minute independent-rating block.
• If a stage does not fit, revise the timing or prompt before the round and keep the revised version consistent for every candidate.
• This timing is a proposed plan. It has not been dry-run with a human interviewer or candidate.

Use one interviewer to lead and one to take notes where possible. Both should rate the evidence independently before discussing it. If one interviewer attends alone, record the initial evidence and rating before checking with the hiring group.

OPM's structured-interview guidance ties questions to job-related competencies and recommends consistent questions and rating standards. That is the design basis here. It does not establish that this specific, unpiloted scorecard predicts performance. Before relying on it, have subject-matter experts map each criterion to actual work, pilot the timing and clarity, and revise uniformly.

Source: U.S. Office of Personnel Management, Structured Interviews; Structured Interview Guide (PDF).

↑ Back to sections

Candidate prompt

Interviewer-only setup: The prompt intentionally leaves message durability, queue behavior, authentication details, report duration and data sensitivity unspecified. Do not silently select a product policy for the candidate. If they ask, use the fixed clarification responses below; otherwise note the assumption and proceed.

Keep candidate instructions separate from interviewer notes. If a candidate asks for a clarification not listed here, answer only what is necessary, then record the answer and give it consistently to the rest of the round. The print button contains only the title, instructions and scenario above; the clarifications and interviewer notes are outside its print target.

↑ Back to sections

Stage 1: design the Python job-processing flow

Core question: “Walk me through one request from the API receiving it to the customer reading a finished report. Where is each state change recorded, and what happens if each step fails?”

Fixed follow-ups: “What does the API return before processing finishes?” “How does a worker learn about work?” “How would support distinguish queued, running, complete and failed?” “Which state is authoritative if a queue message and a database row disagree?”

Listen for a coherent chain, not a preferred architecture. A strong explanation often separates a short request transaction from longer processing; records a durable job identity and state; describes how work is published or discovered; and acknowledges the gap between database commit and message publication. A transactional outbox, a database-backed queue or another design can be reasonable when its failure and operational costs are explained.

Python execution probe, 18–23 minute stage

Use this probe when Python asyncio is part of the job. If the role is synchronous and does not use asyncio, choose the alternative before the round and give that version to every candidate.

async def submit(payload):
    asyncio.create_task(process_report(payload))
    return {"status": "accepted"}

async def process_report(payload):
    report = render_large_report(payload)  # synchronous, CPU-heavy
    await save_report(report)

Two fixed questions, ask both: “What can this endpoint promise if it returns after create_task, and what happens to the work and its error if the process exits?” “What happens to other requests while render_large_report runs, and how could you test that impact?”

Expected reasoning: create_task schedules a coroutine on the current event loop; it does not persist the job. The excerpt retains no strong task reference or completion/error owner, so it cannot support a durable acceptance promise. The synchronous CPU-heavy call runs on the event-loop thread until it returns, delaying other tasks and I/O. A candidate may move durable work to a recoverable queue/worker and isolate CPU-bound work in an appropriate bounded executor or process; credit the explanation of which boundary guarantees recovery, not a specific library name.

Optional follow-up outside the 60-minute score: If cancellation, graceful shutdown and resource cleanup are job-critical, schedule the same extra follow-up for every candidate in the round. Ask how shutdown stops intake, drains or returns work, closes owned resources, and propagates cancellation after cleanup. Record this optional extension separately as an unscored note. If it was not asked, leave the core Python rating based on the two fixed questions; mark the extension “not asked.” Where the role does not need this execution model, do not use the async probe.

Do not require the candidate to propose an outbox by name. Credit another design if they explain how it avoids an untracked gap, or openly bound the risk and describe the recovery path. An exact vendor or framework answer is not the competency.

Technical references: Python task creation covers task scheduling and retaining task references; Python guidance on blocking code explains why blocking work should not run on the event loop; Python asyncio queues describe in-process queue accounting, not durable work across process loss. PostgreSQL transaction tutorial explains transaction boundaries and commit behavior.

↑ Back to sections

Stage 2: isolate tenant data

Fixed scenario change: “A user authenticated for account A changes a job ID in the URL and receives account B's report status. What would you change, and how would you test that it stays fixed?”

Ask the candidate to trace the identity used at each boundary: authenticated principal, account membership, request handler, database query, worker message and result download. A strong answer treats the authenticated account as trusted context, checks membership and authorization on every relevant read/write, and avoids relying on an unguessable job ID as access control. It may describe tenant-scoped queries, foreign-key or uniqueness design, defense-in-depth checks and negative tests that attempt cross-account access.

Follow-up: “The worker receives only a job ID. What would you want it to load before processing?” Look for a deliberate source of tenant identity and validation that the job, result and requesting account remain linked. Do not prescribe row-level security or a particular schema; ask why the chosen control fits the system and how it is verified.

Score the reasoning and evidence, not familiarity with one access-control product. A design that puts the check only in the UI, checks only job creation, or trusts a caller-supplied tenant without binding it to the authenticated identity leaves a concrete gap.

Technical reference: PostgreSQL row security policies are one possible database-level control. They require deliberate policy and role setup; their presence alone does not establish that application access is safe.

↑ Back to sections

Stage 3: recover safely from retries

Fixed scenario change: “A client sends a request, times out before seeing the response, and retries. Later, a worker commits the completed result but crashes before acknowledging its queue message. The message is delivered again. What should happen?”

Separate the two duplicate paths. The client retry needs a stable request identity scoped to the caller or account, with a defined rule for reusing a key with different input. The worker retry needs idempotent processing or deduplication so repeated delivery does not produce conflicting side effects. A database uniqueness constraint or recorded processing state can help, but the answer must explain the transaction and the side effect boundary.

Fixed follow-ups: “When is it safe to acknowledge?” “What if a downstream report generator has an effect outside PostgreSQL?” “What state would an operator inspect after repeated failure?” Credit candidates who say that exactly-once effects across independent systems are not obtained merely by choosing a queue flag. They may use idempotency, an outbox/inbox, reconciliation or compensation, with explicit limits.

Also listen for bounded retries with delay, a terminal or reviewable failure state, poison-message handling and alerting based on age or repeated attempts. These are useful only when connected to the service's recovery policy; naming a dead-letter queue alone is not an end-to-end answer.

Technical references: Python task cancellation describes cooperative cancellation and cleanup in in-process tasks. PostgreSQL transaction isolation documents serialization failures and the need to retry the complete transaction in applicable cases. Neither reference guarantees exactly-once behavior across an external queue and database.

↑ Back to sections

Score evidence against anchored criteria

Rate each competency separately on the same four-point evidence scale. Write the observed statement or decision before choosing a level. A rating describes evidence in this interview; it is not a label for the candidate's ability or potential.

For an early-career role, a level 2 answer may be appropriate where the team can teach the missing step and the job does not require independent ownership. For a senior role, expect broader decisions only when the job grants that responsibility. Calibrate anchors before interviews by mapping them to the actual work; do not change the bar after seeing a candidate.

Do not add an automatic total or hire/no-hire cutoff. Interviewers should discuss the evidence, unresolved gaps, other job-relevant assessments and the support the role provides. If the team later chooses weights or a decision rule, document its job basis, apply it consistently and review its consequences with qualified HR/legal support.

OPM guidance recommends common standards for scoring; it also advises equal weights absent a clear, documented rationale for different weights: OPM scoring FAQ.

↑ Back to sections

Printable evidence sheet

Complete notes independently before the debrief. Record job-related statements, decisions and examples. Avoid appearance, accent, family status, age assumptions, perceived similarity or other personal impressions. If an adjustment affects the format, note only enough to explain the evidence gap or what follow-up is needed.

You can also use your browser's Print command to print this sheet.

async def submit(payload):
    asyncio.create_task(process_report(payload))
    return {"status": "accepted"}

async def process_report(payload):
    report = render_large_report(payload)  # synchronous, CPU-heavy
    await save_report(report)

Before using this scorecard

• Confirm every competency is required for this role and level.
• Have technical subject-matter reviewers check the scenario and anchors.
• Pilot the prompt with someone who is not being evaluated; revise unclear wording before the round.
• Share the format and duration in advance, and offer a way to request an adjustment.
• Keep records only under the organization's candidate-data retention and access rules.

This page provides a structured starting point. It is not legal advice, a validated assessment instrument or an automated recommendation. The organization remains responsible for job analysis, consistent administration, accommodations, record handling and the final human decision.

↑ Back to sections

Choose your next step