Find the load-bearing gaps

1. Time

How many hours per week can you protect for 8–12 weeks? Split the estimate into focused study, implementation, and reading/review. Are there weeks you already know will be constrained?

2. Cloud budget

What monthly Azure and model-API spend is comfortable, what is the hard ceiling, and may the programme use paid APIs? State whether you have access to an Azure subscription, GitHub Actions, and any employer-provided sandbox that is appropriate for public portfolio work.

3. Mathematical depth

Choose one and explain why: engineering-first (metrics and intuition), working theory (derive and calculate the core ideas), or mathematically rigorous (linear algebra, probability, and statistical derivations). Also describe your present comfort with vectors, cosine similarity, probability, and confidence intervals.

1. Python concurrency

Predict the approximate runtime and output order. Then rewrite main so the calls overlap while results remain in input order. Finally, explain how your design should behave if fetch(2) raises.

import asyncio

async def fetch(i: int) -> int:
    await asyncio.sleep(0.2)
    return i

async def main() -> list[int]:
    results: list[int] = []
    for i in range(4):
        results.append(await fetch(i))
    return results

Tests event-loop reasoning, structured concurrency, ordering, and failure semantics—not memorized syntax.

2. Python boundary design

An HTTP endpoint receives a JSON object representing a document update. Where would you use an untyped mapping, a Pydantic model, a dataclass, and a protocol or abstract interface? Sketch the types and boundaries. Explain what you validate at ingestion, what invariants belong in the domain, and which exceptions may cross the HTTP boundary.

3. Idempotent ingestion

A source emits (document_id, source_version, content). Delivery is at least once. A worker can crash after storing transformed chunks but before acknowledging the message. New versions can arrive out of order. Design the writes, idempotency key, transaction boundary, retry policy, and recovery process. What does “correct” mean here?

4. Retrieval measurement

For one query, the relevance labels of the top five retrieved chunks are [0, 1, 1, 0, 1], and the complete judged set contains four relevant chunks. Calculate Precision@3, Recall@3, and reciprocal rank. Which metric would expose a system that finds one excellent chunk first but misses most supporting evidence? Explain one limitation of evaluating chunks instead of source documents.

5. RAG failure analysis

A policy assistant gives a fluent, cited answer, but the cited document was superseded last week. List the distinct failure points you would investigate from source ingestion through final answer. For each, name one piece of evidence that would confirm or reject it. Include how retrieved prompt injection or cross-tenant context could appear.

6. Agent side effects

An agent may prepare and send a refund after human approval. The process can restart at any instruction, the approval can arrive hours later, and network timeouts make the refund API outcome ambiguous. Sketch the states and tool contract. Where are authorization, approval, idempotency, budget, timeout, audit, and compensation enforced? Explain why “the model was told to ask first” is insufficient.

7. Evaluation design

Separate the following into deterministic tests and probabilistic evaluations: schema-valid tool calls, permission enforcement, citation correctness, groundedness, answer usefulness, expected tool choice, acceptable trajectory, latency, and cost. Choose three metrics, define their denominators, and say how you would set regression thresholds without overfitting a small curated dataset.

8. Production incident

After a release, HTTP errors remain flat but p95 latency rises 70%, cost per resolved task doubles, and grounded-answer scores fall for one department. Give your first five investigative steps in order. Name the trace dimensions and version identifiers required to distinguish model, prompt, retrieval, data, orchestration, permissions, and traffic changes.