The AI provider is slow. Does dispatch stop?
An AI-written proposal puts digest generation before notification acknowledgement.
The question
Northline’s transactional notifications confirm parcel handoffs. Each of 1,000 workspaces can request one optional AI digest of its day’s events. The draft instead calls the digest provider synchronously from every notification request.
| Normal | Slow period | |
|---|---|---|
| Notification arrivals per second | 100 | 100 |
| Provider call occupancy, seconds | 0.2 | 8 |
| Notification workers | 80 | 80 |
| Provider request price, USD | 0.006 | 0.006 |
| Daily provider budget, USD | 30 | 30 |
| Daily digest requests, one per workspace | 1000 | 1000 |
Authored constant-rate occupancy model. No actual vendor price or latency percentile is claimed.
Transactional notification acknowledgement must not require an AI digest. Digests may remain pending and may be generated once per workspace from the durable event stream later. The daily provider budget is $30: at most 5,000 provider attempts/day, including retries.
- DraftKeep one worker occupied until the AI digest returns. Retry the provider twice after timeout.
- OperationsThe provider has no spare capacity commitment during an incident. Do not send duplicate calls merely to hide a pending state.
“Move the dependency boundary. Bound enrichment work and defend the visible degraded state.”
Not saved by this site. Your browser may restore text when you navigate back.
A strong answer, step by step
Open each step in turn. This is one strong answer, not the only one.
Step 1: Find the occupancy change
What would you do here? Open the move you’d make.
Add a longer timeout to reduce visible provider errors.
Tempting, but it misses something. The longer occupancy holds scarce workers for even longer.
Move optional enrichment off the transactional acknowledgement path.
The strong move. The product contract expressly permits pending digests.
See how a strong candidate says it
“At 100 calls/second × 8 seconds, the synchronous path needs 800 concurrent calls; it has only 80 workers.”
Why it works: The constant-rate calculation exposes a capacity mismatch. It does not predict a latency percentile.
Step 2: Bound the optional lane
What would you do here? Open the move you’d make.
Admit digest work within a finite budget and leave excess pending.
The strong move. Transactional delivery remains independent and optional backlog stays visible.
Launch an unbounded background call per notification.
Tempting, but it misses something. Background work can still exhaust connections, money or provider capacity.
See how a strong candidate says it
“Use a durable event reference, a bounded enrichment queue and a fixed in-flight provider limit.”
Why it works: A queue transfers waiting; it must have a retention limit and an overload policy.
Step 3: Count attempts in the budget
What would you do here? Open the move you’d make.
Budget 5,000 distinct digests plus unlimited retries.
Tempting, but it misses something. Attempt spend can exceed the stated daily limit.
Apply one daily attempt budget and retry only useful transient failures.
The strong move. The cost boundary includes all attempts.
See how a strong candidate says it
“$30 ÷ $0.006 permits 5,000 provider attempts per day. The desired 1,000 daily digests cost $6 before retries; retries consume the remaining attempt budget.”
Why it works: Two extra attempts for every job would reduce the number of distinct jobs that fit the budget.
Step 4: Show the pending state
What would you do here? Open the move you’d make.
Test slow provider behavior and recovery without duplicate digests.
The strong move. The test matches the dependency and product state that can fail.
Report success for both parts as soon as the event is queued.
Tempting, but it misses something. Queued optional work has not produced the promised digest yet.
See how a strong candidate says it
“Acknowledge transactional delivery independently; show digest pending, and retry from the saved event after the provider recovers.”
Why it works: Do not display an invented digest or report enrichment complete when it is only queued.
The answer, in short
- Decision
- Keep transactional delivery independent; bound and expose optional enrichment.
- Because
- The product permits delayed digests and supplies finite worker and dollar budgets.
- What would change it
- If a digest becomes part of the required transactional result, the contract, capacity and fallback decision must be redesigned.
Common traps
- Calling unbounded work safe because it runs in the background.
- Hiding pending enrichment as completed content.
Check your answer
- Did I respect the optional boundary?
- Did I calculate occupancy with units?
- Do retries count in the dollar budget?
- Is pending different from complete?
Last reviewed Nov 10, 2026