.NET background jobs as Docker workers: fix bugs without a full release

Picture a bug in the code that generates PDF proposals. The fix is three lines. But the PDF code lives inside your main application, so shipping those three lines means a full release: build everything, test everything, deploy everything, and hope nothing else breaks on the way.

On an ad-tech SaaS platform our team works on, we got tired of that. So we moved background jobs out of the main application and into small, single-purpose Docker workers. Today a fix to one worker deploys on its own, without touching the API or anything else.

This post explains the pattern (a job queue, an orchestrator and worker containers), why it pays off, and what to decide if you build one for your own .NET application.

The idea in one sentence

The API doesn't do heavy background processing. It records a job, and a separate worker does it.

Three pieces make that happen:

• The API accepts the request and stores a job in a jobs table, which acts as the job queue.
• The orchestrator is a long-running service that picks up queued jobs and starts the right worker.
• Workers are small .NET console apps, each handling one kind of task, packaged as Docker images.

How the job queue and orchestrator work

Client ──► API ──► jobs table ──► Orchestrator ──► docker run <worker> ──► Worker
                   (pending)                                                 │
                        ▲                                                    │
                        └──────────── status: completed / failed ◄───────────┘

1. The client calls the API. For example: "generate the proposal PDF for campaign 123."
2. The API stores a job with the name of the worker that should handle it (say, pdf-worker), the input it needs, and a status of pending. It returns straight away, often with the job ID so the client can check progress.
3. The orchestrator picks up the job. It sees a pending job for pdf-worker and starts that worker's Docker container.
4. The worker does its job. It reads the job, generates the PDF, updates the database, calls the external service, whatever its task is.
5. The worker exits and the job is closed. The job is marked completed, or failed with the error, and the container goes away.

On our platform this pattern runs the background jobs that generate proposals, render PDFs, take screenshots, send emails and call LLMs.

What a .NET worker console app looks like

A worker is deliberately boring: a .NET console app that starts, processes one job and exits. A simplified sketch:

// Program.cs: a worker handles exactly one kind of background job
var jobId = Guid.Parse(args[0]);

await using var db = new AppDb(Environment.GetEnvironmentVariable("DATABASE_URL"));
var job = await db.Jobs.GetAsync(jobId);

try
{
    var pdf = await ProposalPdf.RenderAsync(job.Payload);
    await db.Proposals.SavePdfAsync(job.Payload.ProposalId, pdf);
    await db.Jobs.MarkCompletedAsync(jobId);
    return 0;
}
catch (Exception ex)
{
    await db.Jobs.MarkFailedAsync(jobId, ex.Message);
    return 1;
}

It has no web server and no shared state with the API, and it only knows about its own task. That smallness is the whole point.

Why Docker workers beat background jobs inside the main app

1. Fix one thing, deploy one thing. A bug in PDF generation means rebuilding and pushing one Docker image. The next PDF job runs the fixed code. The API, the other workers and the rest of the platform don't change, so there's no full release, nothing else to retest, and nothing else that can break.

2. A crash stays small. If a worker hits a bad input and dies, one job fails with a clear error. The API keeps answering, and every other kind of background job keeps running.

3. Each task gets the resources it needs. Screenshots need a headless browser. LLM calls wait on a slow external API. PDFs want memory. As separate containers, each worker gets its own image, dependencies and limits, instead of the main app carrying all of them.

4. Smaller code is easier to change. A new developer can understand a worker in an afternoon. Adding a new kind of background task means adding a new worker, not threading more code through the main application.

5. You're not locked into one language. The orchestrator only starts containers, so a worker could be written in Python if a library only exists there. In practice we keep almost everything in .NET, but the option is there.

Designing the job queue: retries, timeouts and idempotency

The pattern is simple, and the details are where it succeeds or fails:

• Job states. At minimum pending → running → completed / failed, with timestamps. It makes stuck jobs easy to find.
• Retries. Which failures are worth retrying (a timeout from an external API) and which aren't (invalid input)? Set a maximum number of attempts.
• Timeouts. A worker that hangs should be stopped and its job marked failed, not left holding resources forever.
• Idempotent workers. Retries mean a job can run more than once. Design workers so a second run doesn't send a second email or charge a card twice.
• Versioned Docker images. Tag worker images with versions, not just latest. Rolling back a bad fix is then as quick as pointing to the previous tag.
• Logs per job. Store each worker's output against the job ID. When a client asks why their PDF failed, you want the answer in one place.
• Concurrency limits. Decide how many copies of each worker can run at once, so a burst of jobs doesn't overwhelm the server or an external API.

When not to use container workers

Starting a container takes a moment, usually somewhere between well under a second and a few seconds. That's fine for PDFs, reports, emails and AI calls, where users already expect to wait. It's the wrong tool for work that must finish inside a normal page request.

And if your application is small and changes rarely, an in-process option such as Hangfire or a hosted BackgroundService in ASP.NET Core is simpler, and a perfectly good choice. Container workers earn their keep when background jobs are heavy, varied, or need fixing often.

The takeaway

Moving background jobs into small Docker workers turned "we need a full release for this" into "we'll push the fixed worker." Fixes ship faster, failures stay contained, and each piece of the system stays small enough to understand.

If your platform runs heavy background processing inside the main application and every fix feels risky, talk to us. We design and build this kind of architecture as part of our SaaS development and DevOps & cloud work.