A background worker is a long-running process that performs work outside the request that created it. The web app records a job, returns a response to the user, and a worker picks up that job from a queue or database.
This is useful when the task is too slow, unreliable, or expensive to keep a user waiting for it. Sending email, generating a PDF, resizing media, importing a large file, and calling a slow third-party API are common examples.
The parts have different jobs
| PART | WHAT IT DOES | HOW LONG IT RUNS |
|---|---|---|
| Web process | Accepts the request and creates a job | Continuously, but each request should finish quickly |
| Job | Describes one unit of work | Until it succeeds, fails, or expires |
| Queue | Stores and hands out pending jobs | Durably, outside one app process |
| Worker | Claims a job and performs it | Continuously |
| Cron scheduler | Creates work at a set time | Wakes on a schedule or stays active in a process |
A job is not a worker. The job is the instruction. The worker is the process that waits for instructions. A queue connects the process that asks for work to the process that performs it.
One order, without and with a worker
Without a worker, a checkout request might charge the card, create an invoice PDF, send two emails, update a CRM, and then respond. Any slow dependency keeps the customer staring at a spinner, and one timeout can make the whole request look failed.
- 01The API validates the orderIt performs only the work required to accept or reject the purchase.
- 02The API records jobsInvoice, email, and CRM tasks enter a durable queue with an order identifier.
- 03The customer gets a responseThe request finishes without waiting for the slow follow-up work.
- 04The worker claims each jobIt performs the task, records success, and retries failures according to policy.
- 05Exhausted work becomes visibleA dead-letter queue or failed state keeps a bad job from disappearing or retrying forever.
Worker, cron job, or serverless function
| USE | WHEN THE WORK BEGINS | BEST FIT |
|---|---|---|
| Background worker | When a queue or event has work | Continuous processing and low pickup delay |
| Cron job | At a known time | Reports, cleanup, and periodic synchronization |
| Serverless function | When a request or event invokes it | Bursty, short, stateless work |
| Workflow engine | Across several dependent steps | Long processes with retries, waits, and branching |
A worker can also own a schedule, but that does not make every scheduled task a queue. Likewise, a function can consume a queue without staying alive between messages. The useful distinction is who owns the waiting and where unfinished work survives.
Reliability comes from the queue contract
- A job remains available if a worker crashes before acknowledging it.
- Retries have a limit and delay, rather than an immediate infinite loop.
- The handler is idempotent, so processing the same job twice is safe.
- Long jobs expose progress or a heartbeat.
- Failed jobs remain inspectable instead of being deleted.
- Queue depth and oldest-job age are monitored.
Many queues provide at-least-once delivery rather than a promise that code runs exactly once. That is why idempotency belongs in the application. Use a stable job or business identifier to detect a repeated attempt.
What the worker needs from hosting
A worker does not need a public port when it only connects outward to Redis, RabbitMQ, a database, or an API. It does need a process that stays awake, restarts after a crash, receives shutdown signals, and exposes logs. Local disk should be treated as temporary unless the host explicitly guarantees persistence.
Spocket can run the Node or Python worker process, but it does not include the queue or database. Bring that connection string as an environment variable. The /run/workers page covers the runtime, while /blog/serverless-vs-always-on helps if the job volume is bursty enough that a function may cost less.