A close-up portrait of a man with a salt-and-pepper beard wearing a white collared shirt against a textured beige background.
A close-up portrait of a man with a salt-and-pepper beard wearing a white collared shirt against a textured beige background.

Slow work belongs in a job

Jesse James Richard
|
Apr 24, 2026
|
9 min read
#Architecture
#Data

Giant Context does a great deal of work that no one is waiting on. A customer asks for a page and the model takes minutes to build it. A new customer signs up and their whole existing site has to be crawled and imported. On its own schedule, with no one asking, the platform wakes to do work of its own. None of that fits inside the web request that set it off. A browser will not hold a connection open for minutes. The recurring work has no browser behind it at all. The work has to outlive the request that started it, which means it cannot be a request. It has to be a job.

The version that loses quietly

The quick way to start a job is to fire it and forget it. The code that needs the work makes a call to whatever will do it, does not wait, and returns. The page request kicks off the build and hands the user a spinner. The signup starts the import and moves on. It works on the first try, which is the only condition it survives.

It stops working the moment the call it fired does not land. The service is briefly down, the process handling the work restarts mid-flight, or the network drops the one second that matters. The work never happens, with nothing to record that it was supposed to. There is no error to see, because the code that would have raised one already returned. The import quietly does not finish. The first anyone hears of it is a customer asking where their site went.

The table that first fixed this in the codebase says so in its own header:

Replaces fire-and-forget HTTP calls that silently drop on transient errors.

Fire-and-forget is a way of losing work without ever finding out.

A job is a row in a table

The fix is to stop treating the job as a call and start treating it as a record. Before the work runs, a row goes into a table, written in the same transaction as whatever asked for the work. The row says what to do and that it is not yet done. From then on the truth about the job lives in the database, where a dropped connection and a restarted process cannot reach it.

Something still has to pick the row up and run it. A worker claims the next pending job, does the work, and marks the row done, or marks it failed and leaves it for a later attempt. The care is all in the claim, because more than one worker runs at once and no two of them can take the same row. This is the claim:

Claiming the next job

UPDATE ai.embedding_jobsSET status = 'processing',    attempts = attempts + 1,    last_attempted_at = NOW(),    updated_at = NOW()WHERE id = (    SELECT id FROM ai.embedding_jobs    WHERE status = 'pending' AND next_attempt_at <= NOW()    ORDER BY next_attempt_at ASC    FOR UPDATE SKIP LOCKED    LIMIT 1)

FOR UPDATE SKIP LOCKED is the whole trick. Ten workers can run this exact statement at the same instant, each taking a different job, none blocked, none colliding. The retry lives in the same row. A failed job has its next_attempt_at pushed forward and its attempts counted, so that the loop picks it up again later and gives up only after enough tries. There is no Redis here, no queue server, no new service to run. The database that was already there is enough to be the queue.

The tick that runs them

A table of pending rows does nothing on its own. Something has to run on a repeating basis, look for ready work, and do a pass of it. That repeating pass is the tick. The platform runs two kinds of it.

The first is the scheduled tick. A route on the server does one pass of work. A scheduler calls that route on a fixed clock, every few minutes or once a day. This is how recurring work runs, the maintenance that happens whether or not anyone asked, and how the platform's own autonomous work is driven, the same shape as the job that purges deleted files or the one that checks a customer domain's DNS.

A scheduled tick has a floor built in. It runs only as often as the clock allows, so that a job arriving a second after a five-minute tick waits almost five minutes to begin. For work that was never in a hurry that is fine. For work that should run the instant it appears it is too slow. The codebase says as much where the other kind of tick is defined:

Why not a cron tick? [...] NOTIFY-driven dispatch is event-shaped: when work appears, it runs immediately.

The second kind is the event tick. A worker holds one long-lived Postgres connection open on a LISTEN. When a job is written, the same transaction that writes it fires a NOTIFY on that channel, the worker wakes, and it drains the ready work at once. No clock, no floor. Work runs when work exists. The two kinds answer the two shapes of need, the clock-driven and the event-driven.

Scheduled tick
Event tick

Runs on

A fixed clock, every few minutes or once a day

A LISTEN/NOTIFY signal, the moment work appears

Latency floor

Up to the interval, a job can wait almost five minutes

None. Work runs when work exists

Best for

Recurring maintenance nobody asked for

Work that should run the instant it appears

One job becomes many

Some work is a single row. The more demanding work is a graph. Importing a new customer's site is not one job. It is a crawl of their sitemap, then one job for each page it finds, then a step that cannot begin until every one of those pages is in. The unit is no longer a task but a run made of tasks that depend on one another.

The same claim holds, with one clause added. A task is ready only when every task it depends on has already succeeded:

A task waits on its dependencies

WHERE t.status = 'pending'  AND t.next_attempt_at <= now()  AND NOT EXISTS (    SELECT 1 FROM unnest(t.depends_on) dep    LEFT JOIN workflows.tasks d ON d.id = dep    WHERE d.id IS NULL OR d.status <> 'succeeded'  )

The worker takes only tasks whose dependencies are met, runs them in parallel, and leaves the rest pending until their turn arrives. A run is finished when its last task is. If the process dies with tasks half-run, nothing is lost, because the whole state is in the rows. A recovery tick passes through later to reclaim anything a dead worker left stuck. Every part of the breakdown is a row, so a crash cannot take it with it.

Where the slow work goes next

The heaviest jobs in the platform are the AI ones, still running the old way at this point, fired off with a call and a hope. They are what moves next. A model job that runs for many minutes across a dozen stages needs more than a pending row and a retry. It has to remember which stage it reached, so that an attempt resumes instead of starting from the top. It has to keep a record of what happened at each step. That is the same idea one level up, a durable job whose state is a row in Postgres and whose history is an append-only list of events. The job stops being a single claim and becomes a small state machine the database holds for it. The move is the same one, out of the request and into a row, made durable enough for work that runs for minutes and must survive being interrupted in the middle.

The part no one sees

None of this reaches the screen. A customer never sees a job table or a claim query. What they see is that the site they asked to import actually finished, that the page they generated came back, that work they never watched still happened. A job system is invisible when it works and impossible to miss the moment it does not.

For a platform run by one person, the need is sharper still. The product's real work is slow, autonomous, and recurring, with more of it moving that way every month as the model takes on more. Work that lives in a request can run only while someone waits, and vanishes without a trace when the request is gone. Work that lives in a durable job runs while no one watches, survives a crash, retries itself, and says so when it has finally failed. A platform that does slow work reliably can do things a request-shaped product cannot, and can do them while its one maintainer is asleep.

This generalizes past Giant Context. Every platform takes on work that outlives the request that asked for it, so every platform eventually needs a way to run jobs that carry on by themselves. The only real choice is when it gets built. Design it in early, while it costs a table and a claim query, or add it late, after fire-and-forget has already dropped work in production and a customer has noticed before you have. Async work has to be planned into a platform from the start, because a platform that waits until it needs a job system has already been losing the work it would have saved.

The marketing is built from your files

#AI & Agents
#Data
#Signature

A model handed nothing but a request invents, a warm testimonial no customer gave, a feature the product does not have. Giant Context builds a busines...

Jesse James Richard

|

Apr 25, 2026
Read next

Building something like this

I'm Jesse. I build platforms end to end, and I'm open to work. If this is the kind of engineering you need, get in touch.

Contact Jesse
Home
About
Contact
Sitemap
Privacy Policy
Terms of Service
Cookie Policy
Slow work belongs in a job | Jesse James Richard