Retry later without using an attempt

A busy provider is not always a broken job. Defer deliberately, but give the work a deadline so it cannot wait forever.

On this page

Set up

Use the framework/tools and disposable SQL configuration from Installation. The provider in this lab is a controlled readiness flag. The queue, claims, attempts and failure records are real. No HTTP request or provider key is needed.

SQL lab prerequisites

Copy the job and runner

Keep the durable deadline in the job data so it survives process restart. The example has no irreversible provider side effect.

bash

mkdir -p examples
cp -R node_modules/@db3.ai/app/src/queue/examples/. examples/

Run it

Expect two deferred outcomes, attemptsAfterDeferral: 0, then succeeded. An expired application deadline produces failed, with no queued work left and one retained failure.

bash

npx tsx examples/runBackpressure.ts

Distinguish deferral from failure

Throw QueueRetryLaterError(delaySeconds) only for deliberate backpressure. Queue releases the claim and restores the consumed attempt. An ordinary exception follows maxTries, backoff and retryUntilSeconds instead.

Intentional deferral bypasses ordinary retry exhaustion. Check an application deadline before deferring or the job can remain queued indefinitely. The example uses zero seconds only to prove the lifecycle without sleeps; a real provider needs a validated, bounded non-zero delay and usually jitter.

import { QueueableJob, QueueRetryLaterError } from '@db3.ai/app/queue';
import { app } from '@db3.ai/app/server';

/** Durable application deadline, independent of Queue's ordinary retry policy. */
export interface DeferredReportData extends Record<string, unknown> {
	/** Epoch milliseconds after which this report must fail instead of deferring. */
	deadline: number;
}

/** Demonstrates controlled provider backpressure without contacting a real API. */
export class DeferredReportJob extends QueueableJob<DeferredReportData> {
	static readonly jobName = 'reports.defer-example.v1';
	/** Validates the durable deadline on both producer and worker construction. */
	constructor(data: DeferredReportData) {
		if (!data || !Number.isSafeInteger(data.deadline) || data.deadline < 1) throw new Error('A positive deadline in epoch milliseconds is required.');
		super(data);
	}
	/** Applies the application deadline before retry-later can restore an attempt. */
	async handle(): Promise<void> {
		if (Date.now() >= this.data.deadline) throw new Error('Report deadline expired.');
		if (!app().config.get<{ ready: boolean }>('reportProvider')?.ready) throw new QueueRetryLaterError(0, 'Controlled provider backpressure.');
		// Real provider success would produce a result here. This lab has no charge,
		// HTTP request or irreversible side effect, so repeated handling is safe.
	}
}

Translate provider responses carefully

Classify the actual response. Rate limits and a documented Retry-After can justify deferral; invalid credentials or malformed input generally need repair, not endless retries. Validate delay units and upper bounds before throwing.

A timeout may happen after the provider accepted work. Do not blindly repeat a charge, email or generation without an idempotency/reconciliation policy. This recipe does not parse Retry-After or certify a provider’s behavior.

Prove recovery and expiry

The runner changes only the simulated external readiness, not a queued payload or attempt counter. The next real claim succeeds. A separately expired job proves the hard stop independently of readiness.

import { pathToFileURL } from 'node:url';
import { createGeneratedTestDatabase } from '@db3.ai/app/db/test/db';
import { FailedJob, QueuedJob } from '@db3.ai/app/queue';
import { App } from '@db3.ai/app/server';
import { DeferredReportJob } from './DeferredReportJob';

/** Exercises SQL-backed deferral, unchanged attempts, readiness recovery and a hard deadline. */
export async function runBackpressure() {
	const database = await createGeneratedTestDatabase('backpressure_guide');
	const reportProvider = { ready: false };
	const application = new App({ db: database.db, config: { reportProvider }, queue: { queue: 'reports', queueMonitor: false } });
	try {
		await application.db.install(QueuedJob, FailedJob);
		application.queue.registerJob(DeferredReportJob);
		const id = await application.queue.dispatch(new DeferredReportJob({ deadline: Date.now() + 60_000 }), { queue: 'reports', maxTries: 1 });
		const first = await application.queue.workNextJob('reports');
		const second = await application.queue.workNextJob('reports');
		const deferred = await QueuedJob.findOrFail(id);
		reportProvider.ready = true;
		const recovered = await application.queue.workNextJob('reports');
		await application.queue.dispatch(new DeferredReportJob({ deadline: 1 }), { queue: 'reports', maxTries: 1 });
		const expired = await application.queue.workNextJob('reports');
		return { first: first?.status, second: second?.status, attemptsAfterDeferral: deferred.attempts, recovered: recovered?.status, expired: expired?.status, remainingJobs: await QueuedJob.query().count(), failures: (await application.queue.failedJobs(10)).length };
	} finally { try { await application.close(); } finally { await database.destroy(); } }
}

if (process.argv[1] && import.meta.url === pathToFileURL(process.argv[1]).href) console.log(JSON.stringify(await runBackpressure(), null, 2));

Testing

Create a tests/queue directory and save the test below as runBackpressure.test.ts. It imports the example you copied into examples/. Keep the same folder layout so the relative import resolves.

Run from the application root with the development dependencies from Installation. These are consumer tests, not commands that assume a framework checkout. This database lab needs the same test-only SQL credentials when run through Vitest.

import { expect, it } from 'vitest';
import { runBackpressure } from '../../examples/runBackpressure';

it('defers without spending a try, recovers when ready and stops at the app deadline', async () => {
	expect(await runBackpressure()).toEqual({ first: 'deferred', second: 'deferred', attemptsAfterDeferral: 0, recovered: 'succeeded', expired: 'failed', remainingJobs: 0, failures: 1 });
});

Run and extend the test

Add another deferral and keep the attempt count at zero. Keep a deterministic expired deadline rather than waiting for a timer. Test real delay scheduling separately with the production driver.

bash

npx vitest run tests/queue/runBackpressure.test.ts
npx tsc --noEmit --target ES2022 --module ESNext --moduleResolution Bundler --types node --skipLibCheck examples/*.ts

Coverage

Tested: SQL claims/deferrals, preserved attempts, readiness recovery and an application deadline. Explained: ordinary retries, non-zero delay, idempotency and response classification. Follow-up: a provider-specific adapter recipe with Retry-After parsing.

Behaviour tested Executed by the documentation maintenance gate

What this does

Defers twice without consuming an attempt, completes after simulated readiness and terminates at an application deadline.

Expected outputThe guide test passes against the real framework components.

Behaviour testpackages/app/src/queue/tests/examples/runBackpressure.test.ts

This test command requires the framework repository. Use the walkthrough commands in an installed application.

Environment: Real SQL queue; controlled readiness flag, no network or paid API.