Jobs that loop

Most jobs do the same thing to many rows: remind every overdue invoice, re-score every open lead, archive every stale ticket. Written as one long loop, that job has three problems: it outlives its timeout on a big table, a crash halfway through starts it over, and starting over sends the first half's emails again.

ctx.job.each is the way to write it instead. You say which rows and what to do with one; the platform does the paging, the resuming and the duplicate protection.

export default async function remindOverdue(data, ctx) {
  const { sdk } = ctx;

  await ctx.job.each("invoices", { status: "overdue" }, async (invoice) => {
    await sdk.sendEmail("billing", {
      to: invoice.billing_email,
      subject: "Invoice {number} is overdue",
      body: ["Invoice {number} for {amount} was due on {due}."],
      data: { number: invoice.invoice_number, amount: invoice.amount, due: invoice.due_date },
    });
    await sdk.update("invoices", invoice.uid, { reminder_status: "sent" });
  });

  return { ok: true };
}

One firing, many turns

A firing is one occurrence of the schedule (or one Run now), carried to completion. It is made of turns: each turn is one run of your handler, from the top, and handles one page of each each loop.

1. The turn reads a page of matching rows (200 by default) and calls your function once per row.
2. If more rows remain, the turn ends right there. The platform saves where it stopped and the job is due again at once, behind every other app's waiting work.
3. The next turn runs your handler from the top again. each skips straight to the saved place.
4. When each reaches the end, the code after it runs, once, and the firing is complete.

So a job over a million rows is many short turns, interleaved with every other app's work, and a crash or a deploy costs at most the page in hand.

The rows

each(table, filters, fn, options) takes the same filters as sdk.readList, and pages in creation order with a sealed cursor, so rows added or changed during the firing do not shift its place.

• The rows are those within the job's reach. When the job acts for the owner, each row carries its org_uid.
• options.pageSize sets the page (default 200, at most 1,000); options.columns reads only the columns you name.
• A row that changes after its page was read is not read again in this firing.

Several loops in one job

Each each in a handler is a step, and the saved place records which step and where in it. A turn that finds step one finished skips it (the result says skipped: true) and carries on in step two.

await ctx.job.each("invoices", { status: "overdue" }, remind);
await ctx.job.each("subscriptions", { status: "past_due" }, suspend);

What each returns

{ done, failed, pages, failures, failuresTruncated }: rows whose function finished, rows whose function threw, pages read, and the first 100 failures (each a row uid and what went wrong). The same counts are recorded on the run.

A row that fails does not stop the job

If your function throws for a row, or runs past 60 seconds for one row, that row is recorded and skipped, and the rest carry on. One bad address never stops a mailing. A firing that visited every row with some failures ends partial ("197 done, 3 failed"), with the failed rows listed on the run. A row past the time limit is recorded with JOB_ITEM_TIMEOUT.

Throw for a row you could not handle; do not swallow the error. A swallowed error is a row recorded as done.

A turn that fails is retried from where it was

A turn fails when the handler throws outside a row's function, times out (the job's timeout is per turn), or cannot be run. The firing then waits and tries the same turn again, from the same saved place, after 1 minute, then 5, then 15. If the fourth attempt fails too, the firing stops, the owner is told once, and Resume carries on from where it stopped. See Runs, failures and Resume.

ctx.job.attempt is the number of failed attempts so far (0 on a first try), so a retried turn can tell it is one.

Nothing is done twice

A retried turn handles its page again. Rows the failed attempt already handled would be handled twice, so the platform makes the effects you cannot take back happen at most once per row:

• Email and Stripe calls made inside a row's function carry the row's item key. When a retried row asks to send an email it already sent, the app's email log already holds that message, so nothing is sent and sdk.sendEmail answers duplicate: true. "The same email" means the same connection, recipients, sender and subject, so a different email for the same row still goes out, and a send that failed is not counted as sent: the retry tries it again. A Stripe call passes the key to Stripe as its idempotency key.
• Writes inside your app's database are yours to make repeatable. Prefer writes that are harmless twice (set a status, record a timestamp), or wrap the row's work in sdk.withTransaction or sdk.withIdempotency, keyed by the firing and the row:

await ctx.job.each("orders", { status: "unbilled" }, async (order) => {
  await sdk.withIdempotency({ key: `bill:${ctx.job.windowStart}:${order.uid}` }, async () => {
    await sdk.create("charges", { order_uid: order.uid, amount_cents: order.total_cents });
  });
});

ctx.job.windowStart names the firing: every turn, every retry and every Resume of one firing shares it, and the next firing has a new one.

Loops each cannot express

For work that is not "every row of a table" (paging an outside API, walking dates), keep your own place with ctx.job.next. Returning ctx.job.next(position) ends the turn; the handler runs again with ctx.job.position set to that value (anything JSON can hold). Return without it when the work is finished.

export default async function syncCatalog(data, ctx) {
  const page = ctx.job.position ?? 1;
  const { items, hasMore } = await fetchCatalogPage(page); // your own outside call

  for (const item of items) {
    await ctx.sdk.updateWhere("products", { sku: item.sku }, { price_cents: item.priceCents });
  }

  if (hasMore) return ctx.job.next(page + 1);
  return { pages: page };
}

The same retry rules apply: a failed turn is retried from the last position a successful turn saved.

Limits

Job turns run in a lane of their own in the sandbox, so a backlog of jobs never takes the capacity your app's live requests need. Each turn counts toward your plan's request limit like any other run of a handler; while the plan's limit is reached, a turn is held, retried, and carries on once requests are allowed again.

Trying a job before you schedule it

A job-only handler gets a ctx.job when you run it from the console too (with no job or run uid), so you can dry-run or run live code written with ctx.job.each. A console try is a single turn: each loop handles its first page and stops. To see a whole firing, create the job and use Run now.

Next