Workflows

A workflow is a job made of steps. Each finished step’s result is stored, so when an attempt fails, the next attempt skips straight past the steps that already succeeded. That makes long processes with several side effects safe to retry: an onboarding sequence, an order from payment to dispatch, a data import in stages.

Steps

Define a workflow with runnel.workflow and wrap each side effect in ctx.step(name, fn). The name must be unique within the workflow and stable between deploys.

import { runnel } from '@runnel/sdk';

export const onboard = runnel.workflow({
  name: 'onboard-customer',
  retries: { attempts: 10, maxDelay: '1h' },
  run: async (input: { userId: string }, ctx) => {
    const account = await ctx.step('create-account', () => createAccount(input.userId));
    await ctx.step('create-stripe-customer', () => billing.createCustomer(account.id));
    await ctx.step('send-welcome', () => sendWelcome(account.id));
    await ctx.sleep('wait-a-day', '1d');
    await ctx.step('send-tips', () => sendTipsEmail(account.id));
    return { accountId: account.id };
  },
});

If send-welcome throws, the next attempt runs the function again from the top, but create-account and create-stripe-customer return their stored results without being called. Only send-welcome runs again.

Warning Code outside a step runs on every attempt. Keep it to plain logic and reads; anything that writes, sends or charges belongs in a step.

Step results must be JSON-serialisable and under 256 KB. Store large results somewhere else and return the key.

Sleeping

ctx.sleep(name, duration) pauses the workflow for anything from a second to a year. No worker is busy while it sleeps: the run is parked, and a worker picks it up when the time comes, even if every worker has been redeployed in between. Sleeps do not count towards the timeout, which applies to each stretch of work between them.

Waiting for events

Since 2.6, a workflow can wait for something to happen elsewhere in your system, such as a manual approval or a payment callback.

const kyc = await ctx.waitForEvent('kyc-approved', {
  event: 'kyc.approved',
  match: { userId: input.userId },
  timeout: '7d',
});
if (!kyc) return ctx.step('close-application', () => closeApplication(input.userId));

Send the event from anywhere with the SDK or the API. Every waiting run whose match fields equal the event’s data resumes, and receives the event data as the result. After the timeout, waitForEvent returns null.

await runnel.events.send('kyc.approved', { userId: 'usr_311', reviewer: 'm.okafor' });

Replaying from a step

Sometimes a workflow fails for good because of a bug, not a flaky provider. You fix the bug and deploy, and then want to carry on where it stopped. Since 2.8 you can retry a failed run from any step: earlier steps keep their stored results, and the chosen step and everything after it run again on the current version of your code.

runnel runs retry run_01JB7X2K9Q4MZ8 --from-step send-welcome

The same option is on the run page in the dashboard. Replaying from an earlier step than the one that failed is allowed; the steps in between run again, so they should be safe to repeat.

Python and Go

@runnel.workflow(name="onboard-customer")
async def onboard(input: dict, ctx) -> dict:
    account = await ctx.step("create-account", lambda: create_account(input["user_id"]))
    await ctx.step("send-welcome", lambda: send_welcome(account.id))
    await ctx.sleep("wait-a-day", "1d")
    return {"account_id": account.id}

var Onboard = runnel.NewWorkflow("onboard-customer", func(ctx runnel.Context, in Input) error {
	account, err := runnel.Step(ctx, "create-account", func() (Account, error) { return createAccount(in.UserID) })
	if err != nil {
		return err
	}
	if err := runnel.Sleep(ctx, "wait-a-day", 24*time.Hour); err != nil {
		return err
	}
	_, err = runnel.Step(ctx, "send-welcome", func() (bool, error) { return true, sendWelcome(account.ID) })
	return err
})

Changing a workflow that has runs in flight

Runs that are sleeping or waiting resume on whatever version is deployed when they wake. Adding steps at the end is safe. Renaming or removing a step that runs have already passed is also safe: its stored result is simply not read. Reordering steps that are still ahead of sleeping runs changes what those runs do next, so check the dashboard’s “waiting” filter before you deploy such a change. More in Core concepts.