weft

Owned, bounded structured concurrency.

A run is a list of tasks, a bound on how many of them may be in flight at once, and a complete account of what happened to every one of them. The account is the point: start hands back one Outcome per task, including the tasks that crashed, the tasks that were cancelled mid flight, and the tasks that never got a slot. There is no path through this module where a late failure discards work that already succeeded.

Why there is an extra process

The guarantee this module exists to make is no task outlives its scope, and no scope outlives its caller, and it is enforced by link propagation rather than by a collector loop that has to stay alive to do its job. A library that spawns unlinked workers and reaps them from a loop degrades to “no work outlives the VM” the moment something kills that loop.

So every run pays for one extra process, the scope, and the link topology does the work:

caller
  |  link          (process.spawn, which is proc_lib:spawn_link)
  v
scope              (traps exits)
  |  link   link   link
  v         v      v
 w0        w1     w2         (workers, which do NOT trap)

One asymmetry in that picture is worth stating because it shaped the code: a normal exit signal is ignored by a process that does not trap exits. So the links alone do not reap workers when the scope finishes cleanly; on every path where the scope stops early it kills its workers explicitly and waits for their EXITs before returning. settled is the predicate that encodes it, and it counts workers that have already answered but not yet exited (finishing) precisely so that the scope cannot return while one of its children is still alive.

Kill then join, in that order

Cancellation sends every kill first and only then waits for the EXITs. Interleaving the two lets one slow exit postpone every later worker’s cancellation, which turns a bounded teardown into a serial one. begin_cancel does the sending; the ordinary receive loop does the joining.

Delivery is pull, not push

fold consumes outcomes in completion order, and the streaming claim would be a lie if the scope pushed each outcome at the caller as it landed: a slow reducer would make the caller’s mailbox the buffer, and mailboxes are unbounded. Instead the scope holds outcomes and sends one only when the caller has asked for it. The caller replies Next after each outcome, so at most one outcome is ever in flight, and every Delivered message carries the scope’s own inbox so no handshake is needed to bootstrap the conversation.

Back pressure reaches the workers because a slot is occupied from the moment a task is spawned until its outcome has been delivered, not until the task returns. So limit bounds work in flight plus completed-but-unconsumed results together, and a reducer that stops reading stops the run from starting new work. The one place this bound is deliberately relaxed is cancellation, which materialises one NeverStarted per unstarted task in a single burst: those are three-word records and the task list was already a materialised list, so the cost is one that the caller had already paid.

Managed tasks, and what an owner’s exit proves

The contract above conflates two facts on purpose — for a leaf task, “the worker exited” and “the work stopped” are the same fact. They stop being the same fact the moment a task’s real work lives beyond its worker: an HTTP request whose socket belongs to a client library’s own supervisor, a database operation running under a pool. The worker can return while the process holding the socket is still alive, and killing the worker may kill the only process that still carries the child’s cancellation capability.

A managed task (prepared_task) splits the two facts apart. It carries, in addition to begin, a published owner pid — the process whose exit is the lifecycle truth for everything the task started — and a cancel closure that asks that subtree to stop. The scope monitors every owner before it spawns a single worker, so by the time begin can touch the outside world its ownership evidence is already on file; there is no window in which externally visible work exists and nobody holds proof of it. A managed task’s slot is then held until both its worker and its owner have exited, and its outcome is delivered only once both facts are in.

Only a normal owner exit proves the subtree drained. An abnormal one means the proof is gone — not that the work failed, but that nobody can any longer say whether it stopped — and that is a different outcome, DrainProofLost, kept apart from Crashed because a caller recovering resources must treat “unknown liveness” differently from “known death”. A prepared_leaf owner is the documented exemption: a task may declare that its owner provably owns nothing further, and then any exit completes it, so an ordinary crash of a leaf cannot manufacture a false DrainProofLost for work that never had descendants.

The scope’s own exit reason repeats the verdict outward: it exits normally only if every drain was proven, and abnormally when any proof was lost or left unconfirmed. That is what makes scopes compose — an outer witness monitoring this scope’s pid needs no protocol beyond the one monitors already speak, and a scope used as another run’s published owner propagates a lost proof without either side writing translation code.

Owners discovered while the task runs

prepared_task names its owner before the run starts. Work that only learns its owners as it goes — a request worker that prepares a transport and discovers the pid holding the socket — uses managed: its begin receives a Ledger, and adopt or adopt_leaf on it publishes one more owner to the scope, synchronously, with the same parked-work rule as before (adopt first, release on Adopted). A task may hold any number of owners; it is drained only when all of them are, and lost as soon as any one is. The ledger is a value and travels, so a process the worker started can publish on the worker’s behalf.

A late publication — one that arrives after cancellation began, or for a task whose account is already written — is Refused, but the owner is retained and asked to stop all the same. Refusal withholds the permit to start new work; it never withholds the witness, because an owner that exists must be drained whatever the run’s state.

start_witnessed is the run shape for a caller that wants only that witness: no outcomes are delivered, and the scope’s pid — alive exactly while any worker, owner or cancel helper is, exiting with the verdict — is the whole report. cancel_when_exits names a consumer whose death should end such a run when that consumer is not the caller.

Cancellation, and how Abandoned stays honest

A run can be stopped from three directions: on_failure(CancelSiblings) when a task fails, deadline when the wall clock runs out, and cancel_with when some other process decides to end a run the caller is blocked inside. All three converge on begin_cancel.

The scope cancels a worker with process.kill, so a cancelled worker’s exit reason is Killed — and so is the exit reason of a worker some unrelated process killed. Those two are different facts and the module keeps them apart with one boolean: a Killed exit after the scope initiated cancellation is Abandoned; a Killed exit before it is Crashed(Killed), because nobody in this run asked for it.

A managed owner is never killed. Killing it would destroy the one process whose exit still means something, so cancellation asks: the task’s cancel closure runs on a disposable helper process, once, idempotently, and the scope keeps waiting for the owner’s own exit. The helper is disposable for the same reason it exists — a cancel closure that crashes must cost the run a log line, not its witness. By default that wait is unbounded, because “cancelled” without drain proof is not a fact this module is willing to invent. cancel_grace bounds the wait for callers that need a bounded teardown: an owner still alive when the grace expires yields CancellationUnconfirmed — cancellation was requested and nobody proved it landed — and the scope, no longer able to vouch for the subtree, exits abnormally to say so.

Detached runs

start and fold block their caller, which is right for a function and wrong for an actor that must keep serving its mailbox. start_detached hands back a handle instead: pull collects one outcome at a time (the same pull-based protocol, so a slow consumer still throttles the run), cancel_detached requests cancellation without discarding the account, and scope_pid names the scope so an outer witness can monitor it — a normal exit of that pid proves the entire run, owners included, drained. start_relayed is the push adapter for consumers that are actors: a relay process owns the pulling and forwards each outcome as an ordinary message, trading the engine’s backpressure for the consumer’s mailbox, which is exactly the trade an actor’s receive loop already makes.

What the caller may rely on

When start returns, every task has an outcome and no worker from that run is alive. When fold returns — including when the reducer halted it early — the scope has killed and joined everything it spawned before replying, so the same holds. A caller that traps exits does not see the scope’s normal exit as mailbox noise, because the scope drops the link itself just before it returns; the link is live for the whole run, which is the part that matters.

Quick start

import weft

pub fn fetch_all(urls: List(String)) {
  let outcomes =
    urls
    |> list.map(fn(url) { fn() { fetch(url) } })
    |> weft.new
    |> weft.limit(8)
    |> weft.on_failure(weft.KeepGoing)
    |> weft.deadline(30_000)
    |> weft.start

  // Every url is accounted for: bodies for the fetches that succeeded,
  // and one Outcome per fetch that failed, crashed, or ran out of time.
  let #(bodies, rest) = weft.partition(outcomes)
  #(bodies, rest)
}

Types

The scope’s answer to an adoption.

pub type Adoption {
  Adopted
  Refused
}

Constructors

  • Adopted

    Custody crossed: the scope holds the owner under monitor and will wait for its exit. The caller may let the owner’s work begin.

  • Refused

    The run is cancelling, already sealed this task, or the scope is gone. The owner is still retained and asked to stop when a scope is there to do so — refusal withholds the permit to begin new work, never the witness — but the caller must not start anything under it.

An external stop signal. Any process holding one can end a run that another process is blocked inside, which is the case a handle-based cancel(task) cannot reach: the caller is the thing that is blocked.

A signal is a tiny process that does nothing but stay alive. cancel kills it, and every scope watching it sees the death. That is why one signal can serve several runs at once, and why a signal is one-shot: a killed process cannot come back, so a Cancel that has fired stays fired. Handing a spent signal to cancel_with cancels that run immediately, before any task starts.

pub opaque type Cancel

A handle to a run whose caller declined to block.

Handed back by start_detached. The holder collects outcomes with pull, requests cancellation with cancel_detached, and — the part a blocking caller never needs — can hand scope_pid to a monitor: the scope’s normal exit proves the entire run, managed owners included, drained, and an abnormal exit says some proof was lost or unconfirmed.

The handle’s outbox belongs to the process that called start_detached, so pull must be called from that same process. The scope is linked to that process too: if the holder dies, the scope survives the exit signal (it traps), cancels what is running, asks every managed owner to stop, and drains before exiting — a detached run abandoned by its holder tears itself down rather than leaking.

pub opaque type Detached(a, e)

A running managed task’s capability to publish owners to its scope.

Handed to the begin of a task built with managed. It may be sent to any process — the worker that received it, or a process the worker started and wants witnessed — and adopt/adopt_leaf on it are synchronous with the scope. The index rides inside so an adoption is charged to the task that discovered the owner, and the scope pid rides inside so a caller can tell a refusal from a scope that is already gone.

pub opaque type Ledger

What a run does to the remaining tasks when one of them fails.

pub type OnFailure {
  KeepGoing
  CancelSiblings
}

Constructors

  • KeepGoing

    Run everything to completion and report every outcome. The default.

  • CancelSiblings

    The first Failed or Crashed outcome cancels the rest: running tasks become Abandoned and unstarted ones become NeverStarted.

What happened to exactly one task.

Every task in a run produces exactly one of these, so a run is always fully accounted for and never collapses into a single Result that throws away the tasks which did succeed. The index is the task’s position in the list given to new, which is what lets start restore input order and what lets a caller match an outcome back to the work that produced it.

pub type Outcome(a, e) {
  Completed(index: Int, value: a)
  Failed(index: Int, error: e)
  Crashed(index: Int, reason: process.ExitReason)
  Abandoned(index: Int)
  NeverStarted(index: Int)
  DrainProofLost(index: Int, reason: process.ExitReason)
  CancellationUnconfirmed(index: Int)
}

Constructors

  • Completed(index: Int, value: a)

    The task returned Ok.

    Arguments

    index

    The task’s position in the list given to new, counting from zero.

    value

    The value the task returned.

  • Failed(index: Int, error: e)

    The task returned Error. The task itself decided this; nothing went wrong with the run.

    Arguments

    index

    The task’s position in the list given to new, counting from zero.

    error

    The error the task returned.

  • Crashed(index: Int, reason: process.ExitReason)

    The task’s process died instead of returning. The Erlang exit reason is carried rather than flattened to a string, so a caller can match on it.

    Arguments

    index

    The task’s position in the list given to new, counting from zero.

    reason

    Why the worker process exited.

  • Abandoned(index: Int)

    The task was started, then cancelled before it finished. Work was done and thrown away.

    Arguments

    index

    The task’s position in the list given to new, counting from zero.

  • NeverStarted(index: Int)

    The run ended before this task ever got a slot. No work was done.

    Arguments

    index

    The task’s position in the list given to new, counting from zero.

  • DrainProofLost(index: Int, reason: process.ExitReason)

    A managed task’s published owner exited abnormally, so nobody can any longer prove whether the task’s transitive work stopped. This is not a worker crash — the worker’s own fate is irrelevant once the proof is gone — and it is deliberately distinct from Crashed: a caller recovering resources must treat “unknown liveness” differently from “known death”.

    Arguments

    index

    The task’s position in the list given to new, counting from zero.

    reason

    How the owner exited. Killed and Abnormal both land here; so does an owner that was already dead when the scope went to monitor it, because a proof that was never on file was never proof.

  • CancellationUnconfirmed(index: Int)

    Cancellation was requested, the grace set by cancel_grace elapsed, and the task’s owner was still alive: nobody proved the cancellation landed. Only a run with a grace configured can produce this — without one the scope waits for the owner’s exit however long it takes.

    Arguments

    index

    The task’s position in the list given to new, counting from zero.

One task, prepared for a run: at minimum a begin closure, and for managed work also the published owner whose exit is the lifecycle truth for everything the task starts.

Built with task, prepared_task or prepared_leaf, and run with new_prepared. The owner must already be alive when the run starts — that is the parked work pattern: prepare the resource-holding process first, hand its pid here, and let begin release the actual work only once it runs inside the scope, which is by construction after the scope has the owner under monitor.

pub opaque type PreparedTask(a, e)

What one pull produced.

pub type Pulled(a, e) {
  PulledOutcome(outcome: Outcome(a, e))
  AllDelivered
  NotYet
  RunLost(reason: process.ExitReason)
}

Constructors

  • PulledOutcome(outcome: Outcome(a, e))

    One task’s outcome, in completion order.

    Arguments

    outcome

    The outcome pulled.

  • AllDelivered

    Every outcome has been delivered; the run is over and nothing it started is alive. Stop pulling: there is nothing left to pull, and a later pull will report the scope’s exit as RunLost rather than repeating this answer.

  • NotYet

    Nothing landed within the wait. The demand stands — the scope holds at most one granted delivery, so pulling again neither loses nor duplicates an outcome.

  • RunLost(reason: process.ExitReason)

    The scope died without saying Done: either something outside the run destroyed it, or its final exit carried the drain verdict for a run whose account was already delivered.

    Arguments

    reason

    How the scope exited.

A configured run, not yet started.

Built with new or new_prepared and refined with limit, on_failure, cancel_with, deadline and cancel_grace; start, fold and start_detached are the terminal verbs. A Run is an ordinary immutable value, so the same one can be started more than once.

pub opaque type Run(a, e)

A reducer’s verdict on whether to keep consuming outcomes.

This is fold’s answer to the problem a plain fn(acc, outcome) -> acc cannot express: a reducer that has seen enough has no way to say so, and the caller is left reaching for an external cancel signal to make a decision the reducer already made.

pub type Step(acc) {
  Continue(accumulator: acc)
  Halt(accumulator: acc)
}

Constructors

  • Continue(accumulator: acc)

    Keep going: ask the scope for the next outcome.

    Arguments

    accumulator

    The accumulator to carry into the next outcome.

  • Halt(accumulator: acc)

    Stop here: the remaining tasks are cancelled and the run is torn down before fold returns.

    Arguments

    accumulator

    The accumulator fold will return.

Values

pub fn adopt(
  ledger: Ledger,
  owner owner: process.Pid,
  cancel cancel: fn() -> Nil,
) -> Adoption

Publish a transitive owner to a running managed task’s scope.

Only a normal exit of owner will prove its subtree drained; any other exit settles the task as DrainProofLost. Synchronous: returns once the scope has the owner under monitor, or Refused at once if the scope has already exited. May be called from any process holding the ledger, any number of times, for as many owners as the task discovers.

Examples

case weft.adopt(ledger, owner: http_owner, cancel: stop_http) {
  weft.Adopted -> begin_http()
  weft.Refused -> Nil
}
pub fn adopt_leaf(
  ledger: Ledger,
  owner owner: process.Pid,
  cancel cancel: fn() -> Nil,
) -> Adoption

Publish a leaf owner to a running managed task’s scope: an owner that provably owns nothing further, completed by any exit. The prepared_leaf exemption, for owners discovered mid-run.

Examples

case weft.adopt_leaf(ledger, owner: pump, cancel: stop_pump) {
  weft.Adopted -> start_pump()
  weft.Refused -> Nil
}
pub fn cancel(signal: Cancel) -> Nil

Fire a cancel signal, ending every run watching it.

Firing a signal that has already fired does nothing. Runs watching the signal report Abandoned for the tasks that were running and NeverStarted for the tasks that had not begun.

Examples

let stop = weft.cancel_signal()
process.spawn(fn() { weft.new(tasks) |> weft.cancel_with(stop) |> weft.start })

// From any process, at any time, including while the run's caller is
// blocked inside `start`:
weft.cancel(stop)
pub fn cancel_detached(detached: Detached(a, e)) -> Nil

Cancel a detached run without giving up its account.

Unlike a fold reducer’s Halt, which discards what it has not seen, this keeps the deliveries coming: running tasks settle as Abandoned, unstarted ones as NeverStarted, and managed tasks by whatever their owners’ exits prove. Idempotent — cancelling a cancelled or finished run does nothing.

Examples

weft.cancel_detached(detached)
// ... keep pulling until AllDelivered ...
pub fn cancel_grace(run: Run(a, e), within: Int) -> Run(a, e)

Bound how long a cancellation waits for managed owners to exit, in milliseconds.

Without a grace, cancelling a managed task means asking its owner to stop and waiting for the owner’s exit however long that takes, because “cancelled” without drain proof is not a fact this module will invent. With one, an owner still alive when the grace expires settles as CancellationUnconfirmed, the run finishes bounded — and the scope exits abnormally, because it can no longer vouch for the subtree it was witnessing. The grace bounds the wait, and what it buys in boundedness it pays for in the scope’s exit verdict.

One grace timer serves the whole cancellation rather than one per task: it is an acknowledgement window on the teardown, not a per-task timeout, and it is armed when cancellation begins, from whichever direction it came.

Examples

let outcomes =
  weft.new_prepared(tasks)
  |> weft.deadline(30_000)
  |> weft.cancel_grace(2000)
  |> weft.start
pub fn cancel_signal() -> Cancel

Create a fresh cancel signal.

The returned value can be passed to cancel_with on any number of runs and shared with any number of processes. It costs one idle process, which lives until cancel is called on it.

Examples

let stop = weft.cancel_signal()

let outcomes =
  weft.new(tasks)
  |> weft.cancel_with(stop)
  |> weft.start
pub fn cancel_when_exits(
  run: Run(a, e),
  pid: process.Pid,
) -> Run(a, e)

Cancel the run when pid exits, for any reason.

This is the consumer’s death made a cancellation cause without the consumer having to be the process that started the run: a stream whose receiver is gone has nobody to deliver to and should stop its work. The watch is a monitor, never a link, so firing it ends the run and touches nothing else. A pid already dead when the run starts cancels it before any task is spawned, exactly like a spent cancel signal.

Examples

let witness =
  weft.new_prepared([weft.managed(serve)])
  |> weft.cancel_when_exits(consumer)
  |> weft.start_witnessed
pub fn cancel_with(run: Run(a, e), signal: Cancel) -> Run(a, e)

Watch a cancel signal for the duration of the run.

This is the only way to stop a run from outside the process that started it, because that process is blocked inside start or fold and cannot act on its own behalf.

Examples

let stop = weft.cancel_signal()

let outcomes =
  weft.new(tasks)
  |> weft.cancel_with(stop)
  |> weft.start
pub fn deadline(run: Run(a, e), within: Int) -> Run(a, e)

Give the whole run a wall-clock budget, in milliseconds.

Hitting the deadline is not an error, it is a reason some entries in the account are Abandoned (started, then cut short) or NeverStarted (never got a slot). The timer is cancelled and flushed if the run finishes first, so it can never fire into a later receive.

Examples

let outcomes =
  weft.new(tasks)
  |> weft.deadline(30_000)
  |> weft.start
pub fn failures(outcomes: List(Outcome(a, e))) -> List(e)

The errors of the tasks that returned Error, in the order given.

Tasks that crashed, were abandoned, or never started are not errors the tasks chose to return, so they appear here in no form at all; reach for partition when you need to see them.

Examples

let outcomes = [weft.Failed(0, "a"), weft.Crashed(1, process.Killed)]
assert weft.failures(outcomes) == ["a"]
pub fn first_ok(
  tasks: List(fn() -> Result(a, e)),
) -> Result(a, List(Outcome(a, e)))

Run tasks concurrently and return the first one to succeed, falling back to the full account if none did. The losers are killed and joined before a success returns.

A Failed or Crashed task does not end the run here; that is the whole point of the function, and it is why first_ok leaves on_failure at KeepGoing. An empty list is answerable, unlike in race, because the error channel already exists: it yields Error([]).

If something outside the run destroys the scope, the Error carries only the outcomes heard before the death rather than a total account; reach for start, which fills the gap in, when that distinction matters.

Examples

let body = weft.first_ok([fn() { fetch(mirror_a) }, fn() { fetch(mirror_b) }])
// -> Ok(<-body->)
// Nothing succeeded, so the caller gets the whole account in input order.
let answer = weft.first_ok([fn() { Error("a") }, fn() { Error("b") }])
// -> Error([weft.Failed(0, "a"), weft.Failed(1, "b")])
pub fn fold(
  run: Run(a, e),
  from initial: acc,
  with reducer: fn(acc, Outcome(a, e)) -> Step(acc),
) -> acc

Consume outcomes in completion order as they land.

This is how a run whose results do not all fit in memory is processed, and how a caller acts on early results without waiting for the slow tail. The scope holds each outcome until the reducer asks for it, so at most one is ever in flight and a slow reducer stops new work from starting rather than filling a mailbox.

A reducer returning Halt ends the run: the remaining tasks are killed and joined before fold returns, so no worker survives the call. The outcomes of those tasks are not delivered — fold returns an accumulator and has nowhere to put them — so a caller that wants the full account of a run it stopped early should use start with a cancel signal instead.

Examples

// Count successes without holding every value.
let successes =
  weft.new(tasks)
  |> weft.fold(from: 0, with: fn(count, outcome) {
    case outcome {
      weft.Completed(..) -> weft.Continue(count + 1)
      weft.Failed(..) | weft.Crashed(..) -> weft.Continue(count)
      weft.Abandoned(..) | weft.NeverStarted(..) -> weft.Continue(count)
      weft.DrainProofLost(..) -> weft.Continue(count)
      weft.CancellationUnconfirmed(..) -> weft.Continue(count)
    }
  })
// Stop as soon as three values are in hand; the rest are cancelled.
let first_three =
  weft.new(tasks)
  |> weft.fold(from: [], with: fn(found, outcome) {
    let found = case outcome {
      weft.Completed(value:, ..) -> [value, ..found]
      weft.Failed(..) | weft.Crashed(..) -> found
      weft.Abandoned(..) | weft.NeverStarted(..) -> found
      weft.DrainProofLost(..) | weft.CancellationUnconfirmed(..) -> found
    }
    case list.length(found) >= 3 {
      True -> weft.Halt(found)
      False -> weft.Continue(found)
    }
  })
pub fn limit(run: Run(a, e), max: Int) -> Run(a, e)

Set how many tasks may occupy a slot at once.

A slot is held from the moment a task is spawned until its outcome has been handed to the consumer, so this bounds running work and completed-but- unconsumed results together. A max below one is raised to one; there is no unbounded setting, by design.

Examples

let outcomes =
  weft.new(tasks)
  |> weft.limit(8)
  |> weft.start
pub fn managed(
  begin: fn(Ledger) -> Result(a, e),
) -> PreparedTask(a, e)

Prepare a managed task whose owners are discovered while it runs.

This is prepared_task for work whose owners do not exist yet when the run starts: a request worker that prepares a transport, learns the pid holding its socket, and must have that pid witnessed before the socket is allowed to open. begin receives a Ledger; every adopt or adopt_leaf on it publishes one more owner to the scope, and the task’s slot is held and its outcome withheld until the worker and every adopted owner have exited. A task that adopts nothing behaves exactly like task.

Publication is the parked-work pattern applied mid-run: prepare the owner parked, adopt it, and release it only on Adopted. A Refused means the run is already cancelling — the scope retains the owner and asks it to stop, so nothing leaks, but the caller must not begin the work.

Examples

let outcomes =
  weft.new_prepared([
    weft.managed(fn(ledger) {
      let #(owner, cancel, release) = prepare(request)
      case weft.adopt(ledger, owner:, cancel:) {
        weft.Adopted -> Ok(release())
        weft.Refused -> Error(Cancelled)
      }
    }),
  ])
  |> weft.start
pub fn map(
  items: List(a),
  limit max: Int,
  with fun: fn(a) -> Result(b, e),
) -> List(Outcome(b, e))

Apply a fallible function to every item, bounded, and return the account.

Exactly weft.new over the items with limit and start, which is the shape most callers want and the one it is easiest to get wrong by mapping an unbounded async over a list.

Examples

let outcomes = weft.map(urls, limit: 8, with: fetch)
let #(bodies, rest) = weft.partition(outcomes)
pub fn new(tasks: List(fn() -> Result(a, e))) -> Run(a, e)

Begin a run over tasks, bounded by default.

The default limit is the number of schedulers online, not unbounded: spawning one process per item is fine for pure computation and actively harmful for anything holding a socket, a file handle, a database connection or a rate-limited quota. Someone who wants a wider fan-out says so with limit.

The default failure policy is KeepGoing.

Examples

let outcomes =
  urls
  |> list.map(fn(url) { fn() { fetch(url) } })
  |> weft.new
  |> weft.start
pub fn new_prepared(tasks: List(PreparedTask(a, e))) -> Run(a, e)

Begin a run over prepared tasks, plain and managed mixed freely.

This is new for tasks built with task, prepared_task and prepared_leaf. The defaults are the same: a limit of the schedulers online, KeepGoing on failure, no deadline, and no cancellation grace — a cancelled managed task is awaited until its owner exits, however long that takes, unless cancel_grace bounds it.

Examples

let outcomes =
  weft.new_prepared([
    weft.task(fn() { fetch(url) }),
    weft.prepared_task(owner:, cancel:, begin:),
  ])
  |> weft.start
pub fn on_failure(run: Run(a, e), policy: OnFailure) -> Run(a, e)

Set what a failing task does to the tasks beside it.

Examples

// Stop at the first `Failed` or `Crashed`; the rest are reported as
// `Abandoned` or `NeverStarted` rather than being lost.
let outcomes =
  weft.new(tasks)
  |> weft.on_failure(weft.CancelSiblings)
  |> weft.start
pub fn partition(
  outcomes: List(Outcome(a, e)),
) -> #(List(a), List(Outcome(a, e)))

Split an account into the values that succeeded and everything else.

Both halves keep the order they were given in. The second half stays a list of Outcome rather than a list of errors, because “did not succeed” covers four different facts and flattening them is how the accounting gets thrown away one call site at a time.

Examples

let outcomes = [weft.Completed(0, "a"), weft.Failed(1, "no")]
assert weft.partition(outcomes) == #(["a"], [weft.Failed(1, "no")])
pub fn prepared_leaf(
  owner owner: process.Pid,
  cancel cancel: fn() -> Nil,
  begin begin: fn() -> Result(a, e),
) -> PreparedTask(a, e)

Prepare a managed task whose owner provably owns nothing further.

The one difference from prepared_task: any exit of the owner completes the drain obligation, normal or not. This is the declared exemption for single-process owners — an observer, a pump, a relay — where an ordinary crash is an ordinary crash and must not be read as a lost proof over descendants that never existed. Everything else — the monitor-before-begin ordering, the held slot, the cancel helper — is identical.

Examples

let outcomes =
  weft.new_prepared([
    weft.prepared_leaf(owner: pump, cancel: stop_pump, begin: run),
  ])
  |> weft.start
pub fn prepared_task(
  owner owner: process.Pid,
  cancel cancel: fn() -> Nil,
  begin begin: fn() -> Result(a, e),
) -> PreparedTask(a, e)

Prepare a managed task: work whose lifecycle outlives its worker.

owner is the pid whose exit proves the task’s transitive work is gone — only a normal exit proves it; any other exit becomes DrainProofLost. cancel asks the subtree to stop; it must be safe to call more than once and safe to call after the work has already finished, and it runs on a disposable helper so a crash inside it cannot cost the run its witness. begin is the worker’s blocking path, and it runs only after the scope holds the owner under monitor.

The task’s slot is held, and its outcome withheld, until both the worker and the owner have exited. An owner already dead when the run starts yields DrainProofLost without begin ever running: proof that was never on file was never proof.

Examples

// `prepare` parks the real request and hands back its owning pid, a
// cancel capability, and the closure that lets it loose.
let #(owner, cancel, begin) = prepare(request)

let outcomes =
  weft.new_prepared([weft.prepared_task(owner:, cancel:, begin:)])
  |> weft.cancel_grace(2000)
  |> weft.start
pub fn pull(
  detached: Detached(a, e),
  within timeout: Int,
) -> Pulled(a, e)

Collect one outcome from a detached run, waiting at most within milliseconds.

Must be called from the process that called start_detached — the handle’s outbox belongs to it. Demand is unary and idempotent: a pull that times out leaves its demand standing, and the next pull re-grants it harmlessly, so at most one outcome is ever in flight and none is ever dropped or doubled.

Examples

case weft.pull(detached, within: 1000) {
  weft.PulledOutcome(outcome) -> act_on(outcome)
  weft.NotYet -> check_something_else()
  weft.AllDelivered -> done()
  weft.RunLost(reason) -> escalate(reason)
}
pub fn race(
  first: fn() -> Result(a, e),
  rest: List(fn() -> Result(a, e)),
) -> Outcome(a, e)

Run tasks concurrently and return the first one to complete, however it completed. The losers are killed and joined before this returns.

The first task is a separate argument from the rest because a race over nothing has no answer: Outcome has no variant meaning “there was nothing to race”, and wrapping the result in a Result would push an error case onto every call site that already knows its list is not empty. Making the empty case unrepresentable is cheaper than making every caller handle it. A caller holding a runtime list matches on it once and decides for itself what nothing should mean.

Note the difference from first_ok: this returns the first task to finish, even if it finished by failing or crashing. Conflating the two is a common bug, which is why they are separate functions.

Examples

// Whichever mirror answers first, even if it answers with an error.
let outcome = weft.race(fn() { fetch(mirror_a) }, [fn() { fetch(mirror_b) }])
// -> weft.Completed(index: 1, value: <-body->)
pub fn scope_pid(detached: Detached(a, e)) -> process.Pid

The scope process behind a detached run.

This is the pid an outer witness monitors, and the pid a nested run publishes as its owner: a detached run handed to prepared_task as owner: weft.scope_pid(inner) composes ownership — the outer scope’s proof for that task is the inner scope’s own drain verdict, with no translation code on either side.

Examples

let inner = weft.new_prepared(children) |> weft.start_detached

let outer_task =
  weft.prepared_task(
    owner: weft.scope_pid(inner),
    cancel: fn() { weft.cancel_detached(inner) },
    begin: fn() { collect(inner) },
  )
pub fn start(run: Run(a, e)) -> List(Outcome(a, e))

Run everything to completion and return every outcome in input order.

Blocks the calling process until the run is over. When it returns, every task has exactly one outcome and no worker from the run is alive.

Ordered versus unordered falls out of which terminal verb you call rather than out of a flag: start sorts the account back into input order, fold hands outcomes over in completion order as they land.

Examples

let outcomes =
  weft.new([fn() { Ok(1) }, fn() { Error("no") }])
  |> weft.start
// -> [weft.Completed(0, 1), weft.Failed(1, "no")]
let #(values, rest) =
  weft.new(tasks)
  |> weft.limit(4)
  |> weft.on_failure(weft.CancelSiblings)
  |> weft.start
  |> weft.partition
pub fn start_detached(run: Run(a, e)) -> Detached(a, e)

Start a run without blocking, and hand back its handle.

The run is the same in every respect — same scope, same account, same ownership guarantees — only the consumption differs: nothing is delivered until the holder asks with pull, so the run’s backpressure now reaches whatever pace the holder pulls at.

Examples

let detached =
  weft.new_prepared(tasks)
  |> weft.cancel_grace(2000)
  |> weft.start_detached

let watch = process.monitor(weft.scope_pid(detached))
// ... pull outcomes as the mood takes you ...
pub fn start_relayed(
  run: Run(a, e),
  to sink: process.Subject(Pulled(a, e)),
) -> process.Pid

Start a run and push every outcome to sink as an ordinary message, from a relay process this function spawns and returns.

This is the adapter for consumers that are actors: an actor must not block its receive loop inside pull, so the relay owns the pulling and the actor merely receives. Each outcome arrives as PulledOutcome, followed by exactly one AllDelivered or RunLost, after which the relay exits normally.

What is traded away is stated plainly: push delivery makes the sink’s mailbox the buffer, so the engine’s backpressure ends at the relay. That is the same trade every actor’s mailbox already makes, and the limit still bounds how much work runs at once — only the finished outcomes queue without bound.

The relay is linked to the caller, and the scope to the relay, so the ownership chain survives: a dead consumer takes the relay with it, the scope traps that exit, cancels, drains its owners, and exits.

Examples

let relay =
  weft.new_prepared(tasks)
  |> weft.start_relayed(to: sink)
// The actor's selector now receives weft.Pulled(a, e) messages.
pub fn start_witnessed(run: Run(a, e)) -> process.Pid

Start a run whose only report is the scope’s exit, and hand back the scope’s pid.

Nothing is delivered to anyone: each outcome is discarded the moment it is sealed, and its slot returned. What remains is the part a drain witness needs — the scope is alive exactly while any worker, any adopted owner, or any cancel helper is, and it exits normally only if every proof landed. This is the shape for a run started purely to witness work: the caller monitors the returned pid, cancels through a signal or cancel_when_exits, and reads the verdict off the DOWN.

The scope is linked to the caller, as every scope is: a dead caller cancels the run, the owners are asked to stop, and the scope drains before it exits. Use cancel_when_exits to name a consumer that is not the caller.

Examples

let scope =
  weft.new_prepared([weft.managed(run_request)])
  |> weft.cancel_when_exits(consumer)
  |> weft.start_witnessed

let watch = process.monitor(scope)
// A normal DOWN proves everything the request started is gone.
pub fn task(begin: fn() -> Result(a, e)) -> PreparedTask(a, e)

Prepare a plain task: no owner, no drain obligation, exactly the behaviour of a closure given to new.

Examples

let outcomes =
  weft.new_prepared([weft.task(fn() { Ok(1) })])
  |> weft.start
pub fn values(outcomes: List(Outcome(a, e))) -> List(a)

The values of the tasks that returned Ok, in the order given.

Examples

let outcomes = [weft.Completed(0, 1), weft.Abandoned(1), weft.Completed(2, 3)]
assert weft.values(outcomes) == [1, 3]
Search Document