weft
Owned, bounded structured concurrency.
A run is a list of tasks, a bound on how many of them may be in flight at
once, and a complete account of what happened to every one of them. The
account is the point: start hands back one Outcome per task, including
the tasks that crashed, the tasks that were cancelled mid flight, and the
tasks that never got a slot. There is no path through this module where a
late failure discards work that already succeeded.
Why there is an extra process
The guarantee this module exists to make is no task outlives its scope, and no scope outlives its caller, and it is enforced by link propagation rather than by a collector loop that has to stay alive to do its job. A library that spawns unlinked workers and reaps them from a loop degrades to “no work outlives the VM” the moment something kills that loop.
So every run pays for one extra process, the scope, and the link topology does the work:
caller
| link (process.spawn, which is proc_lib:spawn_link)
v
scope (traps exits)
| link link link
v v v
w0 w1 w2 (workers, which do NOT trap)
- The caller spawns the scope linked to itself.
- The scope calls
process.trap_exits(True)and spawns every worker linked to itself, from inside the scope process. - A worker crashing therefore arrives at the scope as a trapped
EXITand becomes aCrashedoutcome. It does not take the scope down with it. - The scope being killed — including by a brutal supervisor kill — propagates down those links to workers that do not trap, so they die with it. Nothing has to still be looping for that to hold.
- The caller dying sends an
EXITto the scope, which traps it, kills its workers, waits for them, and exits.
One asymmetry in that picture is worth stating because it shaped the code:
a normal exit signal is ignored by a process that does not trap exits. So
the links alone do not reap workers when the scope finishes cleanly; on
every path where the scope stops early it kills its workers explicitly and
waits for their EXITs before returning. settled is the predicate that
encodes it, and it counts workers that have already answered but not yet
exited (finishing) precisely so that the scope cannot return while one of
its children is still alive.
Kill then join, in that order
Cancellation sends every kill first and only then waits for the EXITs.
Interleaving the two lets one slow exit postpone every later worker’s
cancellation, which turns a bounded teardown into a serial one. begin_cancel
does the sending; the ordinary receive loop does the joining.
Delivery is pull, not push
fold consumes outcomes in completion order, and the streaming claim would
be a lie if the scope pushed each outcome at the caller as it landed: a slow
reducer would make the caller’s mailbox the buffer, and mailboxes are
unbounded. Instead the scope holds outcomes and sends one only when the
caller has asked for it. The caller replies Next after each outcome, so at
most one outcome is ever in flight, and every Delivered message carries the
scope’s own inbox so no handshake is needed to bootstrap the conversation.
Back pressure reaches the workers because a slot is occupied from the moment
a task is spawned until its outcome has been delivered, not until the task
returns. So limit bounds work in flight plus completed-but-unconsumed
results together, and a reducer that stops reading stops the run from
starting new work. The one place this bound is deliberately relaxed is
cancellation, which materialises one NeverStarted per unstarted task in a
single burst: those are three-word records and the task list was already a
materialised list, so the cost is one that the caller had already paid.
Managed tasks, and what an owner’s exit proves
The contract above conflates two facts on purpose — for a leaf task, “the worker exited” and “the work stopped” are the same fact. They stop being the same fact the moment a task’s real work lives beyond its worker: an HTTP request whose socket belongs to a client library’s own supervisor, a database operation running under a pool. The worker can return while the process holding the socket is still alive, and killing the worker may kill the only process that still carries the child’s cancellation capability.
A managed task (prepared_task) splits the two facts apart. It
carries, in addition to begin, a published owner pid — the process
whose exit is the lifecycle truth for everything the task started — and
a cancel closure that asks that subtree to stop. The scope monitors
every owner before it spawns a single worker, so by the time begin
can touch the outside world its ownership evidence is already on file;
there is no window in which externally visible work exists and nobody
holds proof of it. A managed task’s slot is then held until both its
worker and its owner have exited, and its outcome is delivered only once
both facts are in.
Only a normal owner exit proves the subtree drained. An abnormal one
means the proof is gone — not that the work failed, but that nobody can
any longer say whether it stopped — and that is a different outcome,
DrainProofLost, kept apart from Crashed because a caller recovering
resources must treat “unknown liveness” differently from “known death”.
A prepared_leaf owner is the documented exemption: a task may declare
that its owner provably owns nothing further, and then any exit
completes it, so an ordinary crash of a leaf cannot manufacture a false
DrainProofLost for work that never had descendants.
The scope’s own exit reason repeats the verdict outward: it exits normally only if every drain was proven, and abnormally when any proof was lost or left unconfirmed. That is what makes scopes compose — an outer witness monitoring this scope’s pid needs no protocol beyond the one monitors already speak, and a scope used as another run’s published owner propagates a lost proof without either side writing translation code.
Owners discovered while the task runs
prepared_task names its owner before the run starts. Work that only
learns its owners as it goes — a request worker that prepares a
transport and discovers the pid holding the socket — uses managed:
its begin receives a Ledger, and adopt or adopt_leaf on it
publishes one more owner to the scope, synchronously, with the same
parked-work rule as before (adopt first, release on Adopted). A task
may hold any number of owners; it is drained only when all of them are,
and lost as soon as any one is. The ledger is a value and travels, so a
process the worker started can publish on the worker’s behalf.
A late publication — one that arrives after cancellation began, or for
a task whose account is already written — is Refused, but the owner
is retained and asked to stop all the same. Refusal withholds the
permit to start new work; it never withholds the witness, because an
owner that exists must be drained whatever the run’s state.
start_witnessed is the run shape for a caller that wants only that
witness: no outcomes are delivered, and the scope’s pid — alive exactly
while any worker, owner or cancel helper is, exiting with the verdict —
is the whole report. cancel_when_exits names a consumer whose death
should end such a run when that consumer is not the caller.
Cancellation, and how Abandoned stays honest
A run can be stopped from three directions: on_failure(CancelSiblings)
when a task fails, deadline when the wall clock runs out, and
cancel_with when some other process decides to end a run the caller is
blocked inside. All three converge on begin_cancel.
The scope cancels a worker with process.kill, so a cancelled worker’s exit
reason is Killed — and so is the exit reason of a worker some unrelated
process killed. Those two are different facts and the module keeps them
apart with one boolean: a Killed exit after the scope initiated
cancellation is Abandoned; a Killed exit before it is
Crashed(Killed), because nobody in this run asked for it.
A managed owner is never killed. Killing it would destroy the one
process whose exit still means something, so cancellation asks: the
task’s cancel closure runs on a disposable helper process, once,
idempotently, and the scope keeps waiting for the owner’s own exit. The
helper is disposable for the same reason it exists — a cancel closure
that crashes must cost the run a log line, not its witness. By default
that wait is unbounded, because “cancelled” without drain proof is not a
fact this module is willing to invent. cancel_grace bounds the wait
for callers that need a bounded teardown: an owner still alive when the
grace expires yields CancellationUnconfirmed — cancellation was
requested and nobody proved it landed — and the scope, no longer able to
vouch for the subtree, exits abnormally to say so.
Detached runs
start and fold block their caller, which is right for a function and
wrong for an actor that must keep serving its mailbox. start_detached
hands back a handle instead: pull collects one outcome at a time (the
same pull-based protocol, so a slow consumer still throttles the run),
cancel_detached requests cancellation without discarding the account,
and scope_pid names the scope so an outer witness can monitor it — a
normal exit of that pid proves the entire run, owners included, drained.
start_relayed is the push adapter for consumers that are actors: a
relay process owns the pulling and forwards each outcome as an ordinary
message, trading the engine’s backpressure for the consumer’s mailbox,
which is exactly the trade an actor’s receive loop already makes.
What the caller may rely on
When start returns, every task has an outcome and no worker from that run
is alive. When fold returns — including when the reducer halted it early —
the scope has killed and joined everything it spawned before replying, so
the same holds. A caller that traps exits does not see the scope’s normal
exit as mailbox noise, because the scope drops the link itself just before
it returns; the link is live for the whole run, which is the part that
matters.
Quick start
import weft
pub fn fetch_all(urls: List(String)) {
let outcomes =
urls
|> list.map(fn(url) { fn() { fetch(url) } })
|> weft.new
|> weft.limit(8)
|> weft.on_failure(weft.KeepGoing)
|> weft.deadline(30_000)
|> weft.start
// Every url is accounted for: bodies for the fetches that succeeded,
// and one Outcome per fetch that failed, crashed, or ran out of time.
let #(bodies, rest) = weft.partition(outcomes)
#(bodies, rest)
}
Types
The scope’s answer to an adoption.
pub type Adoption {
Adopted
Refused
}
Constructors
-
AdoptedCustody crossed: the scope holds the owner under monitor and will wait for its exit. The caller may let the owner’s work begin.
-
RefusedThe run is cancelling, already sealed this task, or the scope is gone. The owner is still retained and asked to stop when a scope is there to do so — refusal withholds the permit to begin new work, never the witness — but the caller must not start anything under it.
An external stop signal. Any process holding one can end a run that another
process is blocked inside, which is the case a handle-based cancel(task)
cannot reach: the caller is the thing that is blocked.
A signal is a tiny process that does nothing but stay alive. cancel kills
it, and every scope watching it sees the death. That is why one signal can
serve several runs at once, and why a signal is one-shot: a killed
process cannot come back, so a Cancel that has fired stays fired. Handing a
spent signal to cancel_with cancels that run immediately, before any task
starts.
pub opaque type Cancel
A handle to a run whose caller declined to block.
Handed back by start_detached. The holder collects outcomes with
pull, requests cancellation with cancel_detached, and — the part a
blocking caller never needs — can hand scope_pid to a monitor: the
scope’s normal exit proves the entire run, managed owners included,
drained, and an abnormal exit says some proof was lost or unconfirmed.
The handle’s outbox belongs to the process that called start_detached,
so pull must be called from that same process. The scope is linked to
that process too: if the holder dies, the scope survives the exit signal
(it traps), cancels what is running, asks every managed owner to stop,
and drains before exiting — a detached run abandoned by its holder tears
itself down rather than leaking.
pub opaque type Detached(a, e)
A running managed task’s capability to publish owners to its scope.
Handed to the begin of a task built with managed. It may be sent to
any process — the worker that received it, or a process the worker
started and wants witnessed — and adopt/adopt_leaf on it are
synchronous with the scope. The index rides inside so an adoption is
charged to the task that discovered the owner, and the scope pid rides
inside so a caller can tell a refusal from a scope that is already gone.
pub opaque type Ledger
What a run does to the remaining tasks when one of them fails.
pub type OnFailure {
KeepGoing
CancelSiblings
}
Constructors
-
KeepGoingRun everything to completion and report every outcome. The default.
-
CancelSiblingsThe first
FailedorCrashedoutcome cancels the rest: running tasks becomeAbandonedand unstarted ones becomeNeverStarted.
What happened to exactly one task.
Every task in a run produces exactly one of these, so a run is always fully
accounted for and never collapses into a single Result that throws away
the tasks which did succeed. The index is the task’s position in the list
given to new, which is what lets start restore input order and what lets
a caller match an outcome back to the work that produced it.
pub type Outcome(a, e) {
Completed(index: Int, value: a)
Failed(index: Int, error: e)
Crashed(index: Int, reason: process.ExitReason)
Abandoned(index: Int)
NeverStarted(index: Int)
DrainProofLost(index: Int, reason: process.ExitReason)
CancellationUnconfirmed(index: Int)
}
Constructors
-
Completed(index: Int, value: a)The task returned
Ok.Arguments
- index
-
The task’s position in the list given to
new, counting from zero. - value
-
The value the task returned.
-
Failed(index: Int, error: e)The task returned
Error. The task itself decided this; nothing went wrong with the run.Arguments
- index
-
The task’s position in the list given to
new, counting from zero. - error
-
The error the task returned.
-
Crashed(index: Int, reason: process.ExitReason)The task’s process died instead of returning. The Erlang exit reason is carried rather than flattened to a string, so a caller can match on it.
Arguments
- index
-
The task’s position in the list given to
new, counting from zero. - reason
-
Why the worker process exited.
-
Abandoned(index: Int)The task was started, then cancelled before it finished. Work was done and thrown away.
Arguments
- index
-
The task’s position in the list given to
new, counting from zero.
-
NeverStarted(index: Int)The run ended before this task ever got a slot. No work was done.
Arguments
- index
-
The task’s position in the list given to
new, counting from zero.
-
DrainProofLost(index: Int, reason: process.ExitReason)A managed task’s published owner exited abnormally, so nobody can any longer prove whether the task’s transitive work stopped. This is not a worker crash — the worker’s own fate is irrelevant once the proof is gone — and it is deliberately distinct from
Crashed: a caller recovering resources must treat “unknown liveness” differently from “known death”.Arguments
- index
-
The task’s position in the list given to
new, counting from zero. - reason
-
How the owner exited.
KilledandAbnormalboth land here; so does an owner that was already dead when the scope went to monitor it, because a proof that was never on file was never proof.
-
CancellationUnconfirmed(index: Int)Cancellation was requested, the grace set by
cancel_graceelapsed, and the task’s owner was still alive: nobody proved the cancellation landed. Only a run with a grace configured can produce this — without one the scope waits for the owner’s exit however long it takes.Arguments
- index
-
The task’s position in the list given to
new, counting from zero.
One task, prepared for a run: at minimum a begin closure, and for
managed work also the published owner whose exit is the lifecycle truth
for everything the task starts.
Built with task, prepared_task or prepared_leaf, and run with
new_prepared. The owner must already be alive when the run starts —
that is the parked work pattern: prepare the resource-holding process
first, hand its pid here, and let begin release the actual work only
once it runs inside the scope, which is by construction after the scope
has the owner under monitor.
pub opaque type PreparedTask(a, e)
What one pull produced.
pub type Pulled(a, e) {
PulledOutcome(outcome: Outcome(a, e))
AllDelivered
NotYet
RunLost(reason: process.ExitReason)
}
Constructors
-
PulledOutcome(outcome: Outcome(a, e))One task’s outcome, in completion order.
Arguments
- outcome
-
The outcome pulled.
-
AllDeliveredEvery outcome has been delivered; the run is over and nothing it started is alive. Stop pulling: there is nothing left to pull, and a later
pullwill report the scope’s exit asRunLostrather than repeating this answer. -
NotYetNothing landed within the wait. The demand stands — the scope holds at most one granted delivery, so pulling again neither loses nor duplicates an outcome.
-
RunLost(reason: process.ExitReason)The scope died without saying
Done: either something outside the run destroyed it, or its final exit carried the drain verdict for a run whose account was already delivered.Arguments
- reason
-
How the scope exited.
A configured run, not yet started.
Built with new or new_prepared and refined with limit, on_failure,
cancel_with, deadline and cancel_grace; start, fold and
start_detached are the terminal verbs. A Run is an ordinary immutable
value, so the same one can be started more than once.
pub opaque type Run(a, e)
A reducer’s verdict on whether to keep consuming outcomes.
This is fold’s answer to the problem a plain fn(acc, outcome) -> acc
cannot express: a reducer that has seen enough has no way to say so, and the
caller is left reaching for an external cancel signal to make a decision the
reducer already made.
pub type Step(acc) {
Continue(accumulator: acc)
Halt(accumulator: acc)
}
Constructors
-
Continue(accumulator: acc)Keep going: ask the scope for the next outcome.
Arguments
- accumulator
-
The accumulator to carry into the next outcome.
-
Halt(accumulator: acc)Stop here: the remaining tasks are cancelled and the run is torn down before
foldreturns.Arguments
- accumulator
-
The accumulator
foldwill return.
Values
pub fn adopt(
ledger: Ledger,
owner owner: process.Pid,
cancel cancel: fn() -> Nil,
) -> Adoption
Publish a transitive owner to a running managed task’s scope.
Only a normal exit of owner will prove its subtree drained; any
other exit settles the task as DrainProofLost. Synchronous: returns
once the scope has the owner under monitor, or Refused at once if the
scope has already exited. May be called from any process holding the
ledger, any number of times, for as many owners as the task discovers.
Examples
case weft.adopt(ledger, owner: http_owner, cancel: stop_http) {
weft.Adopted -> begin_http()
weft.Refused -> Nil
}
pub fn adopt_leaf(
ledger: Ledger,
owner owner: process.Pid,
cancel cancel: fn() -> Nil,
) -> Adoption
Publish a leaf owner to a running managed task’s scope: an owner that
provably owns nothing further, completed by any exit. The prepared_leaf
exemption, for owners discovered mid-run.
Examples
case weft.adopt_leaf(ledger, owner: pump, cancel: stop_pump) {
weft.Adopted -> start_pump()
weft.Refused -> Nil
}
pub fn cancel(signal: Cancel) -> Nil
Fire a cancel signal, ending every run watching it.
Firing a signal that has already fired does nothing. Runs watching the
signal report Abandoned for the tasks that were running and NeverStarted
for the tasks that had not begun.
Examples
let stop = weft.cancel_signal()
process.spawn(fn() { weft.new(tasks) |> weft.cancel_with(stop) |> weft.start })
// From any process, at any time, including while the run's caller is
// blocked inside `start`:
weft.cancel(stop)
pub fn cancel_detached(detached: Detached(a, e)) -> Nil
Cancel a detached run without giving up its account.
Unlike a fold reducer’s Halt, which discards what it has not seen,
this keeps the deliveries coming: running tasks settle as Abandoned,
unstarted ones as NeverStarted, and managed tasks by whatever their
owners’ exits prove. Idempotent — cancelling a cancelled or finished run
does nothing.
Examples
weft.cancel_detached(detached)
// ... keep pulling until AllDelivered ...
pub fn cancel_grace(run: Run(a, e), within: Int) -> Run(a, e)
Bound how long a cancellation waits for managed owners to exit, in milliseconds.
Without a grace, cancelling a managed task means asking its owner to stop
and waiting for the owner’s exit however long that takes, because
“cancelled” without drain proof is not a fact this module will invent.
With one, an owner still alive when the grace expires settles as
CancellationUnconfirmed, the run finishes bounded — and the scope exits
abnormally, because it can no longer vouch for the subtree it was
witnessing. The grace bounds the wait, and what it buys in boundedness
it pays for in the scope’s exit verdict.
One grace timer serves the whole cancellation rather than one per task: it is an acknowledgement window on the teardown, not a per-task timeout, and it is armed when cancellation begins, from whichever direction it came.
Examples
let outcomes =
weft.new_prepared(tasks)
|> weft.deadline(30_000)
|> weft.cancel_grace(2000)
|> weft.start
pub fn cancel_signal() -> Cancel
Create a fresh cancel signal.
The returned value can be passed to cancel_with on any number of runs and
shared with any number of processes. It costs one idle process, which lives
until cancel is called on it.
Examples
let stop = weft.cancel_signal()
let outcomes =
weft.new(tasks)
|> weft.cancel_with(stop)
|> weft.start
pub fn cancel_when_exits(
run: Run(a, e),
pid: process.Pid,
) -> Run(a, e)
Cancel the run when pid exits, for any reason.
This is the consumer’s death made a cancellation cause without the consumer having to be the process that started the run: a stream whose receiver is gone has nobody to deliver to and should stop its work. The watch is a monitor, never a link, so firing it ends the run and touches nothing else. A pid already dead when the run starts cancels it before any task is spawned, exactly like a spent cancel signal.
Examples
let witness =
weft.new_prepared([weft.managed(serve)])
|> weft.cancel_when_exits(consumer)
|> weft.start_witnessed
pub fn cancel_with(run: Run(a, e), signal: Cancel) -> Run(a, e)
Watch a cancel signal for the duration of the run.
This is the only way to stop a run from outside the process that started it,
because that process is blocked inside start or fold and cannot act on
its own behalf.
Examples
let stop = weft.cancel_signal()
let outcomes =
weft.new(tasks)
|> weft.cancel_with(stop)
|> weft.start
pub fn deadline(run: Run(a, e), within: Int) -> Run(a, e)
Give the whole run a wall-clock budget, in milliseconds.
Hitting the deadline is not an error, it is a reason some entries in the
account are Abandoned (started, then cut short) or NeverStarted (never
got a slot). The timer is cancelled and flushed if the run finishes first,
so it can never fire into a later receive.
Examples
let outcomes =
weft.new(tasks)
|> weft.deadline(30_000)
|> weft.start
pub fn failures(outcomes: List(Outcome(a, e))) -> List(e)
The errors of the tasks that returned Error, in the order given.
Tasks that crashed, were abandoned, or never started are not errors the
tasks chose to return, so they appear here in no form at all; reach for
partition when you need to see them.
Examples
let outcomes = [weft.Failed(0, "a"), weft.Crashed(1, process.Killed)]
assert weft.failures(outcomes) == ["a"]
pub fn first_ok(
tasks: List(fn() -> Result(a, e)),
) -> Result(a, List(Outcome(a, e)))
Run tasks concurrently and return the first one to succeed, falling back to the full account if none did. The losers are killed and joined before a success returns.
A Failed or Crashed task does not end the run here; that is the whole
point of the function, and it is why first_ok leaves on_failure at
KeepGoing. An empty list is answerable, unlike in race, because the error
channel already exists: it yields Error([]).
If something outside the run destroys the scope, the Error carries only
the outcomes heard before the death rather than a total account; reach for
start, which fills the gap in, when that distinction matters.
Examples
let body = weft.first_ok([fn() { fetch(mirror_a) }, fn() { fetch(mirror_b) }])
// -> Ok(<-body->)
// Nothing succeeded, so the caller gets the whole account in input order.
let answer = weft.first_ok([fn() { Error("a") }, fn() { Error("b") }])
// -> Error([weft.Failed(0, "a"), weft.Failed(1, "b")])
pub fn fold(
run: Run(a, e),
from initial: acc,
with reducer: fn(acc, Outcome(a, e)) -> Step(acc),
) -> acc
Consume outcomes in completion order as they land.
This is how a run whose results do not all fit in memory is processed, and how a caller acts on early results without waiting for the slow tail. The scope holds each outcome until the reducer asks for it, so at most one is ever in flight and a slow reducer stops new work from starting rather than filling a mailbox.
A reducer returning Halt ends the run: the remaining tasks are killed and
joined before fold returns, so no worker survives the call. The outcomes
of those tasks are not delivered — fold returns an accumulator and has
nowhere to put them — so a caller that wants the full account of a run it
stopped early should use start with a cancel signal instead.
Examples
// Count successes without holding every value.
let successes =
weft.new(tasks)
|> weft.fold(from: 0, with: fn(count, outcome) {
case outcome {
weft.Completed(..) -> weft.Continue(count + 1)
weft.Failed(..) | weft.Crashed(..) -> weft.Continue(count)
weft.Abandoned(..) | weft.NeverStarted(..) -> weft.Continue(count)
weft.DrainProofLost(..) -> weft.Continue(count)
weft.CancellationUnconfirmed(..) -> weft.Continue(count)
}
})
// Stop as soon as three values are in hand; the rest are cancelled.
let first_three =
weft.new(tasks)
|> weft.fold(from: [], with: fn(found, outcome) {
let found = case outcome {
weft.Completed(value:, ..) -> [value, ..found]
weft.Failed(..) | weft.Crashed(..) -> found
weft.Abandoned(..) | weft.NeverStarted(..) -> found
weft.DrainProofLost(..) | weft.CancellationUnconfirmed(..) -> found
}
case list.length(found) >= 3 {
True -> weft.Halt(found)
False -> weft.Continue(found)
}
})
pub fn limit(run: Run(a, e), max: Int) -> Run(a, e)
Set how many tasks may occupy a slot at once.
A slot is held from the moment a task is spawned until its outcome has been
handed to the consumer, so this bounds running work and completed-but-
unconsumed results together. A max below one is raised to one; there is no
unbounded setting, by design.
Examples
let outcomes =
weft.new(tasks)
|> weft.limit(8)
|> weft.start
pub fn managed(
begin: fn(Ledger) -> Result(a, e),
) -> PreparedTask(a, e)
Prepare a managed task whose owners are discovered while it runs.
This is prepared_task for work whose owners do not exist yet when the
run starts: a request worker that prepares a transport, learns the pid
holding its socket, and must have that pid witnessed before the socket
is allowed to open. begin receives a Ledger; every adopt or
adopt_leaf on it publishes one more owner to the scope, and the task’s
slot is held and its outcome withheld until the worker and every
adopted owner have exited. A task that adopts nothing behaves exactly
like task.
Publication is the parked-work pattern applied mid-run: prepare the
owner parked, adopt it, and release it only on Adopted. A Refused
means the run is already cancelling — the scope retains the owner and
asks it to stop, so nothing leaks, but the caller must not begin the
work.
Examples
let outcomes =
weft.new_prepared([
weft.managed(fn(ledger) {
let #(owner, cancel, release) = prepare(request)
case weft.adopt(ledger, owner:, cancel:) {
weft.Adopted -> Ok(release())
weft.Refused -> Error(Cancelled)
}
}),
])
|> weft.start
pub fn map(
items: List(a),
limit max: Int,
with fun: fn(a) -> Result(b, e),
) -> List(Outcome(b, e))
Apply a fallible function to every item, bounded, and return the account.
Exactly weft.new over the items with limit and start, which is the
shape most callers want and the one it is easiest to get wrong by mapping an
unbounded async over a list.
Examples
let outcomes = weft.map(urls, limit: 8, with: fetch)
let #(bodies, rest) = weft.partition(outcomes)
pub fn new(tasks: List(fn() -> Result(a, e))) -> Run(a, e)
Begin a run over tasks, bounded by default.
The default limit is the number of schedulers online, not unbounded:
spawning one process per item is fine for pure computation and actively
harmful for anything holding a socket, a file handle, a database connection
or a rate-limited quota. Someone who wants a wider fan-out says so with
limit.
The default failure policy is KeepGoing.
Examples
let outcomes =
urls
|> list.map(fn(url) { fn() { fetch(url) } })
|> weft.new
|> weft.start
pub fn new_prepared(tasks: List(PreparedTask(a, e))) -> Run(a, e)
Begin a run over prepared tasks, plain and managed mixed freely.
This is new for tasks built with task, prepared_task and
prepared_leaf. The defaults are the same: a limit of the schedulers
online, KeepGoing on failure, no deadline, and no cancellation grace —
a cancelled managed task is awaited until its owner exits, however long
that takes, unless cancel_grace bounds it.
Examples
let outcomes =
weft.new_prepared([
weft.task(fn() { fetch(url) }),
weft.prepared_task(owner:, cancel:, begin:),
])
|> weft.start
pub fn on_failure(run: Run(a, e), policy: OnFailure) -> Run(a, e)
Set what a failing task does to the tasks beside it.
Examples
// Stop at the first `Failed` or `Crashed`; the rest are reported as
// `Abandoned` or `NeverStarted` rather than being lost.
let outcomes =
weft.new(tasks)
|> weft.on_failure(weft.CancelSiblings)
|> weft.start
pub fn partition(
outcomes: List(Outcome(a, e)),
) -> #(List(a), List(Outcome(a, e)))
Split an account into the values that succeeded and everything else.
Both halves keep the order they were given in. The second half stays a list
of Outcome rather than a list of errors, because “did not succeed” covers
four different facts and flattening them is how the accounting gets thrown
away one call site at a time.
Examples
let outcomes = [weft.Completed(0, "a"), weft.Failed(1, "no")]
assert weft.partition(outcomes) == #(["a"], [weft.Failed(1, "no")])
pub fn prepared_leaf(
owner owner: process.Pid,
cancel cancel: fn() -> Nil,
begin begin: fn() -> Result(a, e),
) -> PreparedTask(a, e)
Prepare a managed task whose owner provably owns nothing further.
The one difference from prepared_task: any exit of the owner
completes the drain obligation, normal or not. This is the declared
exemption for single-process owners — an observer, a pump, a relay —
where an ordinary crash is an ordinary crash and must not be read as a
lost proof over descendants that never existed. Everything else — the
monitor-before-begin ordering, the held slot, the cancel helper — is
identical.
Examples
let outcomes =
weft.new_prepared([
weft.prepared_leaf(owner: pump, cancel: stop_pump, begin: run),
])
|> weft.start
pub fn prepared_task(
owner owner: process.Pid,
cancel cancel: fn() -> Nil,
begin begin: fn() -> Result(a, e),
) -> PreparedTask(a, e)
Prepare a managed task: work whose lifecycle outlives its worker.
owner is the pid whose exit proves the task’s transitive work is gone —
only a normal exit proves it; any other exit becomes DrainProofLost.
cancel asks the subtree to stop; it must be safe to call more than once
and safe to call after the work has already finished, and it runs on a
disposable helper so a crash inside it cannot cost the run its witness.
begin is the worker’s blocking path, and it runs only after the scope
holds the owner under monitor.
The task’s slot is held, and its outcome withheld, until both the worker
and the owner have exited. An owner already dead when the run starts
yields DrainProofLost without begin ever running: proof that was
never on file was never proof.
Examples
// `prepare` parks the real request and hands back its owning pid, a
// cancel capability, and the closure that lets it loose.
let #(owner, cancel, begin) = prepare(request)
let outcomes =
weft.new_prepared([weft.prepared_task(owner:, cancel:, begin:)])
|> weft.cancel_grace(2000)
|> weft.start
pub fn pull(
detached: Detached(a, e),
within timeout: Int,
) -> Pulled(a, e)
Collect one outcome from a detached run, waiting at most within
milliseconds.
Must be called from the process that called start_detached — the
handle’s outbox belongs to it. Demand is unary and idempotent: a pull
that times out leaves its demand standing, and the next pull re-grants
it harmlessly, so at most one outcome is ever in flight and none is ever
dropped or doubled.
Examples
case weft.pull(detached, within: 1000) {
weft.PulledOutcome(outcome) -> act_on(outcome)
weft.NotYet -> check_something_else()
weft.AllDelivered -> done()
weft.RunLost(reason) -> escalate(reason)
}
pub fn race(
first: fn() -> Result(a, e),
rest: List(fn() -> Result(a, e)),
) -> Outcome(a, e)
Run tasks concurrently and return the first one to complete, however it completed. The losers are killed and joined before this returns.
The first task is a separate argument from the rest because a race over
nothing has no answer: Outcome has no variant meaning “there was nothing to
race”, and wrapping the result in a Result would push an error case onto
every call site that already knows its list is not empty. Making the empty
case unrepresentable is cheaper than making every caller handle it. A caller
holding a runtime list matches on it once and decides for itself what
nothing should mean.
Note the difference from first_ok: this returns the first task to
finish, even if it finished by failing or crashing. Conflating the two is a
common bug, which is why they are separate functions.
Examples
// Whichever mirror answers first, even if it answers with an error.
let outcome = weft.race(fn() { fetch(mirror_a) }, [fn() { fetch(mirror_b) }])
// -> weft.Completed(index: 1, value: <-body->)
pub fn scope_pid(detached: Detached(a, e)) -> process.Pid
The scope process behind a detached run.
This is the pid an outer witness monitors, and the pid a nested run
publishes as its owner: a detached run handed to prepared_task as
owner: weft.scope_pid(inner) composes ownership — the outer scope’s
proof for that task is the inner scope’s own drain verdict, with no
translation code on either side.
Examples
let inner = weft.new_prepared(children) |> weft.start_detached
let outer_task =
weft.prepared_task(
owner: weft.scope_pid(inner),
cancel: fn() { weft.cancel_detached(inner) },
begin: fn() { collect(inner) },
)
pub fn start(run: Run(a, e)) -> List(Outcome(a, e))
Run everything to completion and return every outcome in input order.
Blocks the calling process until the run is over. When it returns, every task has exactly one outcome and no worker from the run is alive.
Ordered versus unordered falls out of which terminal verb you call rather
than out of a flag: start sorts the account back into input order, fold
hands outcomes over in completion order as they land.
Examples
let outcomes =
weft.new([fn() { Ok(1) }, fn() { Error("no") }])
|> weft.start
// -> [weft.Completed(0, 1), weft.Failed(1, "no")]
let #(values, rest) =
weft.new(tasks)
|> weft.limit(4)
|> weft.on_failure(weft.CancelSiblings)
|> weft.start
|> weft.partition
pub fn start_detached(run: Run(a, e)) -> Detached(a, e)
Start a run without blocking, and hand back its handle.
The run is the same in every respect — same scope, same account, same
ownership guarantees — only the consumption differs: nothing is
delivered until the holder asks with pull, so the run’s backpressure
now reaches whatever pace the holder pulls at.
Examples
let detached =
weft.new_prepared(tasks)
|> weft.cancel_grace(2000)
|> weft.start_detached
let watch = process.monitor(weft.scope_pid(detached))
// ... pull outcomes as the mood takes you ...
pub fn start_relayed(
run: Run(a, e),
to sink: process.Subject(Pulled(a, e)),
) -> process.Pid
Start a run and push every outcome to sink as an ordinary message,
from a relay process this function spawns and returns.
This is the adapter for consumers that are actors: an actor must not
block its receive loop inside pull, so the relay owns the pulling and
the actor merely receives. Each outcome arrives as PulledOutcome,
followed by exactly one AllDelivered or RunLost, after which the
relay exits normally.
What is traded away is stated plainly: push delivery makes the sink’s
mailbox the buffer, so the engine’s backpressure ends at the relay. That
is the same trade every actor’s mailbox already makes, and the limit
still bounds how much work runs at once — only the finished outcomes
queue without bound.
The relay is linked to the caller, and the scope to the relay, so the ownership chain survives: a dead consumer takes the relay with it, the scope traps that exit, cancels, drains its owners, and exits.
Examples
let relay =
weft.new_prepared(tasks)
|> weft.start_relayed(to: sink)
// The actor's selector now receives weft.Pulled(a, e) messages.
pub fn start_witnessed(run: Run(a, e)) -> process.Pid
Start a run whose only report is the scope’s exit, and hand back the scope’s pid.
Nothing is delivered to anyone: each outcome is discarded the moment it
is sealed, and its slot returned. What remains is the part a drain
witness needs — the scope is alive exactly while any worker, any
adopted owner, or any cancel helper is, and it exits normally only if
every proof landed. This is the shape for a run started purely to
witness work: the caller monitors the returned pid, cancels through a
signal or cancel_when_exits, and reads the verdict off the DOWN.
The scope is linked to the caller, as every scope is: a dead caller
cancels the run, the owners are asked to stop, and the scope drains
before it exits. Use cancel_when_exits to name a consumer that is not
the caller.
Examples
let scope =
weft.new_prepared([weft.managed(run_request)])
|> weft.cancel_when_exits(consumer)
|> weft.start_witnessed
let watch = process.monitor(scope)
// A normal DOWN proves everything the request started is gone.
pub fn task(begin: fn() -> Result(a, e)) -> PreparedTask(a, e)
Prepare a plain task: no owner, no drain obligation, exactly the
behaviour of a closure given to new.
Examples
let outcomes =
weft.new_prepared([weft.task(fn() { Ok(1) })])
|> weft.start