Generators

cross.stream generators use Nushell closures to create streams of data that are emitted as frames into the store.

Basic Usage

To create a generator, append a Nushell script that evaluates to a configuration record with a run closure using the topic <topic>.spawn:

r#'{
  run: {|| ^tail -F http.log | lines }
}'# | .append log.spawn

The generator will:

• Execute the provided Nushell expression
• Output from the pipeline is streamed as log.recv frames. Text pipelines emit one frame per line, while ByteStream pipelines send binary chunks.
• Automatically restarts if it exits until a terminate frame is seen

All frames produced by the generator use the same context as the .spawn frame so multiple contexts can run generators with the same topic independently.

Lifecycle Events

Generators emit lifecycle events to track their state:

Event	Description
<topic>.start	Generator has started processing
<topic>.recv	Output value from the generator
<topic>.stop	Generator pipeline has stopped. The `meta.reason` field is a string enum with values finished, error, terminate and update. When finished or error, the pipeline will be restarted automatically; terminate means it was stopped manually and the generator loop for this topic/context will shut down. update indicates the generator reloaded due to a new .spawn frame.
<topic>.parse.error	Script failed to parse
<topic>.shutdown	Generator loop has fully exited; ServeLoop evicts it

All events include source_id which is the ID of the generator instance. When a .stop frame has meta.reason set to update, it also includes update_id referencing the spawn that triggered the reload. ServeLoop evicts a generator when it receives a <topic>.shutdown frame.

Configuration Options

Option	Type	Default	Description
duplex	boolean	false	Enable sending input to the generator’s pipeline via <topic>.send
return_options	record	—	Customize output frames (see Return Options)

The return_options field controls the suffix and TTL for the .recv frames produced by the generator.

Bi-directional Communication

When duplex is enabled, you can send data into the generator’s input pipeline via <topic>.send frames:

# Create a websocket connection
r#'{
  run: {|| websocat wss://echo.websocket.org | lines },
  duplex: true
}'# | .append echo.spawn

# Send input to the websocket: note the "\n", wss://echo.websocket.org won't
# reply until it sees a complete line
"hello\n" | .append echo.send

When running this generator:

• Lines received from the websocket server are emitted as <topic>.recv frames
• Content from <topic>.send frames is sent to the websocket server

Error Handling

If a generator encounters an error during spawning a <topic>.parse.error frame is emitted with:

• source_id: ID of the failed spawn attempt
• reason: Error message describing what went wrong

The generator does not start and no stop frame is produced.

When a running generator finishes or fails, it automatically restarts after a 1-second delay.

Stopping Generators

To stop a running generator, append a frame with the topic <topic>.terminate. The generator will stop and emit a <topic>.stop frame with meta.reason set to terminate.

Appending a new <topic>.spawn frame while a generator of the same topic and context is running reloads it with the new script. If the reload fails to parse, you’ll see a <topic>.parse.error frame, and the previous generator continues running.