Your First Stream

Let’s create your first event stream.

Note

This tutorial uses commands from xs.nu. If you haven’t installed it yet, run xs nu --install and then use xs.nu * in Nushell. See the installation guide for details.

Serve

Unlike sqlite, which operates directly on the file system, xs requires a running process to manage access to the local store. This enables features like subscribing to real-time updates from the event stream.

Start an xs store in a dedicated window:

xs serve ./store

13:35:16.868 TRACE         event src/main.rs:185 Starting server with path: "./store"                                                                    xs:185
13:35:16.957  INFO         read options=ReadOptions { follow: On, tail: false, last_id: None, limit: None }                                      xs::store:174
13:35:16.957  INFO         read options=ReadOptions { follow: On, tail: true, last_id: None, limit: None }                                       xs::store:174
13:35:16.963  INFO     5ms insert_frame frame=Frame { id: "03d4por2p16i05i81fjy0fx8u", topic: "xs.start", hash: None, meta: None, ttl: None }    xs::store:410
13:35:16.963 0fx8u xs.start
13:35:16.968  INFO         read options=ReadOptions { follow: On, tail: false, last_id: None, limit: None }                                      xs::store:174

For a long-running setup you might run xs serve ~/.local/share/cross.stream/store under a process supervisor. This is also the fallback location used by xs.nu when $env.XS_ADDR isn’t set. Here we keep the demo scoped to ./store.

To point tools at another store, set XS_ADDR. This can be done temporarily with with-env:

with-env {XS_ADDR: "./store"} { .cat }

Client

append command

OK! Let’s append our first event:

nushell

"a quick note" | .append notes

───────┬─────────────────────────────────────────────────────
 topic │ notes
 id    │ 03d4q1qhbiv09ovtuhokw5yxv
 hash  │ sha256-wIcRiyKpOjA1Z8O+wZvoiMXYgGEzPQOhlA8AOptOhBY=
 meta  │
 ttl   │ forever
───────┴─────────────────────────────────────────────────────

bash

echo "a quick note" | xs append ./store notes

{"topic":"notes","id":"03d4qic9vqkve1krajjtlbavd","hash":"sha256-24yYvzQ4Zd3Go/WevV9ol+KzkdTgQvlyNN2NVSGMjFE=","meta":null,"ttl":"forever"}

cat and cas commands

and then cat the stream:

nushell

.cat

─#─┬──topic───┬────────────id─────────────┬────────────────────────hash─────────────────────────┬─meta─┬───ttl───
 0 │ xs.start │ 03d4q1o70y6ek0ig8hwy9q00n │                                                     │      │
 1 │ notes    │ 03d4q1qhbiv09ovtuhokw5yxv │ sha256-wIcRiyKpOjA1Z8O+wZvoiMXYgGEzPQOhlA8AOptOhBY= │      │ forever
───┴──────────┴───────────────────────────┴─────────────────────────────────────────────────────┴──────┴─────────

These are the raw frames on the stream. The actually content is stored separately in the Content-Addressable Storage (CAS). You can read more about that here.

We have the full expressiveness of Nushell available to us—for example, we can get the content hash of the last frame on the stream using:

.cat | last | $in.hash

sha256-wIcRiyKpOjA1Z8O+wZvoiMXYgGEzPQOhlA8AOptOhBY=

and then use the .cas command to retrieve the content:

.cat | last | .cas $in.hash

a quick note

We can also retrieve the content for a frame by piping it in its entirety directly to .cas:

.cat | last | .cas

a quick note

bash

xs cat ./store

{"topic":"xs.start","id":"03d4qiab9g5vagrlrvxa2vjw0","hash":null,"meta":null,"ttl":null}
{"topic":"notes","id":"03d4qic9vqkve1krajjtlbavd","hash":"sha256-24yYvzQ4Zd3Go/WevV9ol+KzkdTgQvlyNN2NVSGMjFE=","meta":null,"ttl":"forever"}

These are the raw frames on the stream. The actually content is stored separately in the Content-Addressable Storage (CAS). You can read more about that here.

We can retrieve the content for a frame using its content hash:

xs cat ./store | tail -n1 | jq -r .hash

sha256-24yYvzQ4Zd3Go/WevV9ol+KzkdTgQvlyNN2NVSGMjFE=

xs cas ./store sha256-24yYvzQ4Zd3Go/WevV9ol+KzkdTgQvlyNN2NVSGMjFE=

a quick note

head command

Let’s submit another note:

nushell

"submit TPS report" | .append notes
.cat

─#─┬──topic───┬────────────id─────────────┬────────────────────────hash─────────────────────────┬─meta─┬───ttl───
 0 │ xs.start │ 03d4q1o70y6ek0ig8hwy9q00n │                                                     │      │
 1 │ notes    │ 03d4q1qhbiv09ovtuhokw5yxv │ sha256-wIcRiyKpOjA1Z8O+wZvoiMXYgGEzPQOhlA8AOptOhBY= │      │ forever
 2 │ notes    │ 03d4qbrxizqgav09m7hicksb0 │ sha256-KDyb7pypM+8aLiq5obfpCqbMmb6LvvPnCu2+y9eWd0c= │      │ forever
───┴──────────┴───────────────────────────┴─────────────────────────────────────────────────────┴──────┴─────────

We can get the most recent note on the stream using the .head command:

.head notes

───────┬─────────────────────────────────────────────────────
 topic │ notes
 id    │ 03d4qbrxizqgav09m7hicksb0
 hash  │ sha256-KDyb7pypM+8aLiq5obfpCqbMmb6LvvPnCu2+y9eWd0c=
 meta  │
 ttl   │ forever
───────┴─────────────────────────────────────────────────────

.head notes | .cas

submit TPS report

bash

echo "submit TPS report" | xs append ./store notes
xs cat ./store

{"topic":"xs.start","id":"03d4qiab9g5vagrlrvxa2vjw0","hash":null,"meta":null,"ttl":null}
{"topic":"notes","id":"03d4qic9vqkve1krajjtlbavd","hash":"sha256-24yYvzQ4Zd3Go/WevV9ol+KzkdTgQvlyNN2NVSGMjFE=","meta":null,"ttl":"forever"}
{"topic":"notes","id":"03d4qjwnhwudlfyg1ygemmt7b","hash":"sha256-pwB4w9N1v99Uu/96KsKKaIgDMm18QDMOZJsEwCxBtsA=","meta":null,"ttl":"forever"}

We can get the most recent note on the stream using the head command:

xs head ./store notes

{"topic":"notes","id":"03d4qjwnhwudlfyg1ygemmt7b","hash":"sha256-pwB4w9N1v99Uu/96KsKKaIgDMm18QDMOZJsEwCxBtsA=","meta":null,"ttl":"forever"}

xs head ./store notes | jq -r .hash | xargs xs cas ./store

submit TPS report

riffing

Finally, let’s pull a list of all our notes.

nushell

We can filter directly by topic using --topic and then use the each command to pull out the content of each note:

.cat | where topic == "notes" | each { .cas $in.hash }

───┬───────────────────
0 │ a quick note
1 │ submit TPS report
───┴───────────────────

Fun! 🎉

bash

xs cat ./store --topic notes | jq -r .hash | xargs -I{} xs cas ./store {}

a quick note
submit TPS report

Fun! 🎉

Tip

About now, I’d encourage you to checkout the Nushell version of this tutorial, the tab over. You might be reconsidering a bunch of life choices. 🤣 No stress though: check out the Nushell site and give it a try!