Skip to content

MARKDOWN SOURCE

Raw
# dial

A single-node message broker with SQLite file backing.

- **Single node:** one process runs the broker. There is no clustering or replication.
- **SQLite file backing:** every message is written to one SQLite database file, so messages survive restarts and the whole broker can be backed up by copying that file.

The broker has two endpoints for clients, plus a health check:

| Endpoint | Used by | What it does |
| --- | --- | --- |
| `POST /publish/{topic}` | publishers | Stores the request body as one message on `topic`. Returns `201` with `{"id": <n>}`. Requires the API key. |
| `GET /subscribe/{topic}?after=<id>&limit=<n>` | subscribers | Returns messages on `topic` with `id > after`, oldest first. Requires the API key. |
| `GET /up` | health checks | Returns `200` while the broker is serving. No API key needed. |

Message ids increase monotonically across all topics. A subscriber's position in a topic is just the last `id` it has processed.

## Authentication

Publish and subscribe require the broker's API key, sent as a bearer token:

```
Authorization: Bearer <DIAL_API_KEY>
```

A missing or wrong key gets `401`. The key is one shared secret for all clients; the broker has no per-client keys. Keep it out of source control and pass it through the environment.

## Run

```sh
DIAL_API_KEY=<your-key> cargo run --release
```

Configuration is through environment variables:

| Variable | Default | Meaning |
| --- | --- | --- |
| `DIAL_API_KEY` | none, **required** | API key for publish and subscribe. The broker refuses to start if it is unset or empty. |
| `DIAL_DB` | `dial.db` | Path to the SQLite database file. Created if missing. |
| `DIAL_BIND` | `127.0.0.1:8080` | Address the HTTP server listens on. |

## Docker

```sh
docker build -t silenloc/dial .
docker run -e DIAL_API_KEY -p 8080:80 -v dial-storage:/storage silenloc/dial
```

`-e DIAL_API_KEY` passes the key from your shell into the container without putting it on the command line. The image listens on port 80 and keeps the database in `/storage`, a volume that survives container replacement. The image's `ino.deploy.*` labels opt it into release-triggered deployment with ino; see the ino server guide for how `ino` and `ino once remote create` use them. The API key must be set as an environment variable on the deployed app, never baked into the image.

## Publishing

A publisher sends a `POST` whose body is the message. The body must be UTF-8 text; anything else gets `400`. Use JSON or any other text format you like.

```sh
curl -X POST http://127.0.0.1:8080/publish/orders \
  -H "Authorization: Bearer $DIAL_API_KEY" \
  -d '{"sku":"abc-123","qty":2}'
# {"id":1}
```

## Subscribing

A subscriber polls with a cursor. Start with `after=0` to read a topic from the beginning. After processing a batch, set `after` to the largest `id` you received.

```sh
curl -H "Authorization: Bearer $DIAL_API_KEY" \
  'http://127.0.0.1:8080/subscribe/orders?after=0&limit=100'
# [{"id":1,"payload":"{\"sku\":\"abc-123\",\"qty\":2}","created_at":1794845492863}]
```

- `limit` is optional: default `100`, clamped to `1..=1000`.
- An empty array means nothing new has been published after the cursor. Poll again later.
- The broker does not track subscribers. Each subscriber keeps its own `after` value, and it must persist that value itself if it needs to resume after a restart.

## Example: publisher and subscriber in Python

Both scripts read the key from the `DIAL_API_KEY` environment variable.

Publisher (`publish.py`):

```python
import json
import os
import urllib.request

BROKER = "http://127.0.0.1:8080"
API_KEY = os.environ["DIAL_API_KEY"]

def publish(topic: str, message: dict) -> int:
    req = urllib.request.Request(
        f"{BROKER}/publish/{topic}",
        data=json.dumps(message).encode(),
        headers={"Authorization": f"Bearer {API_KEY}"},
        method="POST",
    )
    with urllib.request.urlopen(req) as resp:
        return json.load(resp)["id"]

if __name__ == "__main__":
    print(publish("orders", {"sku": "abc-123", "qty": 2}))
```

Subscriber (`subscribe.py`):

```python
import json
import os
import time
import urllib.request

BROKER = "http://127.0.0.1:8080"
API_KEY = os.environ["DIAL_API_KEY"]
TOPIC = "orders"

def fetch(after: int) -> list[dict]:
    url = f"{BROKER}/subscribe/{TOPIC}?after={after}&limit=100"
    req = urllib.request.Request(url, headers={"Authorization": f"Bearer {API_KEY}"})
    with urllib.request.urlopen(req) as resp:
        return json.load(resp)

def handle(message: dict) -> None:
    print("got", message["id"], json.loads(message["payload"]))

if __name__ == "__main__":
    cursor = 0  # persist this if the subscriber must survive restarts
    while True:
        batch = fetch(cursor)
        for message in batch:
            handle(message)
            cursor = message["id"]
        if not batch:
            time.sleep(1)
```

Run the broker, then in another terminal run `DIAL_API_KEY=<your-key> python3 subscribe.py`, and publish with `DIAL_API_KEY=<your-key> python3 publish.py` or the curl command above.

## Build and test

```sh
cargo test
```

## Limits

- Single node only. Throughput and availability are those of one process and one SQLite file.
- One shared API key for all clients. There is no per-client key, rotation, or expiry, so rotating the key means restarting the broker with the new value and updating every client.
- No acknowledgements, retries, consumer groups, or retention policy. Messages are kept until the database file is removed.
- Payloads are stored as text.

dial

A single-node message broker with SQLite file backing.

  • Single node: one process runs the broker. There is no clustering or replication.
  • SQLite file backing: every message is written to one SQLite database file, so messages survive restarts and the whole broker can be backed up by copying that file.

The broker has two endpoints for clients, plus a health check:

EndpointUsed byWhat it does
POST /publish/{topic}publishersStores the request body as one message on topic. Returns 201 with {"id": <n>}. Requires the API key.
GET /subscribe/{topic}?after=<id>&limit=<n>subscribersReturns messages on topic with id > after, oldest first. Requires the API key.
GET /uphealth checksReturns 200 while the broker is serving. No API key needed.

Message ids increase monotonically across all topics. A subscriber's position in a topic is just the last id it has processed.

Authentication

Publish and subscribe require the broker's API key, sent as a bearer token:

Authorization: Bearer <DIAL_API_KEY>

A missing or wrong key gets 401. The key is one shared secret for all clients; the broker has no per-client keys. Keep it out of source control and pass it through the environment.

Run

DIAL_API_KEY=<your-key> cargo run --release

Configuration is through environment variables:

VariableDefaultMeaning
DIAL_API_KEYnone, requiredAPI key for publish and subscribe. The broker refuses to start if it is unset or empty.
DIAL_DBdial.dbPath to the SQLite database file. Created if missing.
DIAL_BIND127.0.0.1:8080Address the HTTP server listens on.

Docker

docker build -t silenloc/dial .
docker run -e DIAL_API_KEY -p 8080:80 -v dial-storage:/storage silenloc/dial

-e DIAL_API_KEY passes the key from your shell into the container without putting it on the command line. The image listens on port 80 and keeps the database in /storage, a volume that survives container replacement. The image's ino.deploy.* labels opt it into release-triggered deployment with ino; see the ino server guide for how ino and ino once remote create use them. The API key must be set as an environment variable on the deployed app, never baked into the image.

Publishing

A publisher sends a POST whose body is the message. The body must be UTF-8 text; anything else gets 400. Use JSON or any other text format you like.

curl -X POST http://127.0.0.1:8080/publish/orders \
  -H "Authorization: Bearer $DIAL_API_KEY" \
  -d '{"sku":"abc-123","qty":2}'
# {"id":1}

Subscribing

A subscriber polls with a cursor. Start with after=0 to read a topic from the beginning. After processing a batch, set after to the largest id you received.

curl -H "Authorization: Bearer $DIAL_API_KEY" \
  'http://127.0.0.1:8080/subscribe/orders?after=0&limit=100'
# [{"id":1,"payload":"{\"sku\":\"abc-123\",\"qty\":2}","created_at":1794845492863}]
  • limit is optional: default 100, clamped to 1..=1000.
  • An empty array means nothing new has been published after the cursor. Poll again later.
  • The broker does not track subscribers. Each subscriber keeps its own after value, and it must persist that value itself if it needs to resume after a restart.

Example: publisher and subscriber in Python

Both scripts read the key from the DIAL_API_KEY environment variable.

Publisher (publish.py):

import json
import os
import urllib.request

BROKER = "http://127.0.0.1:8080"
API_KEY = os.environ["DIAL_API_KEY"]

def publish(topic: str, message: dict) -> int:
    req = urllib.request.Request(
        f"{BROKER}/publish/{topic}",
        data=json.dumps(message).encode(),
        headers={"Authorization": f"Bearer {API_KEY}"},
        method="POST",
    )
    with urllib.request.urlopen(req) as resp:
        return json.load(resp)["id"]

if __name__ == "__main__":
    print(publish("orders", {"sku": "abc-123", "qty": 2}))

Subscriber (subscribe.py):

import json
import os
import time
import urllib.request

BROKER = "http://127.0.0.1:8080"
API_KEY = os.environ["DIAL_API_KEY"]
TOPIC = "orders"

def fetch(after: int) -> list[dict]:
    url = f"{BROKER}/subscribe/{TOPIC}?after={after}&limit=100"
    req = urllib.request.Request(url, headers={"Authorization": f"Bearer {API_KEY}"})
    with urllib.request.urlopen(req) as resp:
        return json.load(resp)

def handle(message: dict) -> None:
    print("got", message["id"], json.loads(message["payload"]))

if __name__ == "__main__":
    cursor = 0  # persist this if the subscriber must survive restarts
    while True:
        batch = fetch(cursor)
        for message in batch:
            handle(message)
            cursor = message["id"]
        if not batch:
            time.sleep(1)

Run the broker, then in another terminal run DIAL_API_KEY=<your-key> python3 subscribe.py, and publish with DIAL_API_KEY=<your-key> python3 publish.py or the curl command above.

Build and test

cargo test

Limits

  • Single node only. Throughput and availability are those of one process and one SQLite file.
  • One shared API key for all clients. There is no per-client key, rotation, or expiry, so rotating the key means restarting the broker with the new value and updating every client.
  • No acknowledgements, retries, consumer groups, or retention policy. Messages are kept until the database file is removed.
  • Payloads are stored as text.