Skip to content

MARKDOWN SOURCE

Raw
# Local Decisions

A local-only Actix Web service that turns text into structured decisions. It speaks the same
API as [`gliner2`](https://pypi.org/project/gliner2/) — the Python library behind
`fastino/GLiNER2.5-Decide` — and runs the model itself: there is no Python runtime, no
inference server and no request ever leaves the machine.

The service is a port, not a wrapper. The word splitter, the schema encoding, the DeBERTa-v3
encoder and GLiNER's classification head are Rust ports of the Python code, so `POST
/v1/classify_text` with the same body returns the same answer as
`model.classify_text(text, tasks)`.

## Run

```sh
mise install
mise run dev
```

The server listens on `0.0.0.0:8080` by default. On first start it downloads the checkpoint
(~1.9 GB) into `./storage/fastino/GLiNER2.5-Decide` and reuses it afterwards; `storage/` is
git-ignored. Nothing is downloaded at build time and the Docker image ships no weights. Open
`http://127.0.0.1:8080/` for the playground.

```sh
curl -s http://127.0.0.1:8080/v1/classify_text \
  -H 'content-type: application/json' \
  -d '{
        "text": "Your mailbox is almost full. Click here in the next hour or we will delete every message.",
        "tasks": { "label": ["spam", "ham"] }
      }'
```

```json
{ "label": "spam" }
```

## API

Both endpoints take `tasks`: an object whose keys are head names. A value is either a list of
labels or an object with `labels` plus the options `gliner2` supports.

| Field | Type | Meaning |
| --- | --- | --- |
| `text` / `texts` | string / string[] | The text to classify. |
| `tasks` | object | Head name → label set or options. |
| `include_confidence` | bool | Report `{"label", "confidence"}` instead of a bare label. |
| `max_len` | int | Keep at most this many words of the text. |

| Field of a head | Type | Meaning |
| --- | --- | --- |
| `labels` | string[] or object | Candidate labels; an object maps each label to its description. |
| `multi_label` | bool | Return every label above `cls_threshold` instead of the best one. |
| `cls_threshold` | number | Sigmoid cutoff for multi-label heads (default `0.5`). |
| `prompt` | string | Instruction appended to the head, e.g. a question about a passage. |
| `examples` | [string, string][] | Few-shot `input → label` pairs. |
| `class_act` | string | `auto` (default), `sigmoid` or `softmax`. |

| Method | Path | Python equivalent |
| --- | --- | --- |
| `GET` | `/` | — (the playground page) |
| `GET` | `/health` | — |
| `POST` | `/v1/classify_text` | `model.classify_text(text, tasks, ...)` |
| `POST` | `/v1/batch_classify_text` | `model.batch_classify_text(texts, tasks, ...)` |
| `POST` | `/v1/extract` | `model.extract_json(text, structures, ...)` |

`GET /health` is public and never touches the model. The three `POST` routes all require a text
and return JSON.

### Playground

`GET /` serves a small HTML playground: one tab per `POST` route, each with a prepared, editable
example straight from this README — send it and read the JSON back. Every tab also offers a
**bad example (inverted)**: the same sentences flipped into their negative counterpart, so the
other reading is one click away. No assets are loaded from the network; the page is rendered
from memory at startup.

### API key

Set `API_KEY` and the three `POST /v1` routes demand it in the `X-Api-Key` header
(`Authorization: Bearer` works too) and answer `401` otherwise:

```sh
API_KEY=local-secret mise run dev

curl -s http://127.0.0.1:8080/v1/classify_text \
  -H 'X-Api-Key: local-secret' \
  -H 'content-type: application/json' \
  -d '{ "text": "Your mailbox is almost full. Click here in the next hour or we will delete every message.", "tasks": { "label": ["spam", "ham"] } }'
```

The playground page has the key injected when it is rendered, so it authenticates without any
configuration in the browser. `/health` stays public either way. Without `API_KEY` the service is
open, which is the default local mode — the comparison is constant-time and keys are never logged.

### `POST /v1/classify_text`

The equivalent of `model.classify_text(text, tasks, ...)`.

```sh
curl -s http://127.0.0.1:8080/v1/classify_text \
  -H 'content-type: application/json' \
  -d '{
        "text": "Battery dies before lunch, but the keyboard and the screen are the best I have used on a laptop.",
        "tasks": {
          "sentiment": ["positive", "negative", "mixed", "neutral"],
          "aspects": { "labels": ["battery", "keyboard", "screen"], "multi_label": true, "cls_threshold": 0.4 }
        },
        "include_confidence": true
      }'
```

```json
{
  "sentiment": { "label": "mixed", "confidence": 0.94 },
  "aspects": [
    { "label": "battery", "confidence": 0.88 },
    { "label": "keyboard", "confidence": 0.91 },
    { "label": "screen", "confidence": 0.79 }
  ]
}
```

Every example in [`docs/gliner2-5-decide.md`](docs/gliner2-5-decide.md) works as a request body, and
`cargo test` replays all of them through the real checkpoint. Note that the outputs printed in that
document are illustrations of the response shape, not the model's decisions — the accuracy check is
the benchmark, below.

### `POST /v1/extract`

The equivalent of `model.extract_json(text, structures, ...)`: a text plus the property names to
find, answered as a JSON object. A property that the model cannot find comes back as `null`.

```sh
curl -s http://127.0.0.1:8080/v1/extract \
  -H 'content-type: application/json' \
  -d '{
        "text": "the tool costs 2000 and you need to pay taxes of 210",
        "properties": ["taxes", "costs"]
      }'
```

```json
{ "taxes": "210", "costs": "2000" }
```

Values are the matched text, exactly as it appears in the input. `properties` accepts:

| Form | Meaning |
| --- | --- |
| `"taxes"` | One value, or `null`. |
| `"amounts::list"` | Every match, as a list. |
| `"total::list::the gross total"` | Every match, with a description to guide the model. |
| `{ "name": "taxes", "dtype": "list", "threshold": 0.3 }` | The same, spelled out. |

`threshold` sets the span score cutoff for the whole request, `include_confidence` and
`include_spans` add `confidence` and `start`/`end` character offsets, and `max_len` truncates the
text. Several properties are scored in one forward pass:

```sh
curl -s http://127.0.0.1:8080/v1/extract \
  -H 'content-type: application/json' \
  -d '{
        "text": "Berlin is the capital of Germany.",
        "properties": ["country"],
        "include_confidence": true
      }'
```

```json
{ "country": { "text": "Germany", "confidence": 0.98 } }
```

To ask for named structures instead, use `structures`; the answer is keyed by structure name and
holds one object per instance the model finds:

```json
{
  "text": "Invoice 1842 was paid on 4 May 2026.",
  "structures": { "invoice": ["number", "date"] }
}
```

```json
{ "invoice": [{ "number": "1842", "date": "4 May 2026" }] }
```

### `POST /v1/batch_classify_text`

The equivalent of `model.batch_classify_text(texts, tasks, ...)`. Same `tasks` object for every
text; the response is one object per text, in order.

```sh
curl -s http://127.0.0.1:8080/v1/batch_classify_text \
  -H 'content-type: application/json' \
  -d '{
        "texts": [
          "Payroll file has to be corrected before the 5pm cutoff or the whole company is paid late.",
          "I finished it in two nights. The ending is earned, the middle drags, and I would still hand it to a friend."
        ],
        "tasks": { "urgency": ["0", "1", "2", "3", "4", "5"] }
      }'
```

```json
[{ "urgency": "5" }, { "urgency": "7" }]
```

Invalid bodies are answered with `400` and a message, never with a guess:

```json
{ "error": "task 'intent' needs 'labels'" }
```

## Configuration

| Variable | Default | Description |
| --- | --- | --- |
| `PORT` | `8080` | TCP port for the HTTP server. |
| `STORAGE_DIR` | `/storage` | Root for downloaded model directories. |
| `MODEL_REPO` | `fastino/GLiNER2.5-Decide` | Hub repository the checkpoint comes from. |
| `MODEL_DIR` | `$STORAGE_DIR/$MODEL_REPO` | Exact directory the checkpoint is read from. |
| `HF_ENDPOINT` | `https://huggingface.co` | Hub endpoint, overridable for mirrors. |
| `API_KEY` | unset | Key the `POST /v1` routes require in `X-Api-Key` (or `Authorization: Bearer`); the playground embeds it. |
| `HF_TOKEN` | unset | Sent as a bearer token, for gated repositories. |
| `RUST_LOG` | `info` | Log filter. |

## Docker

```sh
docker build -t silenloc/dec_mod .
docker run -d --name dec_mod -p 8080:8080 -v dec_mod-storage:/storage silenloc/dec_mod
```

The image contains the binary and nothing else: the checkpoint is downloaded into the
`/storage` volume on first start and reused on every later start. Mount a volume so the weights
survive container replacement.

## Layout

| Path | Contents |
| --- | --- |
| `src/config.rs` | Environment configuration. |
| `src/engine/deberta.rs` | DeBERTa-v3 encoder port and the classification head. |
| `src/engine/schema.rs` | `gliner2`'s schema encoding, token assembly and label decoding. |
| `src/engine/tokenizer.rs` | Word splitter and the checkpoint tokenizer. |
| `src/engine/download.rs` | Download-on-start provisioning. |
| `src/decisions/mod.rs` | Request validation and response shaping. |
| `src/server/mod.rs` | Routes and the API key check. |
| `src/server/playground.rs` | The HTML page served at `/`. |

## Tests

```sh
mise run verify
```

Unit tests cover the word splitter, the relative-position buckets, the schema token layout,
request validation and label decoding. Two integration tests need the checkpoint and skip without
it:

- `tests/documented_examples.rs` replays every example from the model documentation and checks
  that the answers are well formed and drawn from the offered labels.
- `tests/extraction.rs` exercises `/v1/extract` against the real checkpoint, including the
  property-to-value shape and the request validation.
- `tests/benchmark.rs` reproduces the benchmark from the model card — exact-match accuracy on
  `fastino/fast-decisions`, published as **60.2%** — with the Rust engine. A faithful port lands
  next to that number; a broken one lands far below it. It is ignored by default because it needs
  the dataset:

  ```sh
  mkdir -p storage/fastino/fast-decisions
  for d in support_intent review_sentiment ticket_route; do
    curl -sL -o "storage/fastino/fast-decisions/$d.jsonl" \
      "https://huggingface.co/datasets/fastino/fast-decisions/resolve/main/$d.jsonl"
  done
  cargo test --release --test benchmark -- --ignored --nocapture
  ```

## Third-party

The checkpoint is Apache 2.0 (`fastino/GLiNER2.5-Decide`). The Rust dependencies and their
licenses are listed in [THIRD-PARTY-LICENSES.md](THIRD-PARTY-LICENSES.md).

Local Decisions

A local-only Actix Web service that turns text into structured decisions. It speaks the same API as gliner2 — the Python library behind fastino/GLiNER2.5-Decide — and runs the model itself: there is no Python runtime, no inference server and no request ever leaves the machine.

The service is a port, not a wrapper. The word splitter, the schema encoding, the DeBERTa-v3 encoder and GLiNER's classification head are Rust ports of the Python code, so POST /v1/classify_text with the same body returns the same answer as model.classify_text(text, tasks).

Run

mise install
mise run dev

The server listens on 0.0.0.0:8080 by default. On first start it downloads the checkpoint (~1.9 GB) into ./storage/fastino/GLiNER2.5-Decide and reuses it afterwards; storage/ is git-ignored. Nothing is downloaded at build time and the Docker image ships no weights. Open http://127.0.0.1:8080/ for the playground.

curl -s http://127.0.0.1:8080/v1/classify_text \
  -H 'content-type: application/json' \
  -d '{
        "text": "Your mailbox is almost full. Click here in the next hour or we will delete every message.",
        "tasks": { "label": ["spam", "ham"] }
      }'
{ "label": "spam" }

API

Both endpoints take tasks: an object whose keys are head names. A value is either a list of labels or an object with labels plus the options gliner2 supports.

FieldTypeMeaning
text / textsstring / string[]The text to classify.
tasksobjectHead name → label set or options.
include_confidenceboolReport {"label", "confidence"} instead of a bare label.
max_lenintKeep at most this many words of the text.
Field of a headTypeMeaning
labelsstring[] or objectCandidate labels; an object maps each label to its description.
multi_labelboolReturn every label above cls_threshold instead of the best one.
cls_thresholdnumberSigmoid cutoff for multi-label heads (default 0.5).
promptstringInstruction appended to the head, e.g. a question about a passage.
examples[string, string][]Few-shot input → label pairs.
class_actstringauto (default), sigmoid or softmax.
MethodPathPython equivalent
GET/— (the playground page)
GET/health—
POST/v1/classify_textmodel.classify_text(text, tasks, ...)
POST/v1/batch_classify_textmodel.batch_classify_text(texts, tasks, ...)
POST/v1/extractmodel.extract_json(text, structures, ...)

GET /health is public and never touches the model. The three POST routes all require a text and return JSON.

Playground

GET / serves a small HTML playground: one tab per POST route, each with a prepared, editable example straight from this README — send it and read the JSON back. Every tab also offers a bad example (inverted): the same sentences flipped into their negative counterpart, so the other reading is one click away. No assets are loaded from the network; the page is rendered from memory at startup.

API key

Set API_KEY and the three POST /v1 routes demand it in the X-Api-Key header (Authorization: Bearer works too) and answer 401 otherwise:

API_KEY=local-secret mise run dev

curl -s http://127.0.0.1:8080/v1/classify_text \
  -H 'X-Api-Key: local-secret' \
  -H 'content-type: application/json' \
  -d '{ "text": "Your mailbox is almost full. Click here in the next hour or we will delete every message.", "tasks": { "label": ["spam", "ham"] } }'

The playground page has the key injected when it is rendered, so it authenticates without any configuration in the browser. /health stays public either way. Without API_KEY the service is open, which is the default local mode — the comparison is constant-time and keys are never logged.

POST /v1/classify_text

The equivalent of model.classify_text(text, tasks, ...).

curl -s http://127.0.0.1:8080/v1/classify_text \
  -H 'content-type: application/json' \
  -d '{
        "text": "Battery dies before lunch, but the keyboard and the screen are the best I have used on a laptop.",
        "tasks": {
          "sentiment": ["positive", "negative", "mixed", "neutral"],
          "aspects": { "labels": ["battery", "keyboard", "screen"], "multi_label": true, "cls_threshold": 0.4 }
        },
        "include_confidence": true
      }'
{
  "sentiment": { "label": "mixed", "confidence": 0.94 },
  "aspects": [
    { "label": "battery", "confidence": 0.88 },
    { "label": "keyboard", "confidence": 0.91 },
    { "label": "screen", "confidence": 0.79 }
  ]
}

Every example in docs/gliner2-5-decide.md works as a request body, and cargo test replays all of them through the real checkpoint. Note that the outputs printed in that document are illustrations of the response shape, not the model's decisions — the accuracy check is the benchmark, below.

POST /v1/extract

The equivalent of model.extract_json(text, structures, ...): a text plus the property names to find, answered as a JSON object. A property that the model cannot find comes back as null.

curl -s http://127.0.0.1:8080/v1/extract \
  -H 'content-type: application/json' \
  -d '{
        "text": "the tool costs 2000 and you need to pay taxes of 210",
        "properties": ["taxes", "costs"]
      }'
{ "taxes": "210", "costs": "2000" }

Values are the matched text, exactly as it appears in the input. properties accepts:

FormMeaning
"taxes"One value, or null.
"amounts::list"Every match, as a list.
"total::list::the gross total"Every match, with a description to guide the model.
{ "name": "taxes", "dtype": "list", "threshold": 0.3 }The same, spelled out.

threshold sets the span score cutoff for the whole request, include_confidence and include_spans add confidence and start/end character offsets, and max_len truncates the text. Several properties are scored in one forward pass:

curl -s http://127.0.0.1:8080/v1/extract \
  -H 'content-type: application/json' \
  -d '{
        "text": "Berlin is the capital of Germany.",
        "properties": ["country"],
        "include_confidence": true
      }'
{ "country": { "text": "Germany", "confidence": 0.98 } }

To ask for named structures instead, use structures; the answer is keyed by structure name and holds one object per instance the model finds:

{
  "text": "Invoice 1842 was paid on 4 May 2026.",
  "structures": { "invoice": ["number", "date"] }
}
{ "invoice": [{ "number": "1842", "date": "4 May 2026" }] }

POST /v1/batch_classify_text

The equivalent of model.batch_classify_text(texts, tasks, ...). Same tasks object for every text; the response is one object per text, in order.

curl -s http://127.0.0.1:8080/v1/batch_classify_text \
  -H 'content-type: application/json' \
  -d '{
        "texts": [
          "Payroll file has to be corrected before the 5pm cutoff or the whole company is paid late.",
          "I finished it in two nights. The ending is earned, the middle drags, and I would still hand it to a friend."
        ],
        "tasks": { "urgency": ["0", "1", "2", "3", "4", "5"] }
      }'
[{ "urgency": "5" }, { "urgency": "7" }]

Invalid bodies are answered with 400 and a message, never with a guess:

{ "error": "task 'intent' needs 'labels'" }

Configuration

VariableDefaultDescription
PORT8080TCP port for the HTTP server.
STORAGE_DIR/storageRoot for downloaded model directories.
MODEL_REPOfastino/GLiNER2.5-DecideHub repository the checkpoint comes from.
MODEL_DIR$STORAGE_DIR/$MODEL_REPOExact directory the checkpoint is read from.
HF_ENDPOINThttps://huggingface.coHub endpoint, overridable for mirrors.
API_KEYunsetKey the POST /v1 routes require in X-Api-Key (or Authorization: Bearer); the playground embeds it.
HF_TOKENunsetSent as a bearer token, for gated repositories.
RUST_LOGinfoLog filter.

Docker

docker build -t silenloc/dec_mod .
docker run -d --name dec_mod -p 8080:8080 -v dec_mod-storage:/storage silenloc/dec_mod

The image contains the binary and nothing else: the checkpoint is downloaded into the /storage volume on first start and reused on every later start. Mount a volume so the weights survive container replacement.

Layout

PathContents
src/config.rsEnvironment configuration.
src/engine/deberta.rsDeBERTa-v3 encoder port and the classification head.
src/engine/schema.rsgliner2's schema encoding, token assembly and label decoding.
src/engine/tokenizer.rsWord splitter and the checkpoint tokenizer.
src/engine/download.rsDownload-on-start provisioning.
src/decisions/mod.rsRequest validation and response shaping.
src/server/mod.rsRoutes and the API key check.
src/server/playground.rsThe HTML page served at /.

Tests

mise run verify

Unit tests cover the word splitter, the relative-position buckets, the schema token layout, request validation and label decoding. Two integration tests need the checkpoint and skip without it:

  • tests/documented_examples.rs replays every example from the model documentation and checks that the answers are well formed and drawn from the offered labels.

  • tests/extraction.rs exercises /v1/extract against the real checkpoint, including the property-to-value shape and the request validation.

  • tests/benchmark.rs reproduces the benchmark from the model card — exact-match accuracy on fastino/fast-decisions, published as 60.2% — with the Rust engine. A faithful port lands next to that number; a broken one lands far below it. It is ignored by default because it needs the dataset:

    mkdir -p storage/fastino/fast-decisions
    for d in support_intent review_sentiment ticket_route; do
      curl -sL -o "storage/fastino/fast-decisions/$d.jsonl" \
        "https://huggingface.co/datasets/fastino/fast-decisions/resolve/main/$d.jsonl"
    done
    cargo test --release --test benchmark -- --ignored --nocapture
    

Third-party

The checkpoint is Apache 2.0 (fastino/GLiNER2.5-Decide). The Rust dependencies and their licenses are listed in THIRD-PARTY-LICENSES.md.