MARKDOWN SOURCE
# Local Decisions
A local-only Actix Web service that turns text into structured decisions. It speaks the same
API as [`gliner2`](https://pypi.org/project/gliner2/) — the Python library behind
`fastino/GLiNER2.5-Decide` — and runs the model itself: there is no Python runtime, no
inference server and no request ever leaves the machine.
The service is a port, not a wrapper. The word splitter, the schema encoding, the DeBERTa-v3
encoder and GLiNER's classification head are Rust ports of the Python code, so `POST
/v1/classify_text` with the same body returns the same answer as
`model.classify_text(text, tasks)`.
## Run
```sh
mise install
mise run dev
```
The server listens on `0.0.0.0:8080` by default. On first start it downloads the checkpoint
(~1.9 GB) into `./storage/fastino/GLiNER2.5-Decide` and reuses it afterwards; `storage/` is
git-ignored. Nothing is downloaded at build time and the Docker image ships no weights. Open
`http://127.0.0.1:8080/` for the playground.
```sh
curl -s http://127.0.0.1:8080/v1/classify_text \
-H 'content-type: application/json' \
-d '{
"text": "Your mailbox is almost full. Click here in the next hour or we will delete every message.",
"tasks": { "label": ["spam", "ham"] }
}'
```
```json
{ "label": "spam" }
```
## API
Both endpoints take `tasks`: an object whose keys are head names. A value is either a list of
labels or an object with `labels` plus the options `gliner2` supports.
| Field | Type | Meaning |
| --- | --- | --- |
| `text` / `texts` | string / string[] | The text to classify. |
| `tasks` | object | Head name → label set or options. |
| `include_confidence` | bool | Report `{"label", "confidence"}` instead of a bare label. |
| `max_len` | int | Keep at most this many words of the text. |
| Field of a head | Type | Meaning |
| --- | --- | --- |
| `labels` | string[] or object | Candidate labels; an object maps each label to its description. |
| `multi_label` | bool | Return every label above `cls_threshold` instead of the best one. |
| `cls_threshold` | number | Sigmoid cutoff for multi-label heads (default `0.5`). |
| `prompt` | string | Instruction appended to the head, e.g. a question about a passage. |
| `examples` | [string, string][] | Few-shot `input → label` pairs. |
| `class_act` | string | `auto` (default), `sigmoid` or `softmax`. |
| Method | Path | Python equivalent |
| --- | --- | --- |
| `GET` | `/` | — (the playground page) |
| `GET` | `/health` | — |
| `POST` | `/v1/classify_text` | `model.classify_text(text, tasks, ...)` |
| `POST` | `/v1/batch_classify_text` | `model.batch_classify_text(texts, tasks, ...)` |
| `POST` | `/v1/extract` | `model.extract_json(text, structures, ...)` |
`GET /health` is public and never touches the model. The three `POST` routes all require a text
and return JSON.
### Playground
`GET /` serves a small HTML playground: one tab per `POST` route, each with a prepared, editable
example straight from this README — send it and read the JSON back. Every tab also offers a
**bad example (inverted)**: the same sentences flipped into their negative counterpart, so the
other reading is one click away. No assets are loaded from the network; the page is rendered
from memory at startup.
### API key
Set `API_KEY` and the three `POST /v1` routes demand it in the `X-Api-Key` header
(`Authorization: Bearer` works too) and answer `401` otherwise:
```sh
API_KEY=local-secret mise run dev
curl -s http://127.0.0.1:8080/v1/classify_text \
-H 'X-Api-Key: local-secret' \
-H 'content-type: application/json' \
-d '{ "text": "Your mailbox is almost full. Click here in the next hour or we will delete every message.", "tasks": { "label": ["spam", "ham"] } }'
```
The playground page has the key injected when it is rendered, so it authenticates without any
configuration in the browser. `/health` stays public either way. Without `API_KEY` the service is
open, which is the default local mode — the comparison is constant-time and keys are never logged.
### `POST /v1/classify_text`
The equivalent of `model.classify_text(text, tasks, ...)`.
```sh
curl -s http://127.0.0.1:8080/v1/classify_text \
-H 'content-type: application/json' \
-d '{
"text": "Battery dies before lunch, but the keyboard and the screen are the best I have used on a laptop.",
"tasks": {
"sentiment": ["positive", "negative", "mixed", "neutral"],
"aspects": { "labels": ["battery", "keyboard", "screen"], "multi_label": true, "cls_threshold": 0.4 }
},
"include_confidence": true
}'
```
```json
{
"sentiment": { "label": "mixed", "confidence": 0.94 },
"aspects": [
{ "label": "battery", "confidence": 0.88 },
{ "label": "keyboard", "confidence": 0.91 },
{ "label": "screen", "confidence": 0.79 }
]
}
```
Every example in [`docs/gliner2-5-decide.md`](docs/gliner2-5-decide.md) works as a request body, and
`cargo test` replays all of them through the real checkpoint. Note that the outputs printed in that
document are illustrations of the response shape, not the model's decisions — the accuracy check is
the benchmark, below.
### `POST /v1/extract`
The equivalent of `model.extract_json(text, structures, ...)`: a text plus the property names to
find, answered as a JSON object. A property that the model cannot find comes back as `null`.
```sh
curl -s http://127.0.0.1:8080/v1/extract \
-H 'content-type: application/json' \
-d '{
"text": "the tool costs 2000 and you need to pay taxes of 210",
"properties": ["taxes", "costs"]
}'
```
```json
{ "taxes": "210", "costs": "2000" }
```
Values are the matched text, exactly as it appears in the input. `properties` accepts:
| Form | Meaning |
| --- | --- |
| `"taxes"` | One value, or `null`. |
| `"amounts::list"` | Every match, as a list. |
| `"total::list::the gross total"` | Every match, with a description to guide the model. |
| `{ "name": "taxes", "dtype": "list", "threshold": 0.3 }` | The same, spelled out. |
`threshold` sets the span score cutoff for the whole request, `include_confidence` and
`include_spans` add `confidence` and `start`/`end` character offsets, and `max_len` truncates the
text. Several properties are scored in one forward pass:
```sh
curl -s http://127.0.0.1:8080/v1/extract \
-H 'content-type: application/json' \
-d '{
"text": "Berlin is the capital of Germany.",
"properties": ["country"],
"include_confidence": true
}'
```
```json
{ "country": { "text": "Germany", "confidence": 0.98 } }
```
To ask for named structures instead, use `structures`; the answer is keyed by structure name and
holds one object per instance the model finds:
```json
{
"text": "Invoice 1842 was paid on 4 May 2026.",
"structures": { "invoice": ["number", "date"] }
}
```
```json
{ "invoice": [{ "number": "1842", "date": "4 May 2026" }] }
```
### `POST /v1/batch_classify_text`
The equivalent of `model.batch_classify_text(texts, tasks, ...)`. Same `tasks` object for every
text; the response is one object per text, in order.
```sh
curl -s http://127.0.0.1:8080/v1/batch_classify_text \
-H 'content-type: application/json' \
-d '{
"texts": [
"Payroll file has to be corrected before the 5pm cutoff or the whole company is paid late.",
"I finished it in two nights. The ending is earned, the middle drags, and I would still hand it to a friend."
],
"tasks": { "urgency": ["0", "1", "2", "3", "4", "5"] }
}'
```
```json
[{ "urgency": "5" }, { "urgency": "7" }]
```
Invalid bodies are answered with `400` and a message, never with a guess:
```json
{ "error": "task 'intent' needs 'labels'" }
```
## Configuration
| Variable | Default | Description |
| --- | --- | --- |
| `PORT` | `8080` | TCP port for the HTTP server. |
| `STORAGE_DIR` | `/storage` | Root for downloaded model directories. |
| `MODEL_REPO` | `fastino/GLiNER2.5-Decide` | Hub repository the checkpoint comes from. |
| `MODEL_DIR` | `$STORAGE_DIR/$MODEL_REPO` | Exact directory the checkpoint is read from. |
| `HF_ENDPOINT` | `https://huggingface.co` | Hub endpoint, overridable for mirrors. |
| `API_KEY` | unset | Key the `POST /v1` routes require in `X-Api-Key` (or `Authorization: Bearer`); the playground embeds it. |
| `HF_TOKEN` | unset | Sent as a bearer token, for gated repositories. |
| `RUST_LOG` | `info` | Log filter. |
## Docker
```sh
docker build -t silenloc/dec_mod .
docker run -d --name dec_mod -p 8080:8080 -v dec_mod-storage:/storage silenloc/dec_mod
```
The image contains the binary and nothing else: the checkpoint is downloaded into the
`/storage` volume on first start and reused on every later start. Mount a volume so the weights
survive container replacement.
## Layout
| Path | Contents |
| --- | --- |
| `src/config.rs` | Environment configuration. |
| `src/engine/deberta.rs` | DeBERTa-v3 encoder port and the classification head. |
| `src/engine/schema.rs` | `gliner2`'s schema encoding, token assembly and label decoding. |
| `src/engine/tokenizer.rs` | Word splitter and the checkpoint tokenizer. |
| `src/engine/download.rs` | Download-on-start provisioning. |
| `src/decisions/mod.rs` | Request validation and response shaping. |
| `src/server/mod.rs` | Routes and the API key check. |
| `src/server/playground.rs` | The HTML page served at `/`. |
## Tests
```sh
mise run verify
```
Unit tests cover the word splitter, the relative-position buckets, the schema token layout,
request validation and label decoding. Two integration tests need the checkpoint and skip without
it:
- `tests/documented_examples.rs` replays every example from the model documentation and checks
that the answers are well formed and drawn from the offered labels.
- `tests/extraction.rs` exercises `/v1/extract` against the real checkpoint, including the
property-to-value shape and the request validation.
- `tests/benchmark.rs` reproduces the benchmark from the model card — exact-match accuracy on
`fastino/fast-decisions`, published as **60.2%** — with the Rust engine. A faithful port lands
next to that number; a broken one lands far below it. It is ignored by default because it needs
the dataset:
```sh
mkdir -p storage/fastino/fast-decisions
for d in support_intent review_sentiment ticket_route; do
curl -sL -o "storage/fastino/fast-decisions/$d.jsonl" \
"https://huggingface.co/datasets/fastino/fast-decisions/resolve/main/$d.jsonl"
done
cargo test --release --test benchmark -- --ignored --nocapture
```
## Third-party
The checkpoint is Apache 2.0 (`fastino/GLiNER2.5-Decide`). The Rust dependencies and their
licenses are listed in [THIRD-PARTY-LICENSES.md](THIRD-PARTY-LICENSES.md).Local Decisions
A local-only Actix Web service that turns text into structured decisions. It speaks the same
API as gliner2 — the Python library behind
fastino/GLiNER2.5-Decide — and runs the model itself: there is no Python runtime, no
inference server and no request ever leaves the machine.
The service is a port, not a wrapper. The word splitter, the schema encoding, the DeBERTa-v3
encoder and GLiNER's classification head are Rust ports of the Python code, so POST /v1/classify_text with the same body returns the same answer as
model.classify_text(text, tasks).
Run
mise install
mise run dev
The server listens on 0.0.0.0:8080 by default. On first start it downloads the checkpoint
(~1.9 GB) into ./storage/fastino/GLiNER2.5-Decide and reuses it afterwards; storage/ is
git-ignored. Nothing is downloaded at build time and the Docker image ships no weights. Open
http://127.0.0.1:8080/ for the playground.
curl -s http://127.0.0.1:8080/v1/classify_text \
-H 'content-type: application/json' \
-d '{
"text": "Your mailbox is almost full. Click here in the next hour or we will delete every message.",
"tasks": { "label": ["spam", "ham"] }
}'
{ "label": "spam" }
API
Both endpoints take tasks: an object whose keys are head names. A value is either a list of
labels or an object with labels plus the options gliner2 supports.
| Field | Type | Meaning |
|---|---|---|
text / texts | string / string[] | The text to classify. |
tasks | object | Head name → label set or options. |
include_confidence | bool | Report {"label", "confidence"} instead of a bare label. |
max_len | int | Keep at most this many words of the text. |
| Field of a head | Type | Meaning |
|---|---|---|
labels | string[] or object | Candidate labels; an object maps each label to its description. |
multi_label | bool | Return every label above cls_threshold instead of the best one. |
cls_threshold | number | Sigmoid cutoff for multi-label heads (default 0.5). |
prompt | string | Instruction appended to the head, e.g. a question about a passage. |
examples | [string, string][] | Few-shot input → label pairs. |
class_act | string | auto (default), sigmoid or softmax. |
| Method | Path | Python equivalent |
|---|---|---|
GET | / | — (the playground page) |
GET | /health | — |
POST | /v1/classify_text | model.classify_text(text, tasks, ...) |
POST | /v1/batch_classify_text | model.batch_classify_text(texts, tasks, ...) |
POST | /v1/extract | model.extract_json(text, structures, ...) |
GET /health is public and never touches the model. The three POST routes all require a text
and return JSON.
Playground
GET / serves a small HTML playground: one tab per POST route, each with a prepared, editable
example straight from this README — send it and read the JSON back. Every tab also offers a
bad example (inverted): the same sentences flipped into their negative counterpart, so the
other reading is one click away. No assets are loaded from the network; the page is rendered
from memory at startup.
API key
Set API_KEY and the three POST /v1 routes demand it in the X-Api-Key header
(Authorization: Bearer works too) and answer 401 otherwise:
API_KEY=local-secret mise run dev
curl -s http://127.0.0.1:8080/v1/classify_text \
-H 'X-Api-Key: local-secret' \
-H 'content-type: application/json' \
-d '{ "text": "Your mailbox is almost full. Click here in the next hour or we will delete every message.", "tasks": { "label": ["spam", "ham"] } }'
The playground page has the key injected when it is rendered, so it authenticates without any
configuration in the browser. /health stays public either way. Without API_KEY the service is
open, which is the default local mode — the comparison is constant-time and keys are never logged.
POST /v1/classify_text
The equivalent of model.classify_text(text, tasks, ...).
curl -s http://127.0.0.1:8080/v1/classify_text \
-H 'content-type: application/json' \
-d '{
"text": "Battery dies before lunch, but the keyboard and the screen are the best I have used on a laptop.",
"tasks": {
"sentiment": ["positive", "negative", "mixed", "neutral"],
"aspects": { "labels": ["battery", "keyboard", "screen"], "multi_label": true, "cls_threshold": 0.4 }
},
"include_confidence": true
}'
{
"sentiment": { "label": "mixed", "confidence": 0.94 },
"aspects": [
{ "label": "battery", "confidence": 0.88 },
{ "label": "keyboard", "confidence": 0.91 },
{ "label": "screen", "confidence": 0.79 }
]
}
Every example in docs/gliner2-5-decide.md works as a request body, and
cargo test replays all of them through the real checkpoint. Note that the outputs printed in that
document are illustrations of the response shape, not the model's decisions — the accuracy check is
the benchmark, below.
POST /v1/extract
The equivalent of model.extract_json(text, structures, ...): a text plus the property names to
find, answered as a JSON object. A property that the model cannot find comes back as null.
curl -s http://127.0.0.1:8080/v1/extract \
-H 'content-type: application/json' \
-d '{
"text": "the tool costs 2000 and you need to pay taxes of 210",
"properties": ["taxes", "costs"]
}'
{ "taxes": "210", "costs": "2000" }
Values are the matched text, exactly as it appears in the input. properties accepts:
| Form | Meaning |
|---|---|
"taxes" | One value, or null. |
"amounts::list" | Every match, as a list. |
"total::list::the gross total" | Every match, with a description to guide the model. |
{ "name": "taxes", "dtype": "list", "threshold": 0.3 } | The same, spelled out. |
threshold sets the span score cutoff for the whole request, include_confidence and
include_spans add confidence and start/end character offsets, and max_len truncates the
text. Several properties are scored in one forward pass:
curl -s http://127.0.0.1:8080/v1/extract \
-H 'content-type: application/json' \
-d '{
"text": "Berlin is the capital of Germany.",
"properties": ["country"],
"include_confidence": true
}'
{ "country": { "text": "Germany", "confidence": 0.98 } }
To ask for named structures instead, use structures; the answer is keyed by structure name and
holds one object per instance the model finds:
{
"text": "Invoice 1842 was paid on 4 May 2026.",
"structures": { "invoice": ["number", "date"] }
}
{ "invoice": [{ "number": "1842", "date": "4 May 2026" }] }
POST /v1/batch_classify_text
The equivalent of model.batch_classify_text(texts, tasks, ...). Same tasks object for every
text; the response is one object per text, in order.
curl -s http://127.0.0.1:8080/v1/batch_classify_text \
-H 'content-type: application/json' \
-d '{
"texts": [
"Payroll file has to be corrected before the 5pm cutoff or the whole company is paid late.",
"I finished it in two nights. The ending is earned, the middle drags, and I would still hand it to a friend."
],
"tasks": { "urgency": ["0", "1", "2", "3", "4", "5"] }
}'
[{ "urgency": "5" }, { "urgency": "7" }]
Invalid bodies are answered with 400 and a message, never with a guess:
{ "error": "task 'intent' needs 'labels'" }
Configuration
| Variable | Default | Description |
|---|---|---|
PORT | 8080 | TCP port for the HTTP server. |
STORAGE_DIR | /storage | Root for downloaded model directories. |
MODEL_REPO | fastino/GLiNER2.5-Decide | Hub repository the checkpoint comes from. |
MODEL_DIR | $STORAGE_DIR/$MODEL_REPO | Exact directory the checkpoint is read from. |
HF_ENDPOINT | https://huggingface.co | Hub endpoint, overridable for mirrors. |
API_KEY | unset | Key the POST /v1 routes require in X-Api-Key (or Authorization: Bearer); the playground embeds it. |
HF_TOKEN | unset | Sent as a bearer token, for gated repositories. |
RUST_LOG | info | Log filter. |
Docker
docker build -t silenloc/dec_mod .
docker run -d --name dec_mod -p 8080:8080 -v dec_mod-storage:/storage silenloc/dec_mod
The image contains the binary and nothing else: the checkpoint is downloaded into the
/storage volume on first start and reused on every later start. Mount a volume so the weights
survive container replacement.
Layout
| Path | Contents |
|---|---|
src/config.rs | Environment configuration. |
src/engine/deberta.rs | DeBERTa-v3 encoder port and the classification head. |
src/engine/schema.rs | gliner2's schema encoding, token assembly and label decoding. |
src/engine/tokenizer.rs | Word splitter and the checkpoint tokenizer. |
src/engine/download.rs | Download-on-start provisioning. |
src/decisions/mod.rs | Request validation and response shaping. |
src/server/mod.rs | Routes and the API key check. |
src/server/playground.rs | The HTML page served at /. |
Tests
mise run verify
Unit tests cover the word splitter, the relative-position buckets, the schema token layout, request validation and label decoding. Two integration tests need the checkpoint and skip without it:
-
tests/documented_examples.rsreplays every example from the model documentation and checks that the answers are well formed and drawn from the offered labels. -
tests/extraction.rsexercises/v1/extractagainst the real checkpoint, including the property-to-value shape and the request validation. -
tests/benchmark.rsreproduces the benchmark from the model card — exact-match accuracy onfastino/fast-decisions, published as 60.2% — with the Rust engine. A faithful port lands next to that number; a broken one lands far below it. It is ignored by default because it needs the dataset:mkdir -p storage/fastino/fast-decisions for d in support_intent review_sentiment ticket_route; do curl -sL -o "storage/fastino/fast-decisions/$d.jsonl" \ "https://huggingface.co/datasets/fastino/fast-decisions/resolve/main/$d.jsonl" done cargo test --release --test benchmark -- --ignored --nocapture
Third-party
The checkpoint is Apache 2.0 (fastino/GLiNER2.5-Decide). The Rust dependencies and their
licenses are listed in THIRD-PARTY-LICENSES.md.