PawID — a local agent that reads pet vaccination cards

A vet clinic sees dozens of animals a day, and every one arrives with a paper vaccination card. Somebody has to read it and type it into a system: product names, lot numbers, dates. That's back-office work built entirely on reading a messy document and making judgement calls — and it's where the errors come from.

PawID does that reading on the user's own machine. It turns a photo of a paper vaccination card into a structured, auditable digital record for the pet, using QVAC's OCR model for the hard part. No API keys, no cloud, no inference cost. The data never leaves the device — which matters, because a pet's medical history carries the owner's name, phone number and clinic.

Demo video: https://youtu.be/dnDQq91iTUo Repository: https://github.com/Luis-Yanza/PawID Logo: https://drive.google.com/thumbnail?id=1bWf_XXLN_jgmSW1w4edLwJjidUXIrY-8&sz=w1024

---

How it works

The pipeline has two steps, designed for small models and modest hardware.

Step 1 — image → text (QVAC, local)

The photo goes to QVAC's OCR_LATIN model through @qvac/sdk, running on the ONNX engine. It returns text blocks, each with a confidence score and a bounding box. Both are used later, and both matter.

Before the photo is sent, the browser downscales it to a resolution chosen by the detected hardware. This is not cosmetic: on a card with small print, sending it too small makes the recognizer confuse characters.

Step 2 — text → structured data (deterministic)

The blocks are turned into a pet record with rules, not with a second model:

• Labelled fields (Nombre: Rocky, Fecha: 05/04/2025) are read as label/value pairs.
• Tables — most real cards use them — are resolved by geometry. Values are tied to their vaccine by sharing the same row in the bounding boxes, and the table's column headers (Vacuna | Lote | Fecha | Próx. Dosis | Firma) tell the parser what each column means.
• Products are matched against a veterinary catalogue, tolerant to OCR errors (it reads ANTIRABICA, the catalogue says antirrábica).
• Dates are normalized from eight written formats to ISO.

Reading the column headers turned out to be essential. One of the test cards has only a "Próx. Dosis" column and no application-date column at all. Without reading the header, that future date was being stored as the application date, and the vaccine status came out wrong.

Step 3 — human review, then storage

Nothing is written to the pet's history until a person confirms it. The review screen puts the photo on the left and the extracted fields on the right, with uncertain fields flagged in amber and the reason spelled out:

• the OCR read this with low confidence (59%)
• day and month may be swapped
• the OCR attached an extra character to the start of the date; it was corrected automatically
• the document names a cat vaccine, but the pet is registered as another species

Step 4 — the digital ID

Each pet gets a short ID (PET-7F3K) and a card showing its history and a status per vaccine — 🟢 up to date, 🟡 due soon, 🔴 overdue — computed from the last event plus the booster interval in an editable catalogue (sourced from WSAVA 2024 and AAHA 2022 guidelines). Every history entry links back to the raw OCR text, so any record can be audited in seconds.

---

Where the inference happens

Permalinks pinned to commit 28c92c8:

Every QVAC call is new code, written during the hackathon.

---

Models, hardware and latency

Every step runs in its own process. QVAC's native worker does not survive using the ONNX engine and then llama.cpp within one process lifetime — it dies with SIGSEGV. Isolating the steps also keeps peak memory to one model at a time, which is the only thing that fits in 8 GB.

---

Decisions driven by measurement, not by assumption

Step 2 is deterministic, not an LLM. The llama.cpp binary QVAC ships for Intel Macs kills the worker with SIGSEGV on any prompt longer than ~30 tokens. A 26-token prompt answers fine; a 56-token one crashes. Ruled out systematically: three models (QWEN3_1_7B_INST_Q4, LLAMA_3_2_1B_INST_Q4_0, QWEN3_600M_INST_Q4), ctx_size from 1024 to 4096, no_mmap, explicit CPU device, and @qvac/llm-llamacpp 0.46.0 — which doesn't even start the worker.

This turned out to have a real advantage. Uncertainty flags come from the confidence the OCR model itself reports per block, which is a better-grounded signal than asking a 1.7B model to grade its own work. And by construction, the system cannot invent a value that isn't in the image.

Hardware adaptation only changes what the evidence supports. server/perfil-maquina.js detects physical cores (not logical — an i5 with hyperthreading reports 4 and has 2), RAM and architecture, then sets the inference backend and the image resolution. It deliberately does not touch magRatio, nThreads, canvasSize or recognizerBatchSize, even though QVAC's schema accepts them — those were measured, and hand-tuned values came out worse than the SDK defaults:

A first version of the resolution profile used 900 px, measured against a synthetic card with large type. That was a method error: on a real card with small print, 900 px reads lot T77889 as 777889 — turning a T into a 7. At 1200 px it reads correctly. A misread lot number is worse than a few extra seconds.

---

Known limitations

Stated precisely, because a declared limit is worth more than a demo that hides where it breaks.

1. The OCR confuses visually similar glyphs. T→7, B→8, Q→0, and at low resolution :→;. This is the recognition model, not fixable from PawID's code. What the design does is refuse to hide it: the value sits next to the photo for correction, and nothing reaches the history without human confirmation.
2. Tested on a synthetic card and two AI-generated card images — not on a photograph of a real card. The generated ones have independent layouts we didn't design, with tables, folds, shadows and handwritten signatures, so they do test generalization. But a real phone photo of a real card is still untested.
3. macOS 12 needs two local patches (documented and applied automatically by a postinstall script). QVAC requires macOS 14+; the development machine can't run it.
4. One instance at a time. Two PawID servers in parallel fight over QVAC's worker state and the second fails.
5. Dogs and cats only. Adding a species means adding rows to a JSON catalogue, not writing code.

---

Running it

git clone https://github.com/Luis-Yanza/PawID.git
cd PawID
npm install
npm start

Verified from a clean clone: 186 packages in ~2 min, the patch applies itself, and reading the bundled test card returns the three events correctly. Requires Node.js 20+ and about 6 GB of disk.