Everyone has a notebook that classifies something. Very few people have an endpoint a stranger can POST an image to at 2 a.m. (unauthenticated, on the open internet) and get a sane answer back. The distance between those two things is not model work. It is a series of boring decisions, and in LeafSentry every one of them exists because a real failure mode demanded it.
Decision 1: one inference at a time, on purpose
LeafSentry's throughput is terrible by design: inference is serialized per process behind a capacity limiter, running in a worker thread so CPU-bound work never blocks health checks.
This sounds like a confession. It is actually a load-bearing decision. The weights (~343 MB for the pinned ViT checkpoint) load lazily on first inference. If two requests arrive on a cold process, a naive handler races the load: double downloads, double memory, or a half-initialized model answering requests. Serializing inference makes lazy loading race-free by construction, keeps liveness probes answering during the load, and caps the resource envelope (the container runs with a 2 GB memory limit and 2 CPUs). The README says it plainly: this favors predictable resource use over throughput. It is not a high-volume inference server, and pretending otherwise would be the first step toward an outage.
Decision 2: readiness is a contract, not a vibe
/health/live and /health/ready are different endpoints with different jobs. Live answers once the process can serve HTTP. Ready only returns 200 after the model is actually loaded; until then it returns 503, on purpose.
That asymmetry is what makes deploys survivable. A cold start on a fresh container takes long enough that anything treating "process is up" as "service is ready" would route real traffic into failures. Callers that gate on readiness, including the portfolio demo which polls the public API, never see the loading window.
Decision 3: the container is a cage, and that is the point
The runtime container is deliberately hostile to cleverness: read-only root filesystem, all Linux capabilities dropped, no new privileges, a bounded process count, a tmpfs scratch space, and model weights served from a mounted cache with HF_HUB_OFFLINE=1, so the process physically cannot phone home for a different model. Which model runs is pinned to an exact HuggingFace revision, and deploys use immutable image tags tied to commits.
The container also cannot write anywhere interesting. Neither can it surprise you: deploys here have a rollback story because the previous image tag is still on disk. Public APIs on the open internet do not get "it worked on my machine" as an excuse; they get scanned within minutes.
Decision 4: honest counters, reset on purpose
The service now exposes a public GET /v1/stats: total predictions, breakdown by outcome (accepted, abstained, error), model identity, and process uptime. It is a read-only view over the same Prometheus counters that power internal monitoring. No per-request data leaves the service, and per-request data never will.
The counters reset when the process restarts, and that is a feature, not a bug. An all-time counter would need durable storage, and durable storage for a vanity number is scope creep. The demo page that displays this number says so, in the same sentence.
Decision 5: rate limiting lives at the edge
The public surface is unauthenticated, so abuse prevention cannot live in the app. nginx enforces a strict burst limit on /v1/predictions (two requests of headroom, tight concurrent-connection caps) and looser limits on everything else. The application's own quality gate is the second layer: garbage uploads are rejected with specific findings before inference ever runs, so even a hostile burst mostly costs bytes, not GPU-seconds.
What I would tell someone starting this
Ship the model behind a health endpoint and a rate limiter, then spend the next month on everything that can fail quietly: racing loads, redeploy resets, dependency audits that skip CPU-only wheels (pip-audit does; this repo now checks), and readiness probes that lie. The model is the part you will touch least.