Selected work / Case study
Ollama Provider
An OpenAI-compatible gateway for local Ollama models, combining per-client access policies, request scheduling, streaming, usage records, and a responsive administration dashboard.

The problem
Let applications share local models with separate credentials and usage limits, while giving operators a clear way to review requests, failures, and model availability.
Engineering focus
Coordinate model-aware scheduling, per-key admission, and streaming cleanup in a shared Python gateway, with asynchronous usage records and a separate administrative interface.
Implementation overview
Client application
OpenAI-compatible requests from client applications.
Client policies
Authentication, model restrictions, limits, and per-key admission.
Shared gateway
Model-aware scheduling, proxy streaming, and idempotent cleanup.
Ollama runtime
Pretrained inference in the separate Ollama runtime.
Usage and administration
Asynchronous SQLite records and an admin application serving a static Next.js UI.
Implementation
- Expose OpenAI-compatible model listing, chat, text-completion, and embedding routes; validate each request against model capabilities.
- Manage client keys with expiry, model restrictions, request/concurrency limits, optional token quotas, priorities, and prompt-logging choices.
- Group requests by model and release scheduler tickets and per-key slots when a response completes or its stream is interrupted.
- Write usage records asynchronously to SQLite; retrieve prompt previews with cursor pagination and full details on demand.
- Provide a React/Next.js dashboard for overview, client policies, usage, requests, prompts, and models, exported as static pages served by the admin FastAPI application.
Evidence & source
The supplied report documents mocked backend regression checks and frontend browser journeys. Screenshots show the built application with demo fixtures; the architecture figure is source-derived, not a production deployment observation.
Decisions & limitations
Screenshots show the actual UI with isolated demo data: metrics, hardware status, queues, model inventory, download progress, and answers are simulated. Live GPU throughput and production recovery were not tested in the supplied review. Scheduling assumes one gateway process; soft quotas can be exceeded by in-flight requests. Prompt logging needs operator privacy controls.