Our client is building an enterprise LLM gateway platform — an OpenAI-compatible API layer that routes traffic to multiple model providers while cutting inference cost, enforcing policy, and giving customers full visibility into what their AI workloads actually cost. It serves engineering teams that run LLM workloads in production under real budget, compliance, and governance constraints.
We are looking for a Senior Full-Stack Engineer who is comfortable owning features end to end — from the PostgreSQL schema and the FastAPI endpoint through to the frontend screen the customer actually clicks. This is a single combined role, not a backend seat with occasional frontend work: most features in this product span the gateway middleware, the management API, and the dashboard, and you will be expected to ship all three layers.
Two areas will be your primary focus:
Payments and billing — building the commercial layer on top of the platform’s usage metering: subscription plans, usage-based billing, invoicing, credits, and entitlement enforcement.
Gateway guardrails and telemetry — building the policy enforcement and observability layer that makes the gateway safe, measurable, and auditable for enterprise customers.
You will join a small, senior, distributed team where engineers own their features through to production, take part in architectural decisions, and have real influence over how the platform evolves.
Type of engagement: outstaffing team extension for ongoing collaboration (with daily overlap until 7:00 PM EEST)
Project stack:
Backend: Python 3.13, FastAPI, Uvicorn, Pydantic, Poetry
Data: PostgreSQL 16, SQLAlchemy 2 (async, asyncpg), Redis Stack
LLM Gateway: OpenAI-compatible proxy (LiteLLM) with custom Python middleware, multi-provider model routing
Frontend: React, TypeScript, Vite, Tailwind CSS, Chart.js
Payments: Stripe or equivalent — subscriptions, metered billing, webhooks, invoicing
Testing: pytest, Vitest, Testing Library
Infrastructure: Google Cloud Platform, Kubernetes (GKE), Helm, Terraform, Docker
CI/CD: GitHub Actions
Observability: Prometheus, Grafana, distributed tracing
Code quality: black, isort, flake8, ESLint, Prettier
What’s interesting about this project:
Real LLM infrastructure work — middleware in the request path of live model traffic, where a good optimization directly saves customers money
Measurable impact: every feature is expressed in tokens saved, cost avoided, or requests correctly handled
Substantial ownership of two significant domains — billing and guardrails — rather than incremental work on someone else’s design
Multi-provider LLM work across the major model vendors, plus prompt optimization and cost/quality evaluation
Modern stack with architectural freedom: Python 3+, FastAPI, React, TypeScript, Kubernetes, Terraform
Scope of tasks and ownership:
Own features end to end across the gateway middleware, backend API, and frontend — including schema design, API contracts, UI, tests, and rollout
Build and operate the subscription and usage-based billing system, including payment provider integration, webhook reliability, usage-to-invoice mapping, and entitlement enforcement
Design and implement guardrails that inspect and enforce policy on LLM prompts and completions, with per-tenant configuration and full auditability
Extend the telemetry pipeline and analytics aggregations that power customer-facing cost and usage reporting, and add tracing, metrics, and alerting across services
Write middleware that runs in the live request path, treating latency, failure modes, and graceful degradation as first-class design concerns
Design multi-tenant data models with correct isolation between organizations, workspaces, and keys
Write meaningful automated tests for the code you ship
Participate in architecture decisions, code reviews, and planning; document decisions for a distributed team
Handle sensitive data responsibly — secret encryption at rest, least-privilege acces
