Our client builds an enterprise LLM gateway. It connects customers’ applications and AI agents to multiple model providers through an OpenAI-compatible API. Customers register their own provider API keys. The gateway routes requests, reduces model costs, enforces policies, and tracks cost and usage per agent.
We are looking for a Senior Full-Stack Engineer with strong Python and TypeScript skills. You will build production SaaS features across the backend and frontend. Your work will cover database design, FastAPI endpoints, React screens, tests, and deployment. Features can also include Python middleware in the gateway.
Two areas will be your primary focus:
Payments and billing — building the commercial layer on top of the platform’s usage metering: subscription plans, usage-based billing, invoicing, credits, and entitlement enforcement.
Gateway guardrails and telemetry — building the policy enforcement and observability layer that makes the gateway safe, measurable, and auditable for enterprise customers.
You will join a small, senior, distributed team. You will own features through deployment and take part in architecture decisions. When needed, you will work directly with customers to define, build, deploy, and improve features that use AI.
What’s interesting about this project:
Own features from database design to the customer interface and production deployment.
Work with Python, FastAPI, React, TypeScript, Kubernetes, and Terraform.
Help decide how the product works and how the team builds it.
Build a gateway that connects customer applications and AI agents to multiple model providers.
Measure how gateway changes affect model costs, response quality, and latency.
What you’ll work on
Payment system integration (primary focus): Design and build the billing layer end to end — integrate Stripe (or an equivalent provider) for subscription plans, metered/usage-based charging, invoicing, credits and prepaid balances, and payment lifecycle handling.
LLM gateway guardrails (primary focus): Design and implement the policy layer that inspects traffic flowing through the gateway.
Telemetry and observability (primary focus): Extend the request telemetry pipeline that powers customer-facing analytics — enriching per-request records with cost, token, cache, routing, and guardrail metadata, building efficient time-series aggregations.
Gateway middleware: Build Python hooks inside the LLM proxy. Work includes prompt compaction, model routing based on request complexity, semantic caching, and response enrichment. Keep request handling correct and latency low.
Backend API development: Build and evolve FastAPI services with async SQLAlchemy and PostgreSQL, including multi-tenant data isolation, RBAC, session-based authentication, encrypted secret storage, and Alembic migrations.
Frontend development: Build customer-facing features in React and TypeScript — billing and plan management, guardrail configuration, telemetry and spend analytics views, admin/provisioning screens, and the LLM playground with streamed responses.
Multi-provider model management: Extend provider integrations, model catalogues, budgets, and rate limits so tenants can bring their own models and providers safely.
Technology Stack
Backend: Python 3.13, FastAPI, Uvicorn, Pydantic, Poetry
Data: PostgreSQL 16, SQLAlchemy 2 (async, asyncpg), Redis Stack
LLM Gateway: OpenAI-compatible proxy (LiteLLM) with custom Python middleware, multi-provider model routing
Frontend: React, TypeScript, Vite, Tailwind CSS, Chart.js
Payments: Stripe or equivalent — subscriptions, metered billing, webhooks, invoicing
Testing: pytest, Vitest, Testing Library
Infrastructure: Google Cloud Platform, Kubernetes (GKE), Helm, Terraform, Docker
CI/CD: GitHub Actions
Observability: Prometheus, Grafana, distributed tracing
Code quality: black, isort, flake8, ESLint, Prettier
Responsibilities
Own features end to end across the gateway middleware, backend API, and frontend
