LiteLLM Candidate brief Open roles

Before you interview at LiteLLM

Company

  • Open source AI gateway: one OpenAI compatible API for 140+ model providers
  • Routing, fallbacks, spend tracking and access control, self hosted
  • Two ways in: the Python SDK inside an app, or the proxy a platform team runs for the whole company
  • The company is BerriAI, the product is LiteLLM
Founded
2023, Y Combinator W23
Founders
Krrish Dholakia (CEO), Ishaan Jaffer (CTO)
Team
About 20, most in San Francisco
Office
333 Broadway, SF. Moving soon
Customers
NVIDIA, Netflix, NASA, Adobe, AT&T

Product

Your app

  • Python SDK inside the app, or any OpenAI client pointed at the proxy
  • Switching models is a string change

completion(model="anthropic/claude-sonnet-5-5", messages=messages)

Proxy

  • Checks the virtual key, the team’s budget and the rate limit
  • Runs guardrails and the cache before anything leaves
  • Admin UI for the platform team running it

Router

  • Chooses which deployment serves the request
  • Load balancing, retries, fallbacks to another provider
  • Cache aware routing: self hosted vLLM pods reuse cached prefixes

Translation

  • Your request in the provider’s format, the response back in yours
  • Chat completions, responses, Anthropic messages, embeddings, images, batches
  • Streaming and tool calls
  • Provider launches break here first: the biggest share of open source issues
  • /v1/messages is the first path moving to Rust

140+ providers

  • OpenAI, Anthropic, Bedrock, Vertex AI, Azure, Mistral, Fireworks, self hosted vLLM
  • New models are expected to work on day one

Cost tracking

  • Prices each request per token for that provider
  • One durable charge record per request (UsageMeter, CostCompiler)
  • Budgets and reports read the same record (BudgetEnforcer, CostAggregator)
7,048Integration tests per release
877End to end tests per release
88.4%Test coverage
TueRelease day. LTS line 1.108 in pilot

What’s next

Roadmap ↗
  1. Oct 1
    Shipped

    Lens

    Agents review your agent traces and group the repeated failures. How it works

  2. Oct 8
    Shipped

    litellm-core

    65.6 MB smaller SDK install, more cuts planned

  3. Oct 14
    Shipped

    Decisions API

    /v1/decisions: yes or no, multiple choice and score questions

  4. Oct 31
    Shipped

    Cost accounting engine

    One charge record that budgets and reports both read

  5. Nov 2
    Running

    Tuesday releases and an LTS pilot

    LTS line 1.108 for teams that don’t upgrade weekly

  6. Dec 1
    In progress

    /v1/messages in Rust

    About 20x lower latency on common setups. Progress

  7. Ongoing
    Ongoing

    Stability

    426 bug fixes in October. User journeys under test: 521 to 7,048

  8. Ongoing
    Ongoing

    Agents

    MCP and agent gateway: every agent call, one chokepoint

Open roles

AI EngineerSan Francisco · Onsite$200K to $250K Senior Security EngineerRemote$220K to $260K Senior Security Engineer, Product & PlatformSan Francisco · Onsite$220K to $280K Senior Security Operations / Detection EngineerSan Francisco · Onsite$220K to $280K Senior Support Engineer, EMEA and APACIndia · Remote$120K to $170K Staff Engineer, AISan Francisco · Onsite$250K to $350K Staff Engineer, Systems and ReliabilitySan Francisco · Onsite$250K to $350K Forward Deployed EngineerUnited States$150K to $250K Forward Deployed Engineer (New Grad)United States$150K to $210K Founding Developer RelationsSan Francisco · Onsite$150K to $220K Product EngineerSan Francisco · Onsite$200K to $260K Technical Account ManagerUnited States$150K to $220K Account ExecutiveSan Francisco · Onsite$150K to $180K

Day to day

Every day
  • GitHub issues and Discord
  • Shared Slack channels with customers
  • Calls with their platform teams
Tuesday
  • Release candidates and stable releases ship
Friday
  • All hands opens with what shipped
Ownership
One or two owners per area: Rust, provider layer, cost tracking, MCP, security and infra, support. No PMs
Users
Issues often get fixed first, then the person who filed it gets a call
Customers
Engineers join calls with platform and AI leads at large companies. Expect to explain your work to them
Shipping
Small changes merge and ship. Bigger ones get a short design doc and a whiteboard session
Office
Engineers in person. Krrish and Ishaan sit with the team. Slack for the rest
Pace
Fast. Weekends aren’t expected

Team

Interviews

  1. 30 min

    Intro call with Krrish

    What you’ve built, what you owned, what you want next

  2. 1 to 1.5 hours, video

    Technical round

    System design and coding in the LiteLLM codebase. No LeetCode

  3. Part of a day

    Onsite in SF

    Meet the team, lunch together

  4. Base on the posting

    Offer

    The founders call you

How to prepare
Run the proxy locally and send requests through it
Read the provider code for a model you use, then the router
Bring one thing you built end to end: the problem, who used it, what was hard
Already contributing? Link your merged pull requests

FAQ

What’s the interview process?
  • Intro call with Krrish, 30 minutes
  • Technical round, 1 to 1.5 hours on video: system design and coding inside the LiteLLM codebase, no LeetCode
  • Onsite in San Francisco: meet the team, lunch together
  • The founders call you with the offer
What’s the compensation?
  • Every posting lists its base salary
  • Engineering ranges run from $150K to $350K, by role and level
  • Sales and account roles add commission
How much equity?
  • Most roles include equity
  • The amount depends on level and comes with the offer
Where’s the office?
  • 333 Broadway, San Francisco
  • Moving soon to a building across from the Transamerica Pyramid
  • Everyone who writes code works in person. A few roles, like support, are remote
Do you sponsor visas?
  • Yes, sponsorship and transfers inside the US
  • No new H-1B applications from outside the US
Do you help with relocation?
  • Yes, to San Francisco

Reading

All posts ↗

Open the posting ↗