We are building a self-serve Enterprise AI Gateway that standardizes how every team uses large language models with strong governance. As a Lead AI OPS Engineer, you will design automation-first onboarding, controls, and observability for access, cost, and logging across the platform. Join a small, fast-moving team and apply now.
Responsibilities
-
Design self-service onboarding for teams and agents, from initial request through to functional access
-
Automate key provisioning, rotation, and permission updates
-
Tune rate limits and quotas as adoption and usage increase
-
Deploy and configure models including Vertex AI endpoints, vendor fallbacks, and Model Armor
-
Operate MCP servers behind the gateway under the same governance rules as models
-
Develop Python extensions that integrate the gateway with other enterprise systems
-
Maintain the full platform as code using Terraform, Helm, and Jenkins
-
Monitor and protect production health across availability, latency, and cost with alerting
-
Assist teams using the platform and convert recurring questions into automation
Requirements
-
Proven 5+ years of experience using Python for automation, extensions, and integrations
-
Solid 5+ years of SRE experience with a track record of keeping production systems reliable
-
Deep expertise in Kubernetes, preferably on GKE
-
Hands-on experience with Google Vertex AI, especially endpoints for model serving and Model Armor
-
Practical proficiency with Terraform, Helm, and CI/CD using Jenkins
-
Working knowledge of GenAI/Agentic AI concepts (patterns, frameworks, protocols)
-
Production exposure to an AI gateway (LiteLLM, EPAM DIAL or similar) is highly appreciated
-
Very strong communication skills with the ability to explain platform behavior clearly to teams who use it daily
-
English proficiency at B2 level (Upper-Intermediate) or higher
Nice to have
-
Familiarity with GCP beyond GKE and Vertex AI, such as BigQuery, Cloud Run, and IAM
-
Experience building AI agents, for example using Google's Agent Development Kit (ADK)
-
Experience with AWS Bedrock
We offer
-
International projects with top brands
-
Work with global teams of highly skilled, diverse peers
-
Healthcare benefits
-
Employee financial programs
-
Paid time off and sick leave
-
Upskilling, reskilling and certification courses
-
Unlimited access to the LinkedIn Learning library and 22,000+ courses
-
Global career opportunities
-
Volunteer and community involvement opportunities
-
EPAM Employee Groups
-
Award-winning culture recognized by Glassdoor, Newsweek and LinkedIn
EPAM is an Equal Opportunity Employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, national origin, age, sexual orientation, gender identity or expression, disability, protected veteran status, or any other characteristic protected by applicable law.
