Senior Systems Engineer, Workers AI

About Us

Available Locations: Austin, TX | New York, NY | San Francisco, CA or London, UK (Hybrid)

About the role

You'll design and build the core infrastructure that powers AI inference across Cloudflare's global network — real-time voice, frontier open LLMs, and customer-deployed models running on a heterogeneous fleet of GPUs and next-generation accelerators in hundreds of cities worldwide. Working alongside AI/ML engineers, hardware partners, and Cloudflare product teams, you'll solve hard problems in distributed systems and high-performance computing: sub-second model cold starts, multi-accelerator workload scheduling, efficient KV cache management, and a model deployment platform serving both Cloudflare and customers bringing their own models.

We're building an AI inference platform embedded in the fabric of the internet — something that doesn't exist yet — and this role puts you at the center of it. We're looking for high-agency systems engineers who are energized by foundational infrastructure problems and want to define how AI runs at the edge of the network.

Responsibilities

• Platform Architecture: Develop and maintain core components of the serverless inference platform to ensure high availability and scalability for Cloudflare users.
• Optimization & Performance: Optimize the model scheduling system to significantly increase efficiency and resource utilization across our inference infrastructure, and implement improvements to request routing logic to reduce latency for end-users.
• System Reliability: Drive significant, measurable improvements in the platform's reliability and resilience by identifying and mitigating systemic risks.
• Observability: Expand and refine the observability stack (metrics, logging, tracing) and fine-tune alerts to proactively identify and resolve production issues.
• Technical Leadership: Lead complex, cross-functional technical projects from initial concept and design through final deployment and operationalization.
• Mentorship: Act as a mentor to junior engineers and actively contribute to cultivating a strong, collaborative engineering culture within the team.

Desirable skills, Knowledge & Experience

• Proven experience in systems engineering, with a primary focus on distributed, high-performance systems.
• Expert proficiency in Rust programming, particularly in an asynchronous environment.
• Deep understanding and hands-on experience with relevant networking and application protocols (e.g., TCP, HTTP, WebSocket).
• Solid experience with scaling and performance optimization techniques, including load balancing and caching in a distributed environment.

Bonus Points

• Demonstrable experience with container orchestration platforms, specifically Kubernetes and/or Nomad .
• Familiarity with the unique architectural challenges involved in large-scale inference serving (e.g., LLMs and diffusion models).

Compensation may be adjusted depending on work location.

• For Bay Area based hires: Estimated annual salary of $194,000 - $266,000
• For New York City, New Jersey, Washington, Washington DC, and California (excluding Bay Area) based hires: Estimated annual salary of $185,000 - $254,000