Member of Technical Staff (Software Engineer, GPU Cluster Infrastructure) — United States

For our client, we are seeking a Member of Technical Staff (Software Engineer, GPU Cluster Infrastructure) to join the team of a leader in the Information Technology space.

This role will focus on building and operating the core infrastructure that powers large-scale machine learning and inference workloads across a distributed environment. You will partner closely with researchers and engineers to create a self-serve compute platform that improves reliability, efficiency, and scalability. The work sits at the intersection of Kubernetes, GPU fleet management, and cloud infrastructure, with an emphasis on enabling strong performance across training and real-time inference use cases.

Location: Remote

Compensation: $150,000 – $180,000 annually

Responsibilities

• Build and design a self-serve compute platform for inference engineers and researchers

• Operate and manage the GPU fleet, ensuring reliability and consistency across cloud providers

• Develop scheduling solutions to efficiently allocate GPU resources under real constraints

• Support both long-running training jobs and real-time inference services on the same GPU fleet

• Manage Kubernetes orchestration for GPU resources across multiple clusters and providers

• Implement systems for fault tolerance, autoscaling, and observability to optimize fleet utilization

• Collaborate with teams to define the technical direction and platform architecture

Qualifications

• Deep Kubernetes expertise, including custom operators and multi-cluster management

• Experience managing GPU clusters at scale, specifically with NVIDIA hardware and high-speed networking

• Proficient in orchestrating compute resources across diverse cloud providers such as AWS and GCP

• Strong understanding of distributed systems, including scheduling and fault tolerance

• Skilled in systems-level programming in languages such as Go, Rust, or C++

• Familiarity with both long-running training jobs and high-availability inference services

• Proactive problem solver who takes ownership of end-to-end issues

Benefits

• Opportunities for professional growth in cutting-edge ML and GPU technologies

• Collaborative and innovative work environment encouraging creative problem-solving

• Access to advanced technologies and infrastructure for machine learning

• Flexible work arrangements with resources to support a balanced work-life

Our client is an equal opportunity employer. We encourage you to apply even if you don’t meet every qualification—your background could be exactly what this team needs.