Member of Technical Staff, Training Performance Engineer

Who are we?

Role Overview

As a Performance Engineer in the Pre-Training team you will be responsible for optimizing the performance of our advanced language models and systems. Their primary focus is on improving key model training metrics, such as training throughput, ensuring high accelerator utilization.

The team combines expertise in software engineering, machine learning, and low-level kernel design and development to design robust systems and enhance model performance. You will work on identifying and removing performance bottlenecks, develop cutting-edge training and profiling tools to help Cohere's mission of providing efficient and reliable language understanding and generation capabilities and drive innovation in the field of natural language processing.

Note: We have offices in London, Toronto, New York and San Francisco, but we’re also remote-friendly! This team operates primarily between ET to CET time zones, so we’re seeking candidates in locations that align with these hours for effective collaboration.

Key

Responsibilities

• Design and write high-performant and scalable software for training.
• Understand architectural modifications and design choices and their effects on training throughput and quality.
• Write low-level CUDA, triton kernels to squeeze every last bit of performance from our accelerators.
• Research, implement, and experiment with ideas on our supercompute and data infrastructure.
• Learn from and work with the best researchers in the field.

Qualifications

• Extremely strong software engineering skills.
• Proficiency in Python and related ML frameworks such as JAX, Pytorch and XLA/MLIR.
• Experience writing kernels for GPUs using CUDA, triton, etc
• Experience using large-scale distributed training strategies.
• Familiarity with autoregressive sequence models, such as Transformers.

Bonus : paper at top-tier venues (such as NeurIPS, ICML, ICLR, AIStats, MLSys, JMLR, AAAI, Nature, COLING, ACL, EMNLP).

Working location

This role can be based remotely or from one of our office locations listed on the job description - there is no minimum in-office qualification requirement. We care most about hiring exceptional people regardless of locations, though please check the location listed on the posting for guidance around the core time zone or working hours alignment expected for the role.