Europe Arbeitnow ·
full-timeML Systems Engineer - Model Training And Infrastructure (swe-focused Llms)
Cosine · London Office
Job title: ML Systems Engineer - Model Training and Infrastructure (SWE-focused LLMs) Location: London; full in-office working as default Start date: ASAP Compensation: £80,000 - £110,000 Base Salary & £80,000 - £110,000 Share options. ___________________________________________________________________________ Help build the software engineers of the future Cosine is building autonomous AI engineers that plan, write and ship code inside real development workflows. Our agents work across complex software systems, and our Lumen models are trained to do more than produce code that looks correct. They are built to understand existing architectures, follow established patterns and produce software that engineers can actually maintain. We develop our agent tooling entirely in-house and post-train open-source models for reliable, enterprise-grade coding performance. Our products are designed for on-premise, VPC and fully air-gapped environments, including security-critical settings where control, privacy and robustness are non-negotiable. In 2024, Cosine achieved a 72% score on OpenAI’s SWE-Lancer benchmark, placing us among the strongest real-world software-engineering AI systems evaluated. We’re now looking for an ML Systems Engineer to help train the next generation of Lumen models. This is a highly hands-on role at the intersection of machine learning, software engineering, data and infrastructure. You’ll build the environments in which models learn to write software, develop the pipelines that generate and curate training data, and run the fine-tuning and reinforcement-learning workloads that shape model behaviour. If you’re excited by the idea that the future quality of coding agents will be determined not just by model architecture, but by the quality of their data, environments and reward functions, this is an opportunity to work directly on that problem. ___________________________________________________________________________ The role You’ll work closely with ML researchers, infrastructure engineers and product teams to decide: What Lumen should learn next. How to generate the right training data. How to design environments that reflect real software-engineering work. How to reward models for producing useful, maintainable code. How to measure whether a new training run genuinely improves the experience for engineers. The systems you build will sit directly inside our model-training loop. Models will write code, use tools, run tests and interact with real repositories. Your work will determine how those interactions are generated, evaluated and fed back into future training. This is not a narrow research role and it is not traditional MLOps. You’ll move between custom PyTorch code, distributed data pipelines, Dockerised services, RL environments, evaluation infrastructure and production-quality software. ___________________________________________________________________________ What you’ll do Build the training systems behind Lumen Contribute to the end-to-end training of software-engineering models. Implement supervised fine-tuning pipelines using curated code and conversation datasets. Build reinforcement-learning loops in which models write code, run tests and use development tools. Develop custom PyTorch dataloaders, training objectives and evaluation workflows. Run and analyse fine-tuning and RL experiments across large, modern open-source models. Create the data that teaches models to engineer Develop synthetic data-generation pipelines for future RL and fine-tuning runs. Design systems for producing, filtering, transforming and sampling large-scale datasets. Work with object storage, dataset sharding and data-quality checks. Investigate which examples, tasks and sampling strategies lead to better model behaviour. Turn model failures into concrete improvements to training data and future experiments. Build reliable RL infrastructure Design, build and deploy containerised services that support model training and evaluation. Use Docker and orchestration platforms such as Kubernetes to operate RL infrastructure. Build environments where agents can modify repositories, run tests, use tools and receive meaningful feedback. Improve the reliability, reproducibility and observability of large-scale training runs. Work across Python, Go and the surrounding infrastructure required to run these systems. Improve how we evaluate SWE models Help maintain and extend evaluation suites for code models. Build evaluations around unit tests, benchmark suites, repository-level tasks and real engineering workflows. Analyse model failure modes, reward-hacking risks and regressions. Develop better ways to measure code quality, maintainability, correctness and architectural fit. Feed evaluation results back into model, data and infrastructure decisions. Shape the next training direction Work with research, infrastructure and product teams to identify the highest-value training problems. Turn broad goals such as “mak
| Role | ML Systems Engineer - Model Training And Infrastructure (swe-focused Llms) |
|---|---|
| Company | Cosine |
| Location | London Office |
| Type | full-time |
| Compensation | Not disclosed |
| Deadline | Rolling |
Typical process for this type of role
A general guide — the exact steps for this specific listing may vary; check the original posting for details.
- 1ApplicationSubmit your resume through the apply link.
- 2ScreeningRecruiter reviews your background against the role.
- 3AssessmentA technical test, assignment, or coding round, depending on the role.
- 4Interview(s)One or more rounds with the hiring team.
- 5OfferOffer letter with compensation and start date.
Before you apply
0/4More at Cosine
Other jobs at Cosine
- Strategic Partnerships Manager · London Office
- Technical GTM Lead · London Office
Explore Related Placements
// similar opportunities
You might also like
Engineer, Accounting Data And Infrastructure Systems
Deliveroo
Our Global Structure Deliveroo is now part of DoorDash, bringing together teams with even greater reach, scale, and ambition. Depending on your role, you may co...
Machine Learning Engineer
Scarlet
We pull medical technology from the future to solve human health. Scarlet is authorised to assess and certify medical devices. We combine clinical, technical an...
Manager, Software Engineering (app Infrastructure)
Affirm
At Affirm, we exist for the moments that matter—giving people a clear, predictable way to pay over time, with no hidden fees, no surprises, and no tradeoffs on ...
Engineering Team Lead - 12 Month FTC
Flagstone
What is Flagstone? Flagstone is many things. An online savings platform, reinventing how individuals, businesses, and charities manage, protect, and grow their ...
Engineering Manager, Dedicated Infrastructure
Gitlab
GitLab is the intelligent orchestration platform for DevSecOps. GitLab enables organizations to increase developer productivity, improve operational efficiency,...
Copy Of Research Scientist / Engineer – Training Infrastructure
Lumaai
You'll build the distributed systems that train Luma's large-scale multimodal models across thousands of GPUs, so researchers can focus on innovation on top of ...