Senior Software Engineer - Ai/ ml, Inference Model Enablement, Aws Neuron

About the Role

At Amazon, we develop AWS Neuron, the complete software stack for Trainium—Amazon's custom cloud-scale machine learning accelerators. As a Senior Software Development Engineer on the Inference Model Enablement team at Annapurna Labs, you will onboard and optimize state-of-the-art open-source and customer Large Language Models (dense and MoE architectures) to run efficiently on Trainium hardware. You will drive technical strategy, advance inference usability, optimize infrastructure, and lead architectural decisions across the Neuron serving stack.

Key Responsibilities

1. Model Optimization: Deliver high-performance models using distributed inference libraries on Trainium hardware.
2. Technical & System Leadership: Drive technical excellence in performance optimization and system reliability across the AWS Neuron ecosystem.
3. Architecture Strategy: Author design proposals and drive architectural decisions that impact the entire Neuron serving stack.
4. Cross-Functional Collaboration: Partner with applied scientists, system engineers, compiler/runtime teams, and product managers to define technical strategy.
5. Mentorship & Quality: Mentor engineering team members across work streams and establish high standards for code quality and software design.
6. Open Source Contribution: Drive enablement features delivered through open-source integrations such as the vLLM Neuron plugin.

Basic Qualifications

1. 5+ years of non-internship professional software development experience.
2. 5+ years of programming using modern languages (such as Java, C++, or C#) including object-oriented design.
3. 5+ years leading design or architecture (scaling, reliability, design patterns) of system infrastructure.
4. 5+ years participating in full software development lifecycles (code reviews, testing, CI/CD, operations).
5. Experience serving as a tech lead, mentor, or engineering team lead.

Preferred Qualifications

1. Master’s degree in Computer Science or equivalent field.
2. Deep understanding of Machine Learning and Large Language Model fundamentals (architecture, inference lifecycles, and model execution optimization).