Loading...
Senior Applied Scientist, Amazon AWS Agentic AI, AWS AI Fundamental Research
Amazon
On-siteseniorPosted June 4, 2026
About the Role
Lead the design and development of agentic evaluation frameworks, define evaluation methodologies, create benchmarks, conduct original research, and own the end-to-end lifecycle from research to production deployment.
Requirements
Lead the design and development of agentic evaluation frameworks and evaluation/critic model training for generative AI, advancing the state-of-the-art in foundational models and Agentic AI.
Full Job Description
Amazon is looking for a passionate, talented, and inventive Applied Scientist with a strong machine learning background to help build industry-leading technology in generative AI and foundational models.
As part of our AI team in Amazon AWS, you will work alongside internationally recognized experts to develop novel algorithms and modeling techniques to advance the state-of-the-art in generative AI. Your work will directly impact millions of our customers in the form of products and services that make use of speech, vision and language technology. You will gain hands on experience with Amazon’s heterogeneous speech, text, image and structured data sources, and large-scale computing resources to accelerate advances in machine learning and foundation models. More specifically, you will have the opportunity to impact millions of our customers by researching and building innovative solutions using Agentic AI.
Agentic AI drives innovation at the forefront of artificial intelligence, enabling customers to transform their businesses through generative AI solutions. We build and deliver the foundational AI services that power the future of cloud computing, helping organizations harness the potential of AI to solve their most complex challenges. Join our dynamic team of AI/ML practitioners and applied scientists who work backwards from customer needs to create novel technologies. If you're passionate about shaping the future of AI while making a meaningful impact for customers worldwide, we want to hear from you.
Key job responsibilities
The Senior Applied Scientist will lead the design and development of agentic evaluation frameworks and evaluation/critic model training that assess the quality and effectiveness of AI agents at scale. They will define evaluation methodologies, create benchmarks, and build evaluation models and automated systems that measure agent performance across critical dimensions. The scientist will stay at the forefront of the rapidly evolving field by studying and adopting state-of-the-art methods, conducting original research to advance the science of agent and evaluation. They will own the end-to-end lifecycle from research and data curation through model training to production deployment, working closely with engineering to deliver evaluation capabilities as managed AWS services. They will collaborate with cross-functional stakeholders to translate science insights into actionable improvements, mentor junior scientists, and contribute to the broader research community.
A day in the life
A day in the life
Diverse Experiences
AWS values diverse experiences. Even if you do not meet all of the qualifications and skills listed in the job description, we encourage candidates to apply. If your career is just starting, hasn’t followed a traditional path, or includes alternative experiences, don’t let it stop you from applying.
Why AWS?
Amazon Web Services (AWS) is the world’s most comprehensive and broadly adopted cloud platform. We pioneered cloud computing and never stopped innovating — that’s why customers from the most successful startups to Global 500 companies trust our robust suite of products and services to power their businesses.
Inclusive Team Culture
Here at AWS, it’s in our nature to learn and be curious. Our employee-led affinity groups foster a culture of inclusion that empower us to be proud of our differences. Ongoing events and learning experiences, including our Conversations on Race and Ethnicity (CORE) and AmazeCon) conferences, inspire us to never stop embracing our uniqueness.
Mentorship & Career Growth
We’re continuously raising our performance bar as we strive to become Earth’s Best Employer. That’s why you’ll find endless knowledge-sharing, mentorship and other career-advancing resources here to help you develop into a better-rounded professional.
Work/Life Balance
We value work-life harmony. Achieving success at work should never come at the expense of sacrifices at home, which is why we strive for flexibility as part of our working culture. When we feel supported in the workplace and at home, there’s nothing we can’t achieve in the cloud.
As part of our AI team in Amazon AWS, you will work alongside internationally recognized experts to develop novel algorithms and modeling techniques to advance the state-of-the-art in generative AI. Your work will directly impact millions of our customers in the form of products and services that make use of speech, vision and language technology. You will gain hands on experience with Amazon’s heterogeneous speech, text, image and structured data sources, and large-scale computing resources to accelerate advances in machine learning and foundation models. More specifically, you will have the opportunity to impact millions of our customers by researching and building innovative solutions using Agentic AI.
Agentic AI drives innovation at the forefront of artificial intelligence, enabling customers to transform their businesses through generative AI solutions. We build and deliver the foundational AI services that power the future of cloud computing, helping organizations harness the potential of AI to solve their most complex challenges. Join our dynamic team of AI/ML practitioners and applied scientists who work backwards from customer needs to create novel technologies. If you're passionate about shaping the future of AI while making a meaningful impact for customers worldwide, we want to hear from you.
Key job responsibilities
The Senior Applied Scientist will lead the design and development of agentic evaluation frameworks and evaluation/critic model training that assess the quality and effectiveness of AI agents at scale. They will define evaluation methodologies, create benchmarks, and build evaluation models and automated systems that measure agent performance across critical dimensions. The scientist will stay at the forefront of the rapidly evolving field by studying and adopting state-of-the-art methods, conducting original research to advance the science of agent and evaluation. They will own the end-to-end lifecycle from research and data curation through model training to production deployment, working closely with engineering to deliver evaluation capabilities as managed AWS services. They will collaborate with cross-functional stakeholders to translate science insights into actionable improvements, mentor junior scientists, and contribute to the broader research community.
A day in the life
A day in the life
Diverse Experiences
AWS values diverse experiences. Even if you do not meet all of the qualifications and skills listed in the job description, we encourage candidates to apply. If your career is just starting, hasn’t followed a traditional path, or includes alternative experiences, don’t let it stop you from applying.
Why AWS?
Amazon Web Services (AWS) is the world’s most comprehensive and broadly adopted cloud platform. We pioneered cloud computing and never stopped innovating — that’s why customers from the most successful startups to Global 500 companies trust our robust suite of products and services to power their businesses.
Inclusive Team Culture
Here at AWS, it’s in our nature to learn and be curious. Our employee-led affinity groups foster a culture of inclusion that empower us to be proud of our differences. Ongoing events and learning experiences, including our Conversations on Race and Ethnicity (CORE) and AmazeCon) conferences, inspire us to never stop embracing our uniqueness.
Mentorship & Career Growth
We’re continuously raising our performance bar as we strive to become Earth’s Best Employer. That’s why you’ll find endless knowledge-sharing, mentorship and other career-advancing resources here to help you develop into a better-rounded professional.
Work/Life Balance
We value work-life harmony. Achieving success at work should never come at the expense of sacrifices at home, which is why we strive for flexibility as part of our working culture. When we feel supported in the workplace and at home, there’s nothing we can’t achieve in the cloud.