Principal AI Architect (LLMOps Architect)

It's fun to work in a company where people truly BELIEVE in what they are doing!

We're committed to bringing passion and customer focus to the business.

Principal AI Architect

New York, NY (Onsite)

Fractal is a strategic AI partner to Fortune 500 companies with a vision to power every human decision in the enterprise. Fractal is building a world where individual choices, freedom, and diversity are the greatest assets. An ecosystem where human imagination is at the heart of every decision. Where no possibility is written off, only challenged to get better. We believe that a true Fractalite is the one who empowers imagination with intelligence. Fractal has been featured as a Great Place to Work by The Economic Times in partnership with the Great Place to Work® Institute and recognized as a ‘Cool Vendor’ and a ‘Vendor to Watch’ by Gartner.
 

Please visit Fractal | Intelligence for Imagination for more information about Fractal

Location: New York, NY

Note: This position is not eligible for Immigration Sponsorship at this time.
 

Overview: 
As an LLMOps Architect, you will lead the design and implementation of scalable, secure, and efficient operational pipelines for deploying and managing large language models (LLMs) in production. You will collaborate with cross-functional teams to ensure that LLMs are integrated seamlessly into enterprise applications, with robust monitoring, governance, and performance optimization.

Responsibilities: 

• Architect and implement end-to-end LLMOps pipelines for training, fine-tuning, deployment, and monitoring of LLMs.

• Design and manage infrastructure for prompt orchestration, retrieval-augmented generation (RAG), and agentic workflows.

• Integrate LLMs with enterprise APIs, data sources, and UI frameworks (e.g., Streamlit, Chainlit).

• Establish CI/CD workflows using tools like GitHub Actions, Jenkins, and Docker.

• Ensure observability and traceability using Langtrace, Langfuse, and RAGAS.

• Collaborate with data scientists, ML engineers, and product teams to align LLM capabilities with business goals.

• Implement security, privacy, and compliance controls for LLM deployments.

• Lead technical evaluations and partnerships with platforms like Hugging Face, LangChain, and Azure OpenAI.

Qualifications: 

• Strong programming skills in Python (OOP, decorators, data structures).

• Experience with containerization (Docker) and orchestration (Kubernetes).

• Familiarity with LLM frameworks: LangChain, Langfuse, Langtrace, RAGAS.

• Knowledge of agentic AI frameworks: CrewAI, AutoGen, Taskweaver (preferred).

• Hands-on experience with cloud platforms: AWS, Azure, GCP.

• Understanding of model evaluation, prompt engineering, and guardrails.

• Experience with UI frameworks: Streamlit, Flask, FastAPI. 

Pay:

The wage range for this role takes into account the wide range of factors that are considered in making compensation decisions, including but not limited to skill sets; experience and training; licensure and certifications; and other business and organizational needs.  The disclosed range estimate has not been adjusted for the applicable geographic differential associated with the location at which the position may be filled.  At Fractal, it is not typical for an individual to be hired at or near the top of the range for their role and compensation decisions are dependent on the facts and circumstances of each case.  A reasonable estimate of the current range is: Up to $240,000. In addition, you may be eligible for a discretionary bonus for the current performance period.

Benefits:

As a full-time employee of the company or as an hourly employee working more than 30 hours per week, you will be eligible to participate in the health, dental, vision, life insurance, and disability plans in accordance with the plan documents, which may be amended from time to time.  You will be eligible for benefits on the first day of employment with the Company.  In addition, you are eligible to participate in the Company 401(k) Plan after 30 days of employment, in accordance with the applicable plan terms.   The Company provides for 11 paid holidays and 12 weeks of Parental Leave. We also follow a “free time” PTO policy, allowing you the flexibility to take the time needed for either sick time or vacation.

Fractal provides equal employment opportunities to all employees and applicants for employment and prohibits discrimination and harassment of any type without regard to race, color, religion, age, sex, national origin, disability status, genetics, protected veteran status, sexual orientation, gender identity or expression, or any other characteristic protected by federal, state or local laws.

If you like wild growth and working with happy, enthusiastic over-achievers, you'll enjoy your career with us!

Hiring Related Queries

India: HiringsupportIndia@fractal.ai

Outside India: HiringsupportROW@fractal.ai

Please share resume via job postings

Not the right fit?  Let us know you're interested in a future opportunity by clicking Introduce Yourself in the top-right corner of the page or create an account to set up email alerts as new job postings become available that meet your interest!