HEY,
BOSTON

AI Evaluation Engineer$180,000 Remote (US-based)Are you passionate about shaping how AI is deployed safely, reliably, and at scale? This is a rare opportunity to join a mission-driven tech company as their first AI Evaluation Engineer, a foundational role where you’ll design, build, and own the evaluation systems that safeguard every AI-powered feature before it reaches the real world.This organization builds AI-enabled products that directly helps governments, nonprofits, and agencies deliver financial support to people who need it most. As AI capabilities race forward, ensuring these systems are safe, accurate, and resilient is critical. That’s where you come in.You won’t just be testing models, you’ll be creating the frameworks, pipelines, and guardrails that make advanced LLM features safe to ship. You’ll collaborate with engineers, PMs, and AI safety experts to stress test boundaries, uncover weaknesses, and design scalable evaluation systems that protect end users while enabling rapid innovation. What You’ll DoOwn the evaluation stack – design frameworks that define “good,” “risky,” and “catastrophic” outputs.Automate at scale – build data pipelines, LLM judges, and integrate with CI to block unsafe releases.Stress testing – red team AI systems with challenge prompts to expose brittleness, bias, or jailbreaks.Track and monitor – establish model/prompt versioning, build observability, and create incident response playbooks.Empower others – deliver tooling, APIs, and dashboards that put eval into every engineer’s workflow. Requirements:Strong software engineering background (TypeScript a plus)Deep experience with OpenAI API or similar LLM ecosystemsPractical knowledge of prompting, function calling, and eval techniques (e.g. LLM grading, moderation APIs)Familiarity with statistical analysis and validating data quality/performanceBonus: experience with observability, monitoring, or data science tooling

Senior LLM Research ScientistA frontier-stage research group is building a new class of AI systems designed to reason, plan, and act across the physical world. Their mission is to create intelligent agents capable of experimenting, engineering, and constructing in ways that dramatically accelerate scientific and industrial progress. This team combines deep technical pedigree with real-world wins at scale, including major government-funded initiatives. They operate where advanced model research meets robotics, simulation, and automated engineering systems, offering the kind of impact only possible when first-principles science meets ambitious execution. Joining means stepping into a high-ownership environment where you shape core capabilities end-to-end, influence the direction of physical-world intelligence, and help build technology the world has never seen before. Why This Role Is CompellingWork on cutting-edge reasoning, planning, and tool-use models that directly control autonomous engineering systems.Push the limits of SFT, RLHF, DPO, verifier-guided RL, and long-horizon planning in a setting where your research immediately translates into real-world capability.Operate in a high-velocity research culture with exceptional peers across agent systems, simulation, data, and complex toolchains.Have outsized ownership in a small team tackling one of the most ambitious technical problems of this decade.Role Overview The team is looking for an LLM Research Scientist to pioneer next-generation reasoning and agent architectures. Your work will span model design, alignment strategies, structured tool orchestration, and experimentation with agents interacting across real engineering workflows. This position blends deep research with hands-on systems integration, offering both autonomy and scope to lead foundational progress. Key ResponsibilitiesDevelop advanced models and prompting systems for planning, multi-step reasoning, and structured tool use.Lead training initiatives across SFT, RLHF/DPO, verifier-guided RL, and modular expert architectures to strengthen robustness and controllability.Define schemas, tool-calling strategies, policy constraints, safety mechanisms, and recovery pathways for agent behavior.Partner closely with engineering, simulation, and data teams to test, train, and evaluate models embedded in real production-like toolchains.QualificationsSignificant experience in LLM research, agent reasoning models, or structured tool-use frameworks.Strong background working with SFT, RLHF, DPO, or reinforcement-learning-from-verification methods.Demonstrated ability to design, analyze, and improve long-horizon behaviors and decomposition strategies.Comfortable working across ML research, systems engineering, and real-world experimentation in a fast-moving environment.A track record of excellence and ownership in technically demanding domains.

Senior RL Research Scientist / Reinforcement Learning ScientistJoin a frontier AI team building systems that can act in the physical world, experimenting, optimizing, and controlling real processes through advanced ML, simulation, and automation. This group is pushing the boundaries of physical intelligence, backed by significant long-term funding and a mandate to invent from first principles. If you want to:Work on problems few teams in the world can touchBuild RL systems that power real tools, workflows, and scientific processesOperate in a fast, high-ownership, deeply technical culture…this is the kind of role that defines a career. The Role You’ll design and deploy reinforcement learning systems that control complex tools, optimize multi-step processes, and operate across high-fidelity simulations and digital twins. Expect hands-on research, real-world experimentation, and tight collaboration with teams across ML, simulation, and systems engineering. What You’ll DoBuild RL environments for tool control, workflow optimization, and long-horizon decision-makingDevelop safe and constrained RL methods, verifier-driven rewards, and offline to online training pipelinesCreate state/action representations and evaluation frameworks for reliable policy behaviorWork with cross-functional researchers and engineers to deploy RL agents into real workflowsWhat You BringStrong background in RL, optimal control, or sequential decision-makingExperience applying RL to complex simulated or physical systemsFamiliarity with safe/constrained RL, verifiers, or advanced evaluation pipelinesAbility to design environments, rewards, and diagnostics at scaleComfort working across ML, simulation, and systems interfaces

LLM Evaluations Engineering LeadSF Bay Area (Onsite) Full-time / Permanent We’re partnering with a deep-tech AI company building autonomous, agentic systems for complex physical and real-world environments. The team operates at the edge of what’s possible today, designing AI systems that plan, act, recover, and improve over long horizons in high-stakes settings. They’re hiring an LLM Evaluations Engineering Lead to own the evaluation, verification, and regression layer for agentic LLM systems running end-to-end workflows. This is not a metrics-only role. You’ll be building the guardrails that determine whether the system is actually getting better.Why this role mattersAs agentic LLM systems move into long-horizon planning and execution, evals become the bottleneck. This role defines whether:Agents are actually improvingChanges introduce silent regressionsUncertainty is shrinking or compounding“success” reflects real-world outcomes, not proxy metricsIf evals are wrong, everything downstream is wrong. This role sits directly on that fault line.What you’ll doBuild eval harnesses for agentic LLM systems (offline in-workflow)Design evals for planning, execution, recovery, and safetyImplement verifier-driven scoring and regression gatesTurn eval failures into training signals (SFT / DPO / RL)What they’re looking forStrong experience building evaluation systems for ML models (LLMs strongly preferred)Excellent software engineering fundamentals:PythonData pipelinesTest harnessesDistributed executionReproducibilityDeep understanding of agentic failure modes, including:Tool misuseHallucinated evidenceReward hackingBrittle formatting and schema driftAbility to reason about what to measure, not just how to measure itComfortable operating between research experimentation and production systemsWhy joinWork on frontier agentic AI systems with real-world consequencesOwn a foundational layer that determines system reliability and progressHigh autonomy, strong technical peers, and meaningful equityBuild evals that actually matter, not academic benchmarks

Senior Agentic AI EngineerA frontier AI company is building systems that can act in the physical world, experimenting, engineering, and executing multi-step processes with real-world constraints. Backed by major research funding and operating at the edge of physical-AI innovation, they’re creating capabilities that don’t exist anywhere else. Join to work from first principles, own high-impact systems end-to-end, and help define how agentic AI will operate complex workflows in the real world.Why This Role MattersBuild agent systems that plan, execute, and recover across intricate engineering workflowsShape foundational behaviour patterns for next-gen LLM tool-useJoin early enough to influence architecture, culture, and performance standardsWork on problems that sit far beyond typical “LLM app” engineeringWhat You’ll DoDevelop planners, state machines, and tool-calling flows using frameworks like LangGraphCreate schemas, action definitions, and cross-tool interfaces for reliable, traceable executionBuild error-handling, timeouts, retries, rollbacks, and replay mechanismsPartner with ML, infra, and systems teams to integrate agents into real engineering toolchainsWhat You BringStrong experience with agent systems, structured tool calling, or orchestration frameworksDeep intuition for schemas, deterministic execution, and multi-step workflow designAbility to model failure modes, edge cases, and safe interactions in complex systemsComfort working across AI, systems engineering, and specialised domain tools in a high-precision environment

Senior Agentic AI EngineerRemote (US-based)Full-time / Permanent We’re working with an AI-native company operating at the intersection of healthcare, insurance, and regulated enterprise systems. They’re building production-grade Agentic AI and LLM platforms that automate complex, high-impact decision workflows. They’re hiring a hands-on AI / LLM Lead to own the design, deployment, and evolution of autonomous, agent-driven systems in production.Why this role matters This is a core technical ownership role. You’ll be building AI systems that:Interpret and reason over complex documentsOrchestrate multi-step workflows autonomouslyDrive real decisions in regulated environmentsWhat you’ll doLead end-to-end AI & LLM system design from architecture to productionBuild agent pipelines using LangChain, LangGraph, and adjacent toolingDeploy and optimize open models using vLLM, TGI, and Python inference stacksFine-tune, evaluate, and integrate open-source LLMs (e.g. Llama, Mistral, Qwen)Design robust prompt, planning, and tool-execution strategiesBuild and operate large-scale embeddings retrieval pipelinesApply unsupervised ML (clustering, similarity, anomaly detection) on tabular dataWhat they’re looking forHands-on experience with Agentic AI & autonomous systemsProven track record deploying LLMs in productionExperience with vLLM, TGI, or similar model-serving stacksStrong experience with open-source LLMs (Llama, Mistral, Qwen, etc.)Experience with LangChain, LangGraph, or equivalent frameworksHands-on OCR experience (scanned PDFs, complex layouts)Solid understanding of chunking, embeddings, vector search, and retrievalIdeal mindsetOwns systems end-to-endComfortable making high-stakes technical decisionsThrives in fast-moving, ambiguous environmentsThinks in systems, not scripts

Senior Machine Learning Infra Engineer | San Francisco | Competitive Salary EquityOur client is an early-stage AI company building foundation models for physics to enable end-to-end industrial automation, from simulation and design through optimization, validation, and production. They are assembling a small, elite, founder-led team focused on shipping real systems into production, backed by world-class investors and technical advisors.They are hiring a Machine Learning Cloud Infrastructure Engineer to own the full ML infrastructure stack behind physics-based foundation models. Working directly with the CEO and founding team, you will build, scale, and operate production-grade ML systems used by real customers. What you will doOwn distributed training and fine-tuning infrastructure across multi-GPU and multi-node clustersDesign and operate low-latency, highly reliable inference and model serving systemsBuild secure fine-tuning pipelines allowing customers to adapt models to their data and workflowsDeliver deployments across cloud and on-prem environments, including enterprise and air-gapped setupsDesign data pipelines for large-scale simulation and CFD datasetsImplement observability, monitoring, and debugging across training, serving, and data pipelinesWork directly with customers on deployment, integration, and scaling challengesMove quickly from prototype to production infrastructure What our client is looking for3 years building and scaling ML infrastructure for training, fine-tuning, serving, or deploymentStrong experience with AWS, GCP, or AzureHands-on expertise with Kubernetes, Docker, and infrastructure-as-codeExperience with distributed training frameworks such as PyTorch Distributed, DeepSpeed, or RayProven experience building production-grade inference systemsStrong Python skills and deep understanding of the end-to-end ML lifecycleHigh execution velocity, strong debugging instincts, and comfort operating in ambiguity Nice to haveBackground in physics, simulation, or computer-aided engineering softwareExperience deploying ML systems into enterprise or regulated environmentsFoundation model fine-tuning infrastructure experienceGPU performance optimization experience (CUDA, Triton, etc.)Large-scale ML data engineering and validation pipelinesExperience at high-growth AI startups or leading AI research labsCustomer-facing or forward-deployed engineering experienceOpen-source contributions to ML infrastructure This role suits someone who earns respect through hands-on technical contribution, thrives in intense, execution-driven environments, values deep focused work, and takes full ownership of outcomes. The company offers ownership of core infrastructure, direct collaboration with the CEO and founding team, work on high-impact AI and physics problems, competitive compensation with meaningful equity, an in-person-first culture five days a week, strong benefits, daily meals, stipends, and immigration support.

Machine Learning Engineer (LLM) $170,000 - $200,000 (DOE) Boston OR Berkeley, 2-3 days per week in-office We’re working a fast-growing AI company on a mission to automate complex workflows in the financial services sector, starting with insurance. Their technology leverages cutting-edge AI to simplify high-value processes, from multi-turn conversations to full workflow automation. As an ML Engineer within LLMs, you’ll be building and scaling advanced AI systems that power intelligent, multi-agent workflows. You’ll take ownership of designing, fine-tuning, and productionizing large language models, integrating them with backend systems, and optimizing their performance. You’ll collaborate closely with data science, DevOps, and leadership to shape the AI infrastructure that drives the company’s automation solutions. What You’ll Do:Build, fine-tune, and productionize large language model (LLM) pipelines, including PEFT, RLHF, and DPO workflows.Develop APIs, data pipelines, and orchestration systems for multi-agent, multi-turn AI conversations.Integrate models with backend services, including voice orchestration platforms and transcript generation.Optimize model usage and efficiency, transitioning from external APIs to in-house solutions.Collaborate cross-functionally with data scientists, DevOps, and leadership to deliver scalable machine learning solutions. What We’re Looking For:Essential Skills & Experience:Strong proficiency in Python and ML frameworks (e.g., scikit-learn, TensorFlow, PyTorch).Hands-on experience fine-tuning and training LLMs.PEFT, DPO, Prefence Optimization, post-training, supervised fine tuning, RLHFFamiliarity with AWS suite and deploying ML models to production.Ability to reason deeply about ML principles, architectures, and design choices.Knowledge of multi-agent orchestration and conversational AI systems.Desirable Skills & Experience:Background in voice AI, speech-to-text, or text-to-speech systems.Exposure to financial services or insurance applications.Familiarity with optimizing models for long-context scenarios. If you’d like to hear more, please apply or get in touch!