LLM - Applied AI Research Scientist (USA & LATAM Remote)
About the position Based in San Francisco, California, our client is the world’s leading research accelerator for frontier AI labs and a trusted partner for global enterprises deploying advanced AI systems. they supports customers in two ways: first, by accelerating frontier research with high-quality data, advanced training pipelines, plus top AI researchers who specialize in coding, reasoning, STEM, multilinguality, multimodality, and agents; and second, by applying that expertise to help enterprises transform AI from proof of concept into proprietary intelligence with systems that perform reliably, deliver measurable impact, and drive lasting results on the P&L We are seeking highly skilled Applied AI Research Scientists with deep expertise in Computer Engineering and hardware-centric systems with an MS or Ph.D. in a relevant technical field to design and execute expert-level evaluation tasks that probe the limits of state-of-the-art AI systems. In this role, you will create headroom-level, rigorously verifiable evaluation questions rooted in hardware, architecture, and low-level systems reasoning. Your work will focus on exposing model limitations in areas that require deep technical correctness, precise reasoning, and graduate-level understanding of computing systems—well beyond surface-level explanations. You will work closely with a collaborative, cross-functional team and are expected to be highly detail-oriented, reliable, and committed to accuracy and quality. Responsibilities • Design graduate- and research-level evaluation questions grounded in hardware and computer engineering domains. • Create tasks that require precise, step-by-step technical reasoning with objectively verifiable ground-truth answers. • Develop multimodal prompts, including accurate block diagrams, timing diagrams, microarchitecture diagrams, or circuit-level visuals when appropriate. • Evaluate state-of-the-art AI models on hardware- and systems-heavy reasoning tasks and perform structured side-by-side comparisons. • Identify and document model failure modes related to architectural correctness, performance reasoning, or low-level system behavior. • Provide authoritative solutions and explanations for each evaluation task. • Maintain detailed and accurate records of prompts, expected answers, and evaluation outcomes in shared tracking systems. • Collaborate with reviewers and researchers to refine evaluation quali Requirements • MS or Ph.D. in Computer Engineering, Electrical Engineering, Computer Science, or a closely related field. • Strong expertise in at least two of the below hardware- and systems-focused domains: Computer architecture (pipelines, memory hierarchies, cache coherence, ISA-level reasoning) Hardware systems and performance analysis VLSI design, digital logic, or ASIC/FPGA fundamentals Embedded systems and low-level firmware Operating systems (especially memory management, scheduling, and hardware–software interfaces) Compilers or systems programming with hardware awareness • Proven experience in technical research, evaluation, or rigorous problem formulation in academic, lab, or production-oriented environments. • Strong programming skills (especially Python, C or C++) for analysis, verification, and evaluation workflows. • Excellent written communication skills and a strong attention to technical detail. Apply tot his job