Machine Learning Validation & Data Operations Manager

Department BSD SUR - OHNS: Thirty Million Words - Tech About the Department The TMW Center develops, tests, and implements evidence-based interventions designed to promote very young children’s cognitive and social-emotional development, with a priority placed on that of children living in poverty. TMW Center interventions are designed to be overlaid onto existing health, education, and social service systems working at scale in a given community in order to meet families where they already are. The TMW Center has a robust research and development strategy that includes further development and testing of TMW interventions; harnessing technology to support behavior change, intervention engagement, and analysis; and furthering strategies to engage adult caregivers (parents, early educators and others) in the TMW Center’s interventions across the health, early learning, and social service sectors. The TMW Center is partnering with the Connecticut Office of Early Childhood (OEC) to conduct multi-year field research in Connecticut infant and toddler child care settings. The research will use a novel technology to support teachers and demonstrate how teachers’ language inputs in birth-to-3 child care settings lead to positive child outcomes. Early pilots will generate learnings and inform a classroom implementation and professional development model. A subsequent randomized controlled trial will test whether the new professional development model positively impacts teacher knowledge and behavior, leads to more language interactions between teachers and children, improves job satisfaction and drives positive child outcomes. Job Summary The TMW Center for Early Learning + Public Health (TMW Center) develops, tests, and implements evidence-based interventions designed to promote very young children’s cognitive and social-emotional development, with emerging technologies that enhance—rather than replace—the pivotal role that caregivers play in building healthy young brains. Although there is a rich body of research demonstrating the importance of the early language environment for maximizing early learning, there are very ways to assess the quality of those environments. As a result, when it comes to nurturing their children’s brains, parents often feel like they are in a maze without a map. The TMW Center has launched a wearable device and an accompanying app that uses machine learning to measure and analyze a child’s language environment and provide real-time information as well as personalized feedback and guidance for enhancing that environment. This groundbreaking piece of technology gives parents and caregivers with information they need to engage in robust brain-building interactions — and helps deepen caregiver-child connection. The TMW Center is looking for an ML Validation & Data Operations Manager to lead and monitor operations for continuous algorithm improvement and validation. This will involve guiding the development of data management systems and establishing data assessment strategies to enable efficient and scalable validation processes. The ML Validation & Data Operations Manager will also contribute to defining the vision for current and longer-term algorithm developments. They will collaborate closely with our data science, research, and data management teams to coordinate workstreams, ensure a smooth integration of the developed systems and protocols within broader operations, and align with research developments. This position will report to the CTO, with guidance from the Scientific Director. Responsibilities Lead data labeling effort to build ground-truth corpus for existing and future algorithms, including identifying data requirements and protocols. Determine validation criteria and metrics across models and settings/partnerships/use cases. Collaborate with data management teams, and application development teams to identify and capture the data necessary to perform validation. Recruit, train, and lead a team of data labelers. Ensure alignment between the validation roadmap and Center’s priorities. Establish timelines and strategies for the validation of different algorithms. Work with ML and engineering teams to develop and manage pipelines for continuous algorithm validation and optimization. Promote advances in, and creative ML solutions for validation and data management enhancement (e.g., automation of protocols and training pipelines). Build quality assurance processes to continuously assess reliability of data. Maintain comprehensive records of data sources, methodologies, and results. Establish best practice-based processes related to reproducibility, documentation, and version control. Investigates, analyzes and resolves day-to-day technical problems using standard procedures. Works with stakeholders to gather and analyze requirements for developmental programs. Receives a moderate level of guidance to design applications to meet University and business requirements. Performs code testing on components and works to ensure that appropriate implementation standards are met. Evaluates design alternatives for development cost and solutions using various methods. Supports and maintains existing applications. Works with web developers and responds to requests from users. Performs other related work as needed. Minimum Qualifications Education: Minimum requirements include a college or university degree in related field. Work Experience: Minimum requirements include knowledge and skills developed through 2-5 years of work experience in a related job discipline. Certifications: --- Preferred Qualifications Education: Advanced Degree in economics, data science, data analytics, computer science, software engineering or related fields strongly preferred. Experience: Knowledge and skills developed through 5 years of work experience in a related job discipline, strongly preferred. 2 years of experience managing people, strongly preferred. Preferred Competencies Prior experience setting up data labeling and validation processes. Demonstrated understanding of data science management, Machine Learning and Data operations. Experience building out ML operations teams and processes. Experience with statistical modeling and programming. Demonstrated ability to work independently with little supervision. Excellent strategic planning and execution skills. Strong problem-solving skills. Ability to balance short-term, long-term, and big picture objectives. Working Conditions Office Environment. Application Documents Resume (required) Cover Letter (preferred) When applying, the document(s) MUST be uploaded via the My Experience page, in the section titled Application Documents of the application. Job Family Information Technology Role Impact Individual Contributor Scheduled Weekly Hours 40 Drug Test Required No Health Screen Required No Motor Vehicle Record Inquiry Required No Pay Rate Type Salary FLSA Status Exempt Pay Range $80,000.00 - $120,000.00 The included pay rate or range represents the University’s good faith estimate of the possible compensation offer for this role at the time of posting. Benefits Eligible Yes The University of Chicago offers a wide range of benefits programs and resources for eligible employees, including health, retirement, and paid time off. Information about the benefit offerings can be found in the Benefits Guidebook. Posting Statement The University of Chicago is an equal opportunity employer and does not discriminate on the basis of race, color, religion, sex, sexual orientation, gender, gender identity, or expression, national or ethnic origin, shared ancestry, age, status as an individual with a disability, military or veteran status, genetic information, or other protected classes under the law. For additional information please see the University's Notice of Nondiscrimination. Job seekers in need of a reasonable accommodation to complete the application process should call 773-702-5800 or submit a request via Applicant Inquiry Form. All offers of employment are contingent upon a background check that includes a review of conviction history. A conviction does not automatically preclude University employment. Rather, the University considers conviction information on a case-by-case basis and assesses the nature of the offense, the circumstances surrounding it, the proximity in time of the conviction, and its relevance to the position. The University of Chicago's Annual Security & Fire Safety Report (Report) provides information about University offices and programs that provide safety support, crime and fire statistics, emergency response and communications plans, and other policies and information. The Report can be accessed online at: Paper copies of the Report are available, upon request, from the University of Chicago Police Department, 850 E. 61st Street, Chicago, IL 60637. The University of Chicago is an urban research university that has driven new ways of thinking since 1890. Our commitment to free and open inquiry draws inspired scholars to our global campuses, where ideas are born that challenge and change the world. We empower individuals to challenge conventional thinking in pursuit of original ideas. Students in the College develop critical, analytic, and writing skills in our rigorous, interdisciplinary core curriculum. Through graduate programs, students test their ideas with UChicago scholars, and become the next generation of leaders in academia, industry, nonprofits, and government. To learn more about the university click here Apply tot his job