AI/ML Annotation Engineer - Red Teaming & Safety Evaluation Expert for Generative AI Applications

Join the Future of AI at Workwarp: Shaping the Safety and Inclusivity of Generative AI At Workwarp, we're on a mission to revolutionize the world through cutting-edge AI technologies. As part of our ambitious plans to push the boundaries of what's possible with generative AI, we're seeking a highly skilled AI/ML Annotation Engineer to lead our Red Teaming & Safety Evaluation efforts. This role is crucial in ensuring that our AI applications are not only innovative but also safe, inclusive, and responsible. About the Role As an AI/ML Annotation Engineer - Red Teaming & Safety Evaluation, you will play a pivotal role in assessing, quantifying, and improving the safety and inclusivity of our generative AI-powered features and products. You'll be working closely with our ML Researchers, ML Data Scientists, and ML Engineers to tackle complex problems in machine learning, with a particular focus on large language models, diffusion models, and mixed model systems for multimodal applications. Key Responsibilities Develop and implement red teaming and grading project strategies, interfaces, and guidelines in collaboration with data scientists and product engineering teams. Serve as a liaison between red teamers, data scientists, project managers, and product development engineers to ensure seamless communication and feedback. Prioritize and schedule projects to align with organizational goals and timelines, managing multiple projects simultaneously. Monitor and analyze red teamer performance metrics, providing constructive feedback and guidance for improvement. Work with data scientists and red teaming engineers to distill findings into actionable recommendations for product engineering teams and safety policy development. Research and develop innovative methodologies to enhance red teaming and grading efficiency and quality, leveraging conventional and ML-based approaches. Essential Qualifications To succeed in this role, you should have: Prior experience in linguistic and/or computer vision annotation tasks, such as named-entity recognition, part-of-speech tagging, and image scene classification. Proficiency in scripting languages (e.g., Python) and spreadsheet tools for data management and analysis. Project management expertise, with the ability to collaborate with team members, prioritize projects, and communicate effectively with engineers and annotators. Experience working with sensitive content and exposure to offensive and controversial material. Preferred Qualifications To stand out, you'll ideally have: A Master's or Ph.D. in Linguistics, Computer Vision, or a related field, with a strong foundation in scientific data collection. 2+ years of experience as an annotator, annotation project lead, or ontologist. Prior teaching or scientific research experience, with a track record of publications. A deep understanding of the English language and its applications to Natural Language Processing. Experience with crowd-based annotation and grading projects, as well as developing and implementing sophisticated ontology or taxonomy systems. Familiarity with generative models (LLMs, diffusion models, etc.) and their evaluation, product development, and productivity applications. A passion for fairness and bias in generative AI systems, with a desire to contribute to more equitable technology. What We Offer At Workwarp, we provide a competitive salary and a comprehensive benefits package. As a remote team member, you'll enjoy the flexibility to work from anywhere while being part of a dynamic and innovative team. Our company culture values diversity, inclusion, and creativity, offering a stimulating environment that fosters growth and learning. Career Growth and Learning Benefits This role offers a unique opportunity to work on the cutting edge of AI technology, with ample chances for professional growth and development. You'll be working alongside talented professionals who are experts in their fields, and you'll have access to training and resources to enhance your skills. As we continue to push the boundaries of what's possible with generative AI, you'll be at the forefront of shaping the future of this technology. Work Environment and Company Culture Workwarp is an equal opportunity employer committed to inclusion and diversity. We believe in creating a work environment that is welcoming and inclusive to everyone, regardless of their background, identity, or perspective. Our team is passionate about harnessing the power of AI to drive positive change, and we're looking for like-minded individuals to join us on this journey. Compensation and Benefits We offer a competitive compensation package that reflects your skills and experience. Our benefits include a comprehensive health insurance plan, flexible working hours, and opportunities for professional development. As a remote team member, you'll also enjoy the flexibility to work from anywhere and maintain a healthy work-life balance. Ready to Make a Difference? If you're passionate about AI, safety, and inclusivity, and you're looking for a challenging and rewarding role, we want to hear from you. Apply now to join our team as an AI/ML Annotation Engineer - Red Teaming & Safety Evaluation and be part of shaping the future of generative AI. Apply for this job