Research Intern AI Evaluations

About Karya:

Why was Karya on thecover of the Time Magazinehighlighted by Satya Nadella andchosen by Google as its partner for Project Vaani

In part because Karya is on a mission to provide AI enabled earning and learning opportunities to economically underserved communities thereby building a pathway out of poverty for them. Karya achieves this while also delivering high quality timely and price competitive data to its clients.

Karyas workers make at least 20 times the Indian minimum wage and through our one-of-a-kind digital work platform we have delivered over 40 million digital tasks and are poised to positively impact over 100 thousand workers by the end of the the coming years our goal is to rapidly scale our impact by bringing economic opportunities to millions of underserved users in India.

The Role

At Karya were building AI evaluation systems that measure and understand whether these systems actually work for the communities theyre meant to serve. Were looking for a research intern who can bring a social science lens into this work specifically around evaluating language technologies (speech text and translation systems) built for and with Indian communities.

Youll be embedded with the Evaluations team within Karyas tech team which means youll get a front-row seat to the full arc of AI work happening in India today from how models are built to how theyre tested to how they land in the hands of rural and economically underprivileged communities. Youll be expected to stay current on AI evaluation science itself and use your disciplinary training to ask sharper questions about what these systems are actually doing and to whom.

Concretely this means:

• Applying social science research methods (qualitative and/or quantitative) to the evaluation of AI/ML systems particularly language technologies
• Contribute to developing AI evaluation and testing methodologies
• Investigating how these technologies interact with and affect the communities theyre deployed among especially in rural and low-resource contexts in India
• Working closely with the Evaluations team to understand both the build and evaluate sides of AI systems

This role is well suited to someone whos curious about tech and wants to put their social science training into practice. Youll be working alongside engineers bringing a rigorous footing and a social science researchers lens into a technical space. If youre a social science student or researcher whos been eyeing the AI/tech space and wondering how to get in this is that entry point. Youll work on live evaluation pipelines with communities across India on low- and high-resource language technologies alongside a team that treats social science expertise as core to the work.

Qualifications

Education

• Currently pursuing or recently completed an undergraduate or masters degree in a social science discipline (Sociology Anthropology Development Studies Gender Studies Political Science Economics or Science and Technology Studies (STS)
• No technical degree required but demonstrated interest in and conceptual understanding of AI/ML systems is expected. E.g. through self-directed learning coursework or prior projects at the intersection of technology and society

Skills and Competencies

• Strong grounding in social science research methods qualitative quantitative or mixed (not a hard requirement but research training expected)
• Comfort conducting or supporting focus group discussions (FGDs) and field visits
• Conceptual understanding of AI/ML systems - applied AI/ML literacy - enough to engage meaningfully with concepts like model evaluation human-in-the-loop systems and language technologies (you dont need to code but you should be comfortable with how these systems are built and tested)
• Familiarity with socio-technical systems thinking (i.e. understanding technology and society as mutually shaping not separate concerns)
• Multilingual proficiency in Indian languages is a strong plus
• Background in computational social science is a plus not a requirement
• Strong written and verbal communication skills
• Comfort working in ambiguous early-stage research settings

People matter at Karya and these are some of the perks and benefits we created for our team:
Flexible vacation and leave policy
Flexible work options
Insurance as per industry standards
Access to industry stalwarts and networking opportunities

Karya is an equal opportunity and affirmative action employer.

Qualified applicants will receive consideration without regard to their race colour religion sex sexual orientation gender identity and disability.

Karya invites all qualified interested applicants to apply for career opportunities. If you are a person with a disability and need a reasonable accommodation to use our search to apply for a career opportunity reach out at.