AEO Measurement & Insights Lead

This role owns the measurement and insights infrastructure behind the AEO program. Youll build and maintain the prompt panel used to test how the brand shows up in AI-generated answers evaluate the tools that support it run quality assurance on the responses that come back and turn all of it into dashboards and experiment analysis the team can act on. The core question youre answering: when someone asks an AI platform about this category does the brand show up how is it described and what makes that change.

Youll own the measurement system end to end from designing what gets tracked to auditing the quality of what comes back to building the reporting layer that makes results legible to non-technical stakeholders. The job takes equal parts rigor and communication. Sound experiment design and consistent QA on one side clear and usable findings on the other.

This is a new category so the measurement approach doesnt fully exist yet. Youll be building it and some of it youll figure out as you go.

You will be responsible for:

• Building and maintaining a structured prompt panel to test and track brand visibility and citations across AI platforms.
• Evaluating and recommending tools for prompt panel management response capture and analysis.
• Running quality assurance on response data flagging inconsistencies tool errors or gaps that would compromise the accuracy of findings.
• Designing and building dashboards that make performance and trends clear and actionable for both agency and client stakeholders.
• Designing and running experiments to test hypotheses and isolate what actually drives changes in visibility.
• Analyzing experiment and panel results and translating findings into clear recommendations.
• Documenting measurement methodology and QA processes so the system is repeatable defensible and can transfer to the client team over time.
• Partnering with agency and client stakeholders to align on whats being measured and why and socializing findings as needed.

What Youll Be Measuring

• Visibility. Whether the brand appears in AI-generated answers across a defined prompt panel and how often.
• Share of voice. How the brands presence compares to competitors within the same answers.
• Accuracy and framing. How the brand and its products are described and whether that description is correct and favorable.
• Source attribution. Which pages domains and third-party sources the models pull from when the brand does appear.
• Platform coverage. How results differ across ChatGPT Gemini Perplexity Copilot and AI Overviews.
• Consistency. How stable responses are when the same prompt runs repeatedly and what that variance means for confidence in the data.
• Movement over time. Whether the work the team is doing actually changes any of the above.

The Insights Youll Deliver

• Where the brand is absent from answers it should be in and the likely reason why.
• Which content and structural changes moved citation rates and which didnt.
• Competitive gaps worth acting on and what competitors are doing to earn their citations.
• Prioritized recommendations for the content and optimization team grounded in what the data shows.
• A regular readout that both agency leadership and client stakeholders can follow without a data background.

Qualifications :

Must-have

• Hands-on experience measuring AI search or AEO/GEO performance: prompt testing citation and share-of-voice tracking or LLM response analysis.
• Experience building and maintaining measurement or reporting systems (dashboards tracking panels or similar) in an analytics or research context.
• Strong data QA instincts. Youre comfortable auditing outputs for accuracy and catching errors before they reach a report.
• Experience designing and analyzing experiments whether A/B testing or similar comparative analysis.
• Proficiency with dashboarding and BI tooling and comfort working with structured data.
• Clear written and verbal communication for translating technical findings into recommendations non-technical people can use.

Nice-to-have

• Background at an agency or platform delivering AEO or GEO as a service.
• Technical SEO experience particularly around structured data and content structuring for retrieval.

Additional Information :

We Are Rosie provides strategic consulting and talent solutions for the marketing industry. We partner with more than 200 of the worlds biggest brands and agencies to help them solve unique marketing challenges with the best people swiftly and at scale. Founded by former advertising executive Stephanie Nadi Olson in 2018 We Are Rosie is on a mission to redefine how marketing works with an inclusive human-centric approach thats better for business and better for marketers. Our company has been recognized as an industry trailblazer and earned many accolades including being named to the 2023 Global Top 100 Inspiring Workplaces the Inc.and Adweeks Fastest Growing Agencies in 2021.
 

We Are Rosie is an equal opportunity employer. We do not discriminate on the basis of race color religion age national origin ancestry physical or mental disability medical condition pregnancy genetic information gender sexual orientation gender identity or expression veteran status marital status or any other status protected under federal state or local law.

Remote Work :

Yes

Employment Type :

Contract