Mapping the Skills and Roles of Experimentation in Software Organizations: Evidence from 1,800 Job Postings

As organizations increasingly adopt online controlled experimentation to guide product development, the demand for specialized talent has grown rapidly. Yet there is limited empirical knowledge about which roles companies hire for and what skills are required. We analyze nearly 1,800 experimentation-related job postings (2022 to 2025) from experimentationjobs.com. Using a structured classification, we categorize roles into families including Conversion Rate Optimization (CRO), Data & Analytics, Product, Engineering, Marketing & Growth, and Enablement or Leadership. We then extract skill requirements through qualitative and keyword-based analysis. The study addresses two questions: (RQ1) What experimentation roles are organizations hiring for, and (RQ2) What skills and operational practices are most frequently required? Our findings reveal that experimentation work is cross-functional and that skill demands vary across disciplines. We also identify a gap between requested skills and current educational offerings, providing implications for hiring, training, and capability building.