The results of the report underscore the value of a robust experimentation operation for any digital brand. The data shows initial ideas implemented by digital marketers are very rarely successful.
Optimizely, a digital experience platform (DXP) provider, unveiled a new research report, “The Evolution of Experimentation” that delivers an analysis of the over 127,000 true experiments conducted on Optimizely Web Experimentation and Optimizely Feature Experimentation between 2018 and 2023.
With this report, Optimizely hopes to advance the practice of experimentation around the world by sharing valuable insights and guidance for digital practitioners.
The report reveals that since 2018, Optimizely has observed significant growth in the number of experiments its customers conducted, an uplift in the volume of companies experimenting, and a 227% increase in the usage of feature experimentation as a percentage of all experiments. While data shows broader adaptation of experimentation in recent years, substantial differences remain in the quality of experimentation occurring across the landscape.
“This data confirms what many at Optimizely already know — the best in the game embrace experimentation to deliver more exceptional digital experiences to their users,” said Eric Lang, Senior Consultant of Strategy & Value. “There is no doubt that every test uncovers valuable information on customer behavior, no matter how small an experiment may seem. Those who experiment gain key insights into behavior in order to deliver exceptional digital experiences and develop long-lasting, personal relationships with their customers.”
The results of the report underscore the value of a robust experimentation operation for any digital brand. The data shows initial ideas implemented by digital marketers are very rarely successful. According to the report, 88% of the ideas implemented by digital teams are not winners, meaning they do not result in a positive significant change in user behavior. However, even if a test is not a winner, it provides valuable insights into customer behavior and allows teams to eliminate guesswork and pivot quickly.
“Experimentation allows digital businesses to achieve the highest level of customer-centricity by checking all of their ideas with the only audience that matters – their customers,” said Pete Gray, SVP of Customer Growth Strategy at Optimizely. “The emotional cost of experimentation is to learn how wrong we can be. The prize is the ability to never make a mistake again and to unlock real value by increasing the velocity of improvement.”
