With any design, it is important to know how big a success (or failure!) a new feature is.

Expedia have a proven A/B testing framework and run hundreds of experiments across all elements of their technology stack. Expedia’s Experiment Engineering team provide an in-house toolkit to the rest of the organisation that enable them to implement, manage and run their tests. The results of those tests are delivered in near real-time, providing a way to quickly validate and course-correct the new features being tested.

Behind all of this is a system that deals with over 70,000 experimentation requests per second, including a big data pipeline which analyses terabytes of data.

Over beers and pizza, Caroline & Ralph will introduce the approach Expedia has chosen, demo their in-house tools, give an overview of the technologies in play and - perhaps most importantly - the hard lessons learned along the way.

MVB 1.06, midday, Nov 16