Details

Autor(en) / Beteiligte

• Claeys, Emmanuelle
• Gancarski, Pierre
• Maumy-Bertrand, Myriam
• Wassner, Hubert

Titel

Dynamic Allocation Optimization in A/B-Tests Using Classification-Based Preprocessing

Ist Teil von

• IEEE transactions on knowledge and data engineering, 2023-01, Vol.35 (1), p.335-349

Ort / Verlag

IEEE

Erscheinungsjahr

2023

Quelle

IEEE Xplore

Beschreibungen/Notizen

• An A/B-Test evaluates the impact of a new technology by running it in a real production environment and testing its performance on a set of items. Recent development efforts around A/B-Tests revolve around dynamic allocation. They allow for quicker determination of the best variation (A or B), thus saving money for the user. However, dynamic allocation by traditional methods requires certain assumptions, which are not always valid in reality. This is often due to the fact that the populations being tested are not homogeneous. This article reports on a new reinforcement learning methodology which has been deployed by the commercial A/B-Test platform AB Tasty. We provide a new method that not only builds homogeneous groups of users, but also allows the best variation for these groups to be found in a short period of time. This article provides numerical results on AB Tasty's data, in addition to public datasets, tha demonstrate an improvement over traditional methods.