Sie befinden Sich nicht im Netzwerk der Universität Paderborn. Der Zugriff auf elektronische Ressourcen ist gegebenenfalls nur via VPN oder Shibboleth (DFN-AAI) möglich. mehr Informationen...
Dynamic Allocation Optimization in A/B-Tests Using Classification-Based Preprocessing
Ist Teil von
IEEE transactions on knowledge and data engineering, 2023-01, Vol.35 (1), p.335-349
Ort / Verlag
IEEE
Erscheinungsjahr
2023
Quelle
IEEE Xplore
Beschreibungen/Notizen
An A/B-Test evaluates the impact of a new technology by running it in a real production environment and testing its performance on a set of items. Recent development efforts around A/B-Tests revolve around dynamic allocation. They allow for quicker determination of the best variation (A or B), thus saving money for the user. However, dynamic allocation by traditional methods requires certain assumptions, which are not always valid in reality. This is often due to the fact that the populations being tested are not homogeneous. This article reports on a new reinforcement learning methodology which has been deployed by the commercial A/B-Test platform AB Tasty. We provide a new method that not only builds homogeneous groups of users, but also allows the best variation for these groups to be found in a short period of time. This article provides numerical results on AB Tasty's data, in addition to public datasets, tha demonstrate an improvement over traditional methods.