Details

Autor(en) / Beteiligte

• Moreno-Torres, J. G.
• Saez, J. A.
• Herrera, F.

Titel

Study on the Impact of Partition-Induced Dataset Shift on k-Fold Cross-Validation

Ist Teil von

• IEEE transaction on neural networks and learning systems, 2012-08, Vol.23 (8), p.1304-1312

Ort / Verlag

New York, NY: IEEE

Erscheinungsjahr

2012

Link zum Volltext

• IEEE_Xplore

Quelle

IEEE Xplore Digital Library

Beschreibungen/Notizen

• Cross-validation is a very commonly employed technique used to evaluate classifier performance. However, it can potentially introduce dataset shift, a harmful factor that is often not taken into account and can result in inaccurate performance estimation. This paper analyzes the prevalence and impact of partition-induced covariate shift on different k -fold cross-validation schemes. From the experimental results obtained, we conclude that the degree of partition-induced covariate shift depends on the cross-validation scheme considered. In this way, worse schemes may harm the correctness of a single-classifier performance estimation and also increase the needed number of repetitions of cross-validation to reach a stable performance estimation.