Details

Autor(en) / Beteiligte

• Ruelens, F.
• Claessens, B. J.
• Quaiyum, S.
• De Schutter, B.
• Babuska, R.
• Belmans, R.

Titel

Reinforcement Learning Applied to an Electric Water Heater: From Theory to Practice

Ist Teil von

• IEEE transactions on smart grid, 2018-07, Vol.9 (4), p.3792-3800

Ort / Verlag

IEEE

Erscheinungsjahr

2018

Link zum Volltext

• Volltext

Quelle

IEEE Xplore

Beschreibungen/Notizen

• Electric water heaters have the ability to store energy in their water buffer without impacting the comfort of the end user. This feature makes them a prime candidate for residential demand response. However, the stochastic and nonlinear dynamics of electric water heaters, makes it challenging to harness their flexibility. Driven by this challenge, this paper formulates the underlying sequential decision-making problem as a Markov decision process and uses techniques from reinforcement learning. Specifically, we apply an auto-encoder network to find a compact feature representation of the sensor measurements, which helps to mitigate the curse of dimensionality. A well-known batch reinforcement learning technique, fitted <inline-formula> <tex-math notation="LaTeX">{Q} </tex-math></inline-formula>-iteration, is used to find a control policy, given this feature representation. In a simulation-based experiment using an electric water heater with 50 temperature sensors, the proposed method was able to achieve good policies much faster than when using the full state information. In a laboratory experiment, we apply fitted <inline-formula> <tex-math notation="LaTeX">{Q} </tex-math></inline-formula>-iteration to an electric water heater with eight temperature sensors. Further reducing the state vector did not improve the results of fitted <inline-formula> <tex-math notation="LaTeX">{Q} </tex-math></inline-formula>-iteration. The results of the laboratory experiment, spanning 40 days, indicate that compared to a thermostat controller, the presented approach was able to reduce the total cost of energy consumption of the electric water heater by 15%.