Details

Autor(en) / Beteiligte

• Zhou, Guo-Bing
• Wu, Jianxin
• Zhang, Chen-Lin
• Zhou, Zhi-Hua

Titel

Minimal Gated Unit for Recurrent Neural Networks

Ist Teil von

• International journal of automation and computing, 2016-06, Vol.13 (3), p.226-234

Ort / Verlag

Beijing: Institute of Automation, Chinese Academy of Sciences

Erscheinungsjahr

2016

Quelle

Alma/SFX Local Collection

Beschreibungen/Notizen

• Recurrent neural networks （RNN） have been very successful in handling sequence data. However, understanding RNN and finding the best practices for RNN learning is a difficult task, partly because there are many competing and complex hidden units, such as the long short-term memory （LSTM） and the gated recurrent unit （GRU）. We propose a gated unit for RNN, named as minimal gated unit （MCU）, since it only contains one gate, which is a minimal design among all gated hidden units. The design of MCU benefits from evaluation results on LSTM and GRU in the literature. Experiments on various sequence data show that MCU has comparable accuracy with GRU, but has a simpler structure, fewer parameters, and faster training. Hence, MGU is suitable in RNN＇s applications. Its simple architecture also means that it is easier to evaluate and tune, and in principle it is easier to study MGU＇s properties theoretically and empirically.