Understanding Gated Recurrent Unit (GRU) Deep Neural Network

You've seen how a basic RNN works.In this video, you learn about the Gated Recurrent Unit which is a modification to the RNN hidden layer that makes it much better capturing long range connections and helps a lot with the vanishing gradient problems.Let's take a look.You've already seen the formula for computing the activations at time t of RNN.It's the activation function applied tothe parameter Wa times the activations in the previous time set,the current input and then plus ba. So I'm going to draw this as a picture.So the RNN unit, I'm going to draw as a picture,drawn as a box which inputs a of t-1, the activation for the last time-step.And also inputs xt and these two go together.And after some weights and after this type of linear calculation,if g is a tanh activation function,then after the tanh, it computes the output activation a.And the output activation a(t) might also be passed to say a softener unit or something that could then be used to output yt. So this is maybe a visualization of the RNN unit of the hidden layer of the RNN in terms of a picture.And I want to show you this picture because we're going to use a similar picture to explain the GRU or the Gated Recurrent Unit.Lots of the idea of GRU were due to these two papers respectively by Yu Young Chang, Kagawa, Gaza Hera, Chang Hung Chu and Jose Banjo.And I'm sometimes going to refer to this sentence which we'd seen in the last video to motivate that............

• #science_technology
• #gated_recurrent_unit
• #gru
• #neural_network
• #machine_learning
• #artificial_intelligence
• #deep_learning
• #recurrent_neural_network