A long time ago I remember I first tried an LSTM implementation in Torch, trying to generate text based on some large text corpus from shakespear (see reference below). The result that could be achieved at that time were already quite surprising. But now I want to give this concept another try. Except that this time, it should be implemented in Tensorflow instead of torch. SO let's get started!
<sxh python>
from tensorflow.python.keras import backend as K
def loss(labels, logits):
return K.sparse_categorical_crossentropy(labels, logits, from_logits=True) </sxh>
QUEENE: A fieter carver in the place, This child was parted body to the state, Here at the prince that is all the world, So two thy brother and what thou dost consent That is straight deliver them. ROMEO: O, if I had straight shall you be meterch'd; One cords apong no other reasons. MISTRESS OVERDONE: What's thy name? CORIOLANUS: Come, come, we'll play a better-house But that the sea prove a stolemnman: Why, I, pleased in the park, And bring all the rest from the king in lived to slander with the flood, And fellow'st thou, which lies your king and quick and will not but bid me fain, of what containt do amend him roar Than honourable and familiar curses; And soon perish. CORIOLANUS: You bless my daughter Katharina, A mother said that you may call him Aid the Lord Auberchere how 'tis boot doth light again. This is the heat's face. Second Senator: Then we may cry 'Charge it is that one would seem to be; The fault of heaven shall not be distraught, With such a miserable stay.
⇒ Not too bad for a training phase of less than 1 hour! (60 epochs, 1024 GRU units in a single hidden layer)