Common use of Embedding Clause in Contracts

Embedding. Due to the fact that some events are related to others or often proceed a specific event, an embedding layer is used to plot the events in a higher dimensional space, with more similar events ‘closer’ to one another. The size of the embedding is a hyperparameter (i.e a model parameter that needs to be tuned). The output of the embedding layer is fed into a dropout layer (10%). Dropout is implemented to ensure a certain level of self-regularization during the training of the model. It ignores a part of the neurons (in this case 10%) during the forward or backwards pass in the training phase. This regularizes the importance of the neurons in a network, resulting in less overfitting in general.