Algorithms

BiDAF is a popular machine learning model for Question and Answering tasks. This article illustrates how BiDAF uses three embedding mechanisms to convert words into their vector representations. (By Meraldo Antonio) This article is the second … Read more

This article illustrates the workings of BiDAF, an NLP model that has pushed the envelope in the Question and Answer domain. (By Meraldo Antonio) The year 2016 saw the publication of BiDAF by a team … Read more

A compiled visualisation of the common convolutional neural networks.

Recurrent neural networks are a class of artificial neural networks which are often used with sequential data. The 3 most common types of recurrent neural networks are vanilla recurrent neural network (RNN), long short-term memory … Read more

Gradient descent is an optimisation method for finding the minimum of a function. It is commonly used in deep learning models to update the weights of the neural network through backpropagation.

In this post, I will summarise the common gradient descent optimisation algorithms used in popular deep learning frameworks (e.g. TensorFlow, Keras, PyTorch, Caffe). The purpose of this post is to make it easy to read and digest (using consistent nomenclature) since there aren’t many of such summaries out there, and as a cheat sheet if you want to implement them from scratch.