Recurrent Neural Networks (RNN)

Recurrent Neural Networks are a type of neural network that uses a bidirectional model architecture, where outputs from nodes affect the incoming inputs to some extent (in contrast to the well known feedforward architecture that neural nets have traditionally used). There are many different styles of implementation, yet they often incorporate sequential or language related data to produce models that can remember and learn from the data and dynamically to decide what information matters during learning and what doesn’t.

Common Applications

Common Problem Types

  • NLP

  • Time Series

  • Music/Sound/Audio

  • Biological/Genetic

A Brief History

With roots in the 1920’s, Amari (1972) is generally credited with being the first to make RNN’s adaptive, that is, learn to change its outputs given its inputs by changing connection weights. Slow and steady developments in the 70’s and 80’s gave rise to one of the most popular RNN derivatives, the LSTM ("Long Short-Term Memory"). By 2016, LSTM models accounted for utilizing over a quarter of the total computational resources allotted for neural net inference at Google.

Learn more from our source about the history of RNN’s (and NN in general!)

Code Examples

All of the code examples are written in Python, unless otherwise noted.

Containers

These are code examples in the form of Jupyter notebooks running in a container that come with all the data, libraries, and code you’ll need to run it. Click here to learn why you should be using containers, along with how to do so.
Quickstart: Download Docker, then run the commands below in a terminal.

Time Series RNN

A great example from the Tensorflow authors building an RNN using time series data.

#pull container, only needs to be run once
docker pull ghcr.io/thedatamine/starter-guides:time-series-rnn

#run container
docker run -p 8888:8888 -it ghcr.io/thedatamine/starter-guides:time-series-rnn

LSTM (Long Short-Term Memory)

An implementation of an LSTM model trained on stock data to predict what the value will be in the near future.

#pull container, only needs to be run once
docker pull ghcr.io/thedatamine/starter-guides:lstm

#run container
docker run -p 8888:8888 -it ghcr.io/thedatamine/starter-guides:lstm

Need help implementing any of this code? Feel free to reach out to [email protected] and we can help!

Resources

All resources are chosen by Data Mine staff to be of decent quality, and most if not all content is free.

Websites

Videos

Books

Articles