Neural Networks

You will need the following zip file for this tutorial. Please download it now by clicking here.

Objective

The aim of this session is to teach you about neural networks - simulated "brains" that learn what output is suitable by studying training data.

Installation & Setup

As always, we're using Python3. We will be using Python for the foreseeable future, so please ensure you have it installed.

In this session we will be using two Python libraries: TensorFlow and textgenrnn. Both can be installed from the command prompt or terminal using the following commands:

pip install tensorflow
pip install textgenrnn

As always, if you run into issues when installing these libraries, please ask a committee member for help.

You will also need to download some test data. This comes in the form of compressed text files which can be downloaded from our Google Drive. Please make sure to decompress the folder to an easy-to-reach location on your computer.

This practical is based on the example code from the textgenrnn Github repository.

Let's Get Coding!

Last week we learned about decision trees - a structure that queries some input data with basic questions and decisions to produce a suitable output.

This week, we're learning about neural networks, which are a step up from a decision tree. A neural network takes some input data and passes it through a series of "neurons" to produce some kind of output.

Each neuron is actually a mathematical function that alters the input data. The result is then passed to the next neurons.

To demonstrate, let's have a play with the textgenrnn library. textgenrnn is a simple-to-use library built using TensorFlow and Keras, two very well-known and versatile machine learning libraries.

First you'll need to create a new Python file and import the textgenrnn class from the textgenrnn module.


# Import textgenrnn from the module
from textgenrnn import textgenrnn
            

This will allow you to access the library's resources.

Next, you need to create a new instance of the textgenrnn class. This is done very simply, as shown below. Remember to store it inside a variable, or it will not work.


# Create an instance of the textgenrnn object
txtgn = textgenrnn()
            

Doing this creates a default neural network that can make very basic, but very random, decisions. Add the following code and then run the file to see what I mean.


# Generate some text
txtgn.generate()
            

Well done! You have created your first neural net. But it outputs things like this:


Bought my damage community is three students.
What is your finance was selling the background and contest it will stop seeing it about the other way this?
Don't stop twittering a person to come on the start of the same shotgun and have no idea what to do anything!
            

There are words in an order and it even knows to end a sentence with punctuation! But it doesn't make any sense. What it needs is some training on how to write properly. So let's give it some examples!

For this you will need to have downloaded the datasets zip file from our Google Drive. Make sure to decompress it and put the folder in a place that you can get to easily.

Add the following code to your file.


# Train the neural network on some data
txtgn.train_from_file('path_to_your_dataset_folder/datasets/hacker_news_2000.txt', num_epochs=1)

# Generate some text based on the trained network
txtgn.generate()
            

For this to work you will need to get the filepath for your datasets folder. This is done differently between Mac, Windows, and Linux devices, so if you need any help, please ask a committee member. Where it says 'path_to_your_dataset_folder', put the file path for this folder. Otherwise, Python will not be able to access the training data.

When you run this code, it will take longer than usual. This is because the network is being trained. Each neuron has a weight that can be adjusted. By adjusting the weight of each neuron and the connections they have with each other, you can fine-tune the network to become better at what it does.

That's the important thing to remember: Neural networks are very good at doing one thing.

Going Further

So, how could this be used?

Well, it is already being used by a lot of people to do some cool things. Both CollegeHumor and RoosterTeeth have made sketches written by neural networks.

Max Woolf, creator of textgenrnn has done some projects involving neural networks. Check out the Articles/Projects section of the textgenrnn repository.