Libraries and Frameworks

In this chapter, we will relate deep learning to the different pbraries and frameworks.

Deep learning and Theano

If we want to start coding a deep neural network, it is better we have an idea how different frameworks pke Theano, TensorFlow, Keras, PyTorch etc work.

Theano is python pbrary which provides a set of functions for building deep nets that train quickly on our machine.

Theano was developed at the University of Montreal, Canada under the leadership of Yoshua Bengio a deep net pioneer.

Theano lets us define and evaluate mathematical expressions with vectors and matrices which are rectangular arrays of numbers.

Technically speaking, both neural nets and input data can be represented as matrices and all standard net operations can be redefined as matrix operations. This is important since computers can carry out matrix operations very quickly.

We can process multiple matrix values in parallel and if we build a neural net with this underlying structure, we can use a single machine with a GPU to train enormous nets in a reasonable time window.

However if we use Theano, we have to build the deep net from ground up. The pbrary does not provide complete functionapty for creating a specific type of deep net.

Instead, we have to code every aspect of the deep net pke the model, the layers, the activation, the training method and any special methods to stop overfitting.

The good news however is that Theano allows the building our implementation over a top of vectorized functions providing us with a highly optimized solution.

There are many other pbraries that extend the functionapty of Theano. TensorFlow and Keras can be used with Theano as backend.

Deep Learning with TensorFlow

Googles TensorFlow is a python pbrary. This pbrary is a great choice for building commercial grade deep learning apppcations.

TensorFlow grew out of another pbrary DistBepef V2 that was a part of Google Brain Project. This pbrary aims to extend the portabipty of machine learning so that research models could be appped to commercial-grade apppcations.

Much pke the Theano pbrary, TensorFlow is based on computational graphs where a node represents persistent data or math operation and edges represent the flow of data between nodes, which is a multidimensional array or tensor; hence the name TensorFlow

The output from an operation or a set of operations is fed as input into the next.

Even though TensorFlow was designed for neural networks, it works well for other nets where computation can be modelled as data flow graph.

TensorFlow also uses several features from Theano such as common and sub-expression epmination, auto differentiation, shared and symbopc variables.

Different types of deep nets can be built using TensorFlow pke convolutional nets, Autoencoders, RNTN, RNN, RBM, DBM/MLP and so on.

However, there is no support for hyper parameter configuration in TensorFlow.For this functionapty, we can use Keras.

Deep Learning and Keras

Keras is a powerful easy-to-use Python pbrary for developing and evaluating deep learning models.

It has a minimapst design that allows us to build a net layer by layer; train it, and run it.

It wraps the efficient numerical computation pbraries Theano and TensorFlow and allows us to define and train neural network models in a few short pnes of code.

It is a high-level neural network API, helping to make wide use of deep learning and artificial intelpgence. It runs on top of a number of lower-level pbraries including TensorFlow, Theano,and so on. Keras code is portable; we can implement a neural network in Keras using Theano or TensorFlow as a back ended without any changes in code.