Training neural networks can be a computationally and a time expensive task because they can depend on calculating the gradients of thousands or even millions of parameters. In recent years there have been advancements in multi-core processing with multi-core cpu (central processing units) which can help speed up the process.
But even though we can take advantage of the multi-core processing, most cpus only have either 4 or 8 cores. Fortunately we use utilize the thousands of cores available on graphic processing units to speed up our neural network training process.
For example, lets compare the GTX 980 TI graphic processing unit to the Intel Core i7-6700K 8M Skylake cpu . The GTX 980 TI has 704x time more physical cores then the cpu. Unfortunately, writing code to run on the gpu is not a trivial task. This is where libraries like Theano can help. Theano is a python library that translate python code into code which can be run on the gpu.
Since Theano is still in development, it is not user friendly and not easy to install. The goal of this post is to serve as a concise and updated tutorial on how to get Theano working on Windows 10 using the gpu.
For the remainder of this tutorial I am going to assume that most of your python libraries are installed through the python distribution known as MiniConda.
Link : https://www.continuum.io/downloads
Install Visual Studio
Install Miniconda (includes Python)
conda install pip six nose numpy scipy
Install CUDA Toolkit
conda install mingw libpython
pip install theano
Add a new system variable to environment variables with variable name THEANO_FLAGS and variable value
Confirm that C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\bin is in the PATH variable, if not add it.
Theano should now be able to run the code on GPU.
Run python and import theano. If you see the following message then it means that the installation was correctly done.
This is confirmed to work on Windows 10 version 14393.576 with Cuda 7.5.
If the top method does not work or results in an error. Try the following method.
Anaconda automatically installs two of the python libraries required to run Theano: scipy and numpy. Once you have Anaconda installed, install Theano using the following command in command prompt.
pip install theano
Now, download and install the 64 bit version of mingw.
Once Mingw-w64 is installed add
to your environment variable path.
Then create a temporary folder on you desktop and copy and paste python35.dll from C:\Anaconda3 folders. Now, use the command prompt to change directory to the folder with python35.dll and run the following command.
gendef python35.dll dlltool --as-flags=--64 -m i386:x86-64 -k --output-lib libpython35.a --input-def python35.def
After you run the code, there should be a file named python35.a in the folder. Copy and paste that file into C:\Anaconda3\libs.
Next, create a file named .theanorc. in C:\Users\<your username> by using the rename textbox in explorer, with the following content.
Don't forget the dot after .theanorc , so the file should be named .theanorc. and placed within your C:\Users\<your username> directory.
Then download and install Visual Studio Community 2013 using the following link.
Make sure you download the 2013 version and not the 2015 version, otherwise Theano will not work properly.
After Visual Studio is installed, install CUDA toolkit 7.5 from the following link.
Theano should now be configured to using the GPU. Open python and run import theano. If you see message such as,
Using gpu device 0: GeForce GTX 560 Ti (CNMeM is disabled, CuDNN not available),
then theano is configured properly and ready to use.