- 2016 -

Set Up OpenAI Gym and PLE

Install OpenAI Gym

Install some system packages

apt-get install -y python-numpy python-dev cmake zlib1g-dev libjpeg-dev xvfb libav-tools xorg-dev python-opengl libboost-all-dev libsdl2-dev swig

Install the OpenAI Gym

git clone https://github.com/openai/gym.git
cd gym
sudo pip install -e '.[all]'

ALE (Arcade Learning Environment has already been in the OpenAI Gym)

Install PLE (PyGame Learning Environment)

Install PyGame

apt-get install -y python-pygame

Install PLE

git clone https://github.com/ntasfi/PyGame-Learning-Environment.git
cd PyGame-Learning-Environment/
sudo pip install -e .

Install gym-ple

git clone https://github.com/lusob/gym-ple.git
cd gym-ple/
sudo pip install -e .

Also need to install doom-py.

sudo pip install doom_py
Configures

Set Up the Desktop PC for Deep Learning

About the Desktop PC

CPU: I7-6700
GPU: GTX 1080
Memory: 16GB

Install the Ubuntu 16.04

Omitted

Install the driver for GTX1080

sudo add-apt-repository ppa:graphics-drivers/ppa
sudo apt-get update
sudo apt-get install nvidia-370
sudo apt-get install mesa-common-dev
sudo apt-get install freeglut3-dev

Install CUDA 8

Download from the official site, Nvidia Developer account is needed. Choose Linux, x86_64, Ubuntu, 16.04, runfile (local) and download the installer. Notice that using deb will come across some troubles, just use the runfile.

Install the CUDA 8

sudo sh cuda_8.0.27_linux.run --override
Do you accept the previously read EULA?
accept/decline/quit: accept
Install NVIDIA Accelerated Graphics Driver for Linux-x86_64 361.77?
(y)es/(n)o/(q)uit: n
Install the CUDA 8.0 Toolkit?
(y)es/(n)o/(q)uit: y
Enter Toolkit Location
 [ default is /usr/local/cuda-8.0 ]: 
Do you want to install a symbolic link at /usr/local/cuda?
(y)es/(n)o/(q)uit: y
Install the CUDA 8.0 Samples?
(y)es/(n)o/(q)uit: y
Enter CUDA Samples Location
 [ default is /home/andi ]:

Then install the patch

sudo sh cuda_8.0.27.1_linux.run
Do you accept the previously read EULA?
accept/decline/quit: accept
Enter CUDA Toolkit installation directory
 [ default is /usr/local/cuda-8.0 ]:

Then edit the ~/.bashrc (add the code at the end of .bashrc)

export PATH=/usr/local/cuda-8.0/bin${PATH:+:${PATH}}
export LD_LIBRARY_PATH=/usr/local/cuda-8.0/lib64${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}

Install the CUDNN

Download cuDNN v5.1 Library for Linux from the official site

Unzip the CUDNN and copy it to the dir of CUDA

tar xvzf cudnn-8.0-linux-x64-v5.1.tgz
sudo cp cuda/include/cudnn.h /usr/local/cuda/include/
sudo cp cuda/lib64/libcudnn* /usr/local/cuda/lib64/
sudo chmod a+r /usr/local/cuda/include/cudnn.h /usr/local/cuda/lib64/libcudnn*

Install Bazel

Add Bazel distribution URI as a package source (one time setup)

$ echo "deb [arch=amd64] http://storage.googleapis.com/bazel-apt stable jdk1.8" | sudo tee /etc/apt/sources.list.d/bazel.list
$ curl https://bazel.io/bazel-release.pub.gpg | sudo apt-key add -

Update and install Bazel

$ sudo apt-get update && sudo apt-get install bazel
$ sudo apt-get upgrade bazel

Install the TensorFlow from sources

At the moment(when this post is done) we have to compile the source code of TensorFlow in order to support CUDA 8.

Install some dependencies

sudo apt-get install python-numpy swig python-dev python-pip
sudo apt-get install libcurl3-dev

Clone the TensorFlow repository

git clone https://github.com/tensorflow/tensorflow

Enter the TensorFlow’s dir and run ./configure

./configure
Please specify the location of python. [Default is /usr/bin/python]: 
Do you wish to build TensorFlow with Google Cloud Platform support? [y/N] y
Google Cloud Platform support will be enabled for TensorFlow
Do you wish to build TensorFlow with Hadoop File System support? [y/N] N
No Hadoop File System support will be enabled for TensorFlow
Found possible Python library paths:
  /usr/local/lib/python2.7/dist-packages
  /usr/lib/python2.7/dist-packages
Please input the desired Python library path to use.  Default is [/usr/local/lib/python2.7/dist-packages]
/usr/local/lib/python2.7/dist-packages
Do you wish to build TensorFlow with GPU support? [y/N] y
GPU support will be enabled for TensorFlow
Please specify which gcc should be used by nvcc as the host compiler. [Default is /usr/bin/gcc]: 
Please specify the Cuda SDK version you want to use, e.g. 7.0. [Leave empty to use system default]: 8.0
Please specify the location where CUDA 8.0 toolkit is installed. Refer to README.md for more details. [Default is /usr/local/cuda]: 
Please specify the Cudnn version you want to use. [Leave empty to use system default]: 5.1
Please specify the location where cuDNN 5.1 library is installed. Refer to README.md for more details. [Default is /usr/local/cuda]: 
Invalid path to cuDNN  toolkit. Neither of the following two files can be found:
/usr/local/cuda-8.0/lib64/libcudnn.so.5.1
/usr/local/cuda-8.0/libcudnn.so.5.1
.5.1
Please specify the Cudnn version you want to use. [Leave empty to use system default]: 5
Please specify the location where cuDNN 5 library is installed. Refer to README.md for more details. [Default is /usr/local/cuda]: 
Please specify a list of comma-separated Cuda compute capabilities you want to build with.
You can find the compute capability of your device at: https://developer.nvidia.com/cuda-gpus.
Please note that each additional compute capability significantly increases your build time and binary size.
[Default is: "3.5,5.2"]: 6.1

Create the pip package and install

bazel build -c opt --config=cuda //tensorflow/tools/pip_package:build_pip_package
bazel-bin/tensorflow/tools/pip_package/build_pip_package /tmp/tensorflow_pkg
sudo pip install /tmp/tensorflow_pkg/tensorflow-0.11.0rc0-py2-none-any.whl

Configures

Related Papers

Deep Q Networks (DQN)

Some Changes and Optimizations

Deep Reinforcement Learning in 3D environment

Experiment Environments

Learning

Initial Plan



Course Work

Learning Resources

Reinforcement Learning

Convolutional Neural Network

Deep Reinforcement Learning

Learning