Docker for AWS

Launch Stack on AWS (Costs Money!)

  1. Picked 1 Manager Node (t2.micro), Picked 1 Worker Node (t2.medium or t2.large)
    1. Or 1 Manager Node(t2.medium or t2.large), 0 Worker Nodes(doesn’t matter)
  2. Picked one of my AWS security keys
  3. Launch using Create Button (Costs Money!)
  4. Services->EC2->Docker-Manager to get Public DNS
    1. Or CloudFormation->Stacks->Docker->Outputs->Managers
      1. Click on link under “Managers->Value” and see Public DNS of Manager Node
  5. Ssh to manager node using AWS security key and docker as username
    1. Either use “Connect” in EC2 or
    2. ssh -i “mykey.pem” docker@<Public DNS>
      1. Make sure mykey.pem has correct permissions
        1. chmod 400 mykey.pem
    3. or convert .pem to .ppk and use putty
  6. Launch a docker container (choose one for now)
    1. docker service create --name dsnb -p 8888:8888 jupyter/datascience-notebook
      1. github link, dockerhub link
      2. took about 20 minutes to start up 🙁
  7. docker node ls
  8. docker node inspect <node-name>
  9. docker service ls
  10. docker service inspect <node-service-name>
    1. docker service inspect dsnb
  11. docker service ps <node-service-name>
    1. docker service ps dsnb
  12. containerid=`docker ps | grep jupyter | cut -d' ' -f1`
  13. docker exec -it $containerid bash
  14. Connect to port on Manager to access services on worker nodes
    1. CloudFormation->Stacks->Docker->Outputs->DefaultDNSTarget for web address
    2. jupyter notebook list
    3. Open <DefaultDNSTarget>:<port>?token=<token> in web browser
      1. Use 8888 for <port> in this example
  15. Make New Python 2 Notebook and then:
    1. !git clone
    2. !git clone
      1. Python-Lectures/01.ipynb to start, rest are good too
    3. Open notebook out of any cloned directory
  16. CloudFormation->Stacks->Docker->Other Actions->Delete Stack when done using


  1. Deploy your app

Other Jupyter/Ipython Github Repositories to clone

  1. !git clone
  2. !git clone
  3. !git clone
    1. notebooks/ipython-notebooks/notebooks/language/Intro.ipynb to start
  4. !git clone
  5. !git clone
  6. !git clone
    1. skimage-tutorials/lectures/00_images_are_arrays.ipynb
  7. !git clone
    1. docs
  8. !git clone
  9. !git clone
  10. !git clone
  11. !git clone
  12. !git clone
  13. !git clone
  14. !git clone
  15. !git clone
  16. !git clone
  17. !git clone
  18. !git clone
  19. !git clone
  20. !git clone

Docker images to run

  1. Jupyter
    1. jupyter/datascience-notebook

Docker images that will need to modify to run here, but looking to the future

  1. Kaggle
    1. kaggle/python
    2. kaggle/rstats
  2. Quantlib
    1. docker service create –name quantlib -p 8888:8888 lballabio/quantlib-notebook
    2. Videos at vimeo, blog w/ex, book
  3. QSTK
    1. ipython docker image
    2. ontouchstart docker image
    3. Twisted logic docker image
  4. BitQuant docker image
  5. Needs Nvidia-docker so may not work here
    1. Torch/iTorch/Ubuntu 14.04 Docker image: docker pull kaixhin/torch
    2. Torch/iTorch/CUDA 7/Ubuntu 14.04 Docker image: docker pull kaixhin/cuda-torch
  6. OSGeo list of Docker images
  7. Docker container for QGIS models (github, dockerhub)
    • can use with OSGeoLive
  8. My Dockerfiles at GitHub

Alternatives to Docker for AWS for running Jupyter Notebooks

  1. CoCalc (subscription based)
    1. Website, Pricing, Wikipedia, Chrome Extension, GitHub
    2. Or Run your own CoCalc using their Docker image
  2. Python Anywhere (website, blog, help, forums, pricing, modules, twitter, edu, edu help)
  3. Anaconda Enterprise Notebooks (Pricey! Website)

Build and Install latest OpenCV with CUDA 6.5 on an AWS GPU instance with Ubuntu 14.04

Directions for installing Cuda 6.5 on AWS GPU Instance is here:

Install all the necessary packages to build opencv:
sudo apt-get update
sudo apt-get upgrade
sudo apt-get install build-essential git cmake cmake-curses-gui zlib1g-dev libeigen3-dev

sudo apt-get install checkinstall pkg-config yasm libjpeg-dev libjasper-dev libavcodec-dev libavformat-dev libswscale-dev libdc1394-22-dev libxine-dev libgstreamer0.10-dev libgstreamer-plugins-base0.10-dev libv4l-dev python-dev python-numpy libtbb-dev libqt4-dev libgtk2.0-dev libmp3lame-dev libopencore-amrnb-dev libopencore-amrwb-dev libtheora-dev libvorbis-dev libxvidcore-dev x264 v4l-utils

Make the necessary source and build folders:
(I assume you are using bash here)
export todaysdate=`date "+%Y%m%d"`
mkdir opencvsrc$todaysdate
mkdir opencvbld$todaysdate

Download the opencv source:
cd opencvsrc$todaysdate
git clone

Go into the build directory and use ccmake to configure your build.
cd ../opencvbld$todaysdate
ccmake ../opencvsrc$todaysdate/opencv

Hit ‘c’ to configure and then change these variables by moving to that line and hitting Enter.


Hit ‘c’ to configure
Hit ‘g’ to generate

Now build and install opencv.
sudo make install

For getting test data and running tests on that data do this:
git clone git://
export OPENCV_TEST_DATA_PATH=/home/ubuntu/opencv_extra/testdata/

Lots of tests are in /usr/local/bin

If you just want to use my public AMI with the OpenCV source from March 25,2015 it is here (for US-East): ami-fa200b92