Docker for AWS

Launch Stack on AWS (Costs Money!)

  1. Picked 1 Manager Node (t2.micro), Picked 1 Worker Node (t2.medium or t2.large)
    1. Or 1 Manager Node(t2.medium or t2.large), 0 Worker Nodes(doesn’t matter)
  2. Picked one of my AWS security keys
  3. Launch using Create Button (Costs Money!)
  4. Services->EC2->Docker-Manager to get Public DNS
    1. Or CloudFormation->Stacks->Docker->Outputs->Managers
      1. Click on link under “Managers->Value” and see Public DNS of Manager Node
  5. Ssh to manager node using AWS security key and docker as username
    1. Either use “Connect” in EC2 or
    2. ssh -i “mykey.pem” docker@<Public DNS>
      1. Make sure mykey.pem has correct permissions
        1. chmod 400 mykey.pem
    3. or convert .pem to .ppk and use putty
  6. Launch a docker container (choose one for now)
    1. docker service create --name dsnb -p 8888:8888 jupyter/datascience-notebook
      1. github link, dockerhub link
      2. took about 20 minutes to start up ūüôĀ
  7. docker node ls
  8. docker node inspect <node-name>
  9. docker service ls
  10. docker service inspect <node-service-name>
    1. docker service inspect dsnb
  11. docker service ps <node-service-name>
    1. docker service ps dsnb
  12. containerid=`docker ps | grep jupyter | cut -d' ' -f1`
  13. docker exec -it $containerid bash
  14. Connect to port on Manager to access services on worker nodes
    1. CloudFormation->Stacks->Docker->Outputs->DefaultDNSTarget for web address
    2. jupyter notebook list
    3. Open <DefaultDNSTarget>:<port>?token=<token> in web browser
      1. Use 8888 for <port> in this example
  15. Make New Python 2 Notebook and then:
    1. !git clone
    2. !git clone
      1. Python-Lectures/01.ipynb to start, rest are good too
    3. Open notebook out of any cloned directory
  16. CloudFormation->Stacks->Docker->Other Actions->Delete Stack when done using


  1. Deploy your app

Other Jupyter/Ipython Github Repositories to clone

  1. !git clone
  2. !git clone
  3. !git clone
    1. notebooks/ipython-notebooks/notebooks/language/Intro.ipynb to start
  4. !git clone
  5. !git clone
  6. !git clone
    1. skimage-tutorials/lectures/00_images_are_arrays.ipynb
  7. !git clone
    1. docs
  8. !git clone
  9. !git clone
  10. !git clone
  11. !git clone
  12. !git clone
  13. !git clone
  14. !git clone
  15. !git clone
  16. !git clone
  17. !git clone
  18. !git clone
  19. !git clone
  20. !git clone

Docker images to run

  1. Jupyter
    1. jupyter/datascience-notebook

Docker images that will need to modify to run here, but looking to the future

  1. Kaggle
    1. kaggle/python
    2. kaggle/rstats
  2. Quantlib
    1. docker service create –name quantlib -p 8888:8888 lballabio/quantlib-notebook
    2. Videos at vimeo, blog w/ex, book
  3. QSTK
    1. ipython docker image
    2. ontouchstart docker image
    3. Twisted logic docker image
  4. BitQuant docker image
  5. Needs Nvidia-docker so may not work here
    1. Torch/iTorch/Ubuntu 14.04 Docker image: docker pull kaixhin/torch
    2. Torch/iTorch/CUDA 7/Ubuntu 14.04 Docker image: docker pull kaixhin/cuda-torch
  6. OSGeo list of Docker images
  7. Docker container for QGIS models (github, dockerhub)
    • can use with OSGeoLive
  8. My Dockerfiles at GitHub

Alternatives to Docker for AWS for running Jupyter Notebooks

  1. CoCalc (subscription based)
    1. Website, Pricing, Wikipedia, Chrome Extension, GitHub
    2. Or Run your own CoCalc using their Docker image
  2. Python Anywhere (website, blog, help, forums, pricing, modules, twitter, edu, edu help)
  3. Anaconda Enterprise Notebooks (Pricey! Website)