SciPy 2017

Scientific Computing with Python (SciPy 2017, YouTube Official Playlist)

Tutorials

Talks

Japanese Kids Shows, Movies, Games, and Videos for Immersion

I have been trying to learn Japanese lately and have been using a lot of Japanese kids shows, Movies, and Games for immersion.  Enjoy!

Youtube Channels

Youtube Playlists

Steam Japanese Search

Steam in Japanese

Reddit Videogame Thread

List of Japanese Pokemon Names

List of Pokemon Moves

Good Steam games that can be switched to Japanese spoken dialog w/ Japanese subtitles

  1. Skyrim: Special Edition
  2. The Witcher 3: Wild Hunt
  3. Rise of the Tomb Raider
  4. Doom
  5. Romance of the Three Kingdoms XIII / 三國志13
  6. Nier: Automata
  7. Code of Princess
  8. Borderlands 2
  9. Borderlands: The Pre-Sequel
  10. Fallout 4
  11. Dishonored 2
  12. Lego City Undercover
  13. Resident Evil 7
  14. FINAL FANTASY X/X-2 HD Remaster
  15. Final Fantasy Type-0
  16. Ultra Street Fighter 4
  17. Street Fighter V
  18. Grid 2
  19. Grid Autosport
  20. Alan Wake
  21. Resident Evil Revelations
  22. Resident Evil Revelations 2
  23. Yaiba: Ninja Gaiden Z
  24. Dynasty Warriors 8:Xtreme Legends Complete Edition/ 真・三國無双7 with 猛将伝
  25. Sid Meier’s Civilization VI
  26. METAL GEAR SOLID V: GROUND ZEROES
  27. METAL GEAR SOLID V: THE PHANTOM PAIN
  28. Disgaea PC
  29. Disgaea 2 PC
  30. I am Setsuna
  31. Castle of Shikigami 式神の城
  32. THE KING OF FIGHTERS XIV STEAM EDITION
  33. Japanese School Life
  34. Astro Boy: Edge of Time
  35. Learn Japanese To Survive! Hiragana Battle
  36. Learn Japanese To Survive! Katakana War

Good Steam games: Japanese spoken dialog w/ No Japanese subtitles

  1. Final Fantasy 13
  2. Final Fantasy 13-2
  3. Lego Worlds
  4. Senko no Ronde 2 旋光の輪舞2

Good Steam games: Japanese text only, no Japanese audio

  1. Call of Duty: Black Ops 3
  2. DARK SOULS: Prepare To Die Edition
  3. Dark Souls 3
  4. Resident Evil 5
  5. Resident Evil / biohazard HD REMASTER
  6. Resident Evil 0 / biohazard 0 HD REMASTER
  7. Half Life 2
  8. Portal
  9. Portal 2
  10. Romance of the Three Kingdoms 1,2,3 (No English, Only Japanese text)
  11. Mafia 3
  12. Saint’s Row 4
  13. Sid Meier’s Civilization V
  14. Flinthook
  15. Mighty Gunvolt
  16. Tekken 7
  17. Mega Man Legacy Collection / ロックマン クラシックス コレクション
  18. Devil May Cry 4 Special Edition
  19. Sid Meier’s Civilization: Beyond Earth
  20. Tetris Ultimate
  21. Witch and Hero(魔女と勇者)
  22. Duke Nukem 3D: 20th Anniversary World Tour
  23. Dead Rising 1 & 2
  24. Dead Rising 3 Apocalypse Edition
  25. 軒轅劍外傳穹之扉(The Gate of Firmament)
  26. Stardew Valley
  27. 4人打ちアクション麻雀 / ACTION MAHJONG (No English)
  28. Kyurinaga’s Revenge
  29. Double Dragon IV
  30. Demon Hunter 3: Revelation
  31. Higurashi When They Cry

Good Nintendo Console Games

  1. Switch – Region Free so North American and Japanese versions work
    1. PuyoPuyo Tetris (youtube playlist) – JP version – Get the Game!
    2. I am Setsuna (youtube playlist) – Get the Game!
    3. Minna de WaiWai! Spelunker (youtube playlist) – Get the Game!
    4. Disgaea 5 Complete (youtube playlist) – Get the Game!
    5. Seiken Densetsu Collection (only Japanese)
    6. Dragon Quest Heroes I & II (only Japanese)
    7. Nobunaga’s Ambition: Sphere of Influence (only Japanese)
    8. Monster Hunter XX (only Japanese)
    9. Zelda: Breath of the Wild?
    10. Bomberman R?
  2. 3DS
    1. Pokémon Sun – Get the Game!
    2. Pokémon Moon – Get the game!
    3. Pokémon Ultra Sun – Get the Game!
    4. Pokémon Ultra Moon – Get the Game!
    5. Phoenix Wright: Ace Attorney Trilogy – No Japanese Voices only text
  3. DS (North America)
    1. My Japanese Coach DS (youtube playlist) – Get the Game!
    2. Castlevania: Portrait of Ruin (Japanese Voices/Options L+A)
    3. Castlevania: Order of Ecclesia (Japanese Voices/Options)
    4. Puyo Pop Fever (youtube playlist) – Get the Game!
    5. Sonic Rush (English Voices & Text for NA DS, English Voices/Japanese Text for Japanese DS)
    6. Sonic Rush Adventure (English Voices & Text for NA DS, English Voices/Japanese Text for Japanese DS)
  4. DS (Japanese) – Will work with North American DS
    1. Little Charo (Article)
    2. Kanji Dictionary: Kanji Sonomama Rakubiki Jiten – Get the Game!
    3. Ninokuni: Shikkoku No Madoushi – Get the Game!
    4. Pokemon DS games
    5. Zelda DS games?
    6. Any Japanese DS game with NTR product code (TWL product code doesn’t work on North American 3DS, it will work on NA DS)
  5.  Wii
    1. Xenoblade Chronicles

Ok Japanese shows with Japanese Subtitles that match the Dialogue

  1. Placeholder

Japanese Video Game Consoles

  1. Japanese 3ds at Play Asia

IPad/Android Games/Apps

  1. Anki
  2. Mindsnacks Japanese ()
  3. Duolingo (youtube playlist)
  4. Fantasy Life Online?

Doulingo wikia

Good Japanese kids shows with Japanese Subtitles that match the Japanese Dialogue

  1. Placeholder

Good Japanese kids shows – Japanese Subtitles don’t match dialogue or no subtitles

  1. Placeholder

Good Movies with Japanese Subtitles that match the Japanese Dialogue

  1. Placeholder

Good Movies – Japanese Subtitles don’t match dialogue or no subtitles

  1. Placeholder

Docker for AWS

Launch Stack on AWS (Costs Money!)

  1. Picked 1 Manager Node (t2.micro), Picked 1 Worker Node (t2.medium or t2.large)
    1. Or 1 Manager Node(t2.medium or t2.large), 0 Worker Nodes(doesn’t matter)
  2. Picked one of my AWS security keys
  3. Launch using Create Button (Costs Money!)
  4. Services->EC2->Docker-Manager to get Public DNS
    1. Or CloudFormation->Stacks->Docker->Outputs->Managers
      1. Click on link under “Managers->Value” and see Public DNS of Manager Node
  5. Ssh to manager node using AWS security key and docker as username
    1. Either use “Connect” in EC2 or
    2. ssh -i “mykey.pem” docker@<Public DNS>
      1. Make sure mykey.pem has correct permissions
        1. chmod 400 mykey.pem
    3. or convert .pem to .ppk and use putty
  6. Launch a docker container (choose one for now)
    1. docker service create --name dsnb -p 8888:8888 jupyter/datascience-notebook
      1. github link, dockerhub link
      2. took about 20 minutes to start up 🙁
  7. docker node ls
  8. docker node inspect <node-name>
  9. docker service ls
  10. docker service inspect <node-service-name>
    1. docker service inspect dsnb
  11. docker service ps <node-service-name>
    1. docker service ps dsnb
  12. containerid=`docker ps | grep jupyter | cut -d' ' -f1`
  13. docker exec -it $containerid bash
  14. Connect to port on Manager to access services on worker nodes
    1. CloudFormation->Stacks->Docker->Outputs->DefaultDNSTarget for web address
    2. jupyter notebook list
    3. Open <DefaultDNSTarget>:<port>?token=<token> in web browser
      1. Use 8888 for <port> in this example
  15. Make New Python 2 Notebook and then:
    1. !git clone https://github.com/TrackDR/Geospatial-Jupyter-Notebooks
    2. !git clone https://github.com/rajathkumarmp/Python-Lectures
      1. Python-Lectures/01.ipynb to start, rest are good too
    3. Open notebook out of any cloned directory
  16. CloudFormation->Stacks->Docker->Other Actions->Delete Stack when done using

Docs

  1. Deploy your app

Other Jupyter/Ipython Github Repositories to clone

  1. !git clone https://github.com/TrackDR/ModdedIpythonNotebooks
  2. !git clone https://github.com/JWarmenhoven/ISLR-python
  3. !git clone https://github.com/jdwittenauer/ipython-notebooks
    1. notebooks/ipython-notebooks/notebooks/language/Intro.ipynb to start
  4. !git clone https://github.com/masinoa/machine_learning
  5. !git clone https://github.com/jakevdp/sklearn_tutorial
  6. !git clone https://github.com/scikit-image/skimage-tutorials
    1. skimage-tutorials/lectures/00_images_are_arrays.ipynb
  7. !git clone https://github.com/spatialaudio/digital-signal-processing-lecture
    1. docs
  8. !git clone https://github.com/AllenDowney/ThinkDSP
  9. !git clone https://github.com/AllenDowney/ThinkBayes2
  10. !git clone https://github.com/AllenDowney/ThinkStats2
  11. !git clone https://github.com/AllenDowney/ThinkComplexity2
  12. !git clone https://github.com/AllenDowney/ModSimPy
  13. !git clone https://github.com/AllenDowney/CompStats
  14. !git clone https://github.com/multivac61/wave_digital_notebook
  15. !git clone https://github.com/learnml/machine-learning-specialization
  16. !git clone https://github.com/ipython/ipython-in-depth
  17. !git clone https://github.com/yhilpisch/dx
  18. !git clone https://github.com/yhilpisch/py4fi
  19. !git clone https://github.com/quantopian/research_public
  20. !git clone https://github.com/darribas/gds_scipy16
  21. https://github.com/jupyter/jupyter/wiki/A-gallery-of-interesting-Jupyter-Notebooks#introductory-tutorials

Docker images to run

  1. Jupyter
    1. jupyter/datascience-notebook

Docker images that will need to modify to run here, but looking to the future

  1. Kaggle
    1. kaggle/python
    2. kaggle/rstats
  2. Quantlib
    1. docker service create –name quantlib -p 8888:8888 lballabio/quantlib-notebook
    2. Videos at vimeo, blog w/ex, book
  3. QSTK
    1. ipython docker image
    2. ontouchstart docker image
    3. Twisted logic docker image
  4. BitQuant docker image
  5. Needs Nvidia-docker so may not work here
    1. Torch/iTorch/Ubuntu 14.04 Docker image: docker pull kaixhin/torch
    2. Torch/iTorch/CUDA 7/Ubuntu 14.04 Docker image: docker pull kaixhin/cuda-torch
  6. OSGeo list of Docker images
  7. Docker container for QGIS models (github, dockerhub)
    • can use with OSGeoLive
  8. My Dockerfiles at GitHub

Alternatives to Docker for AWS for running Jupyter Notebooks

  1. CoCalc (subscription based)
    1. Website, Pricing, Wikipedia, Chrome Extension, GitHub
    2. Or Run your own CoCalc using their Docker image
  2. Python Anywhere (website, blog, help, forums, pricing, modulestwitter, edu, edu help)
  3. Anaconda Enterprise Notebooks (Pricey! Website)

My Video Game Playlists in Japanese for Immersion

I have taken my own advice and started playing a few games in Japanese to try to learn the language better.  I record them and then upload to YouTube.  Here are a few of my current playlists.

Duolingo Japanese

I am Setsuna (Nintendo Switch) Jap/Eng

Disgaea 5 Complete (Nintendo Switch) Jap/Eng

Minna de WaiWai! Spelunker

Corpse Party 3DS Jap/Eng

Puyo Pop Fever DS

PuyoPuyo Tetris Japanese Demo

Mindsnacks Japanese

Pokémon Ultra Moon

 

Geospatial Python

Libraries

Ecosystems

Docker

Examples

Courses

  • GeoPython – AutoGIS 2016 at University of Helsinki (webpage)
  • Geographic Data Science with PySAL and the pydata stack Beginner | SciPy 2016 | Sergio Rey (youtube, github, page)

 

My Video Game Playlists in Chinese for Immersion

I have taken my own advice and started playing a few games in Mandarin Chinese to try to learn the language better.  I record them and then upload to YouTube.  Here are a few of my current playlists.

Pokémon Moon – 3DS

The Gate of the Firmament

Season of 12 Colors

Chinese Skill

Mindsnacks Mandarin

Rise of The Tomb Raider

Half-Life 2

My Chinese Coach – DS

The Talos Principle

Mayjasmine episode01 What is God?

Gaokao.Love.100Days

Romance of the Three Kingdoms 13

Weapon Shop Fantasy

Quantitative Finance Resources

Datasets

  1. Quandl: page (stocks guide)
    1. Python api
    2. Browse databases
    3. Premium content
  2. Google Finance (page)
  3. Yahoo Finance (page)
  4. Kaggle
    1. Winton Stock Market Challenge (page, data)

Software/Frameworks

  1. Quantopian (main, github)
  2. Python with Finance
    1. Dx Analytics (main, github)
    2. Python for Finance (page)
    3. Derivative Analytics with Python (page)
    4. Quant Platform (page)
    5. Python Quants (page)
  3. QSTK (page, github)
  4. Quantlib (page, github)
  5. Investopedia (page, simulator)

Videos/Courses

  1. Computational Investing Part 1 – GT – Coursera Page
  2. Machine Learning for Trading – GT – Udacity Page
    1. Intro material (youtube playlist)
    2. Machine learning (youtube playlist)
  3. UW – Intro to Computational Finance (Cousera)
  4. Quantopian (Getting Started)
  5. Sentdex
    1. Python for Finance with Zipline and Quantopian (youtube playlist)
    2. Scikit-learn ML with Python (youtube playlist)
    3. Programming to aid Fundamental Investing (youtube playlist)
    4. ML for Forex, Stock Analysis, Algo Trading (youtube playlist)
    5. Big Data with Stock Trading (youtube playlist)
    6. Monte Carlo Python (youtube playlist)
    7. Matplotlib (youtube playlist)
  6. Python Quants (page)
    1. 2014 For Python Quants (playlist)
  7. Cousera – Michigan
    1. Principles of Valuation: Time Value of Money (page)
    2. Risk and Return (page)
    3. Alternative Methods (page)
    4. Projects and Companies (page)
    5. Capstone (page)
  8. Cousera – Chicago – Asset Pricing
    1. Part 1
    2. Part 2
  9. Quantlib
    1. vimeo page, youtube playlist
    2. ebook
    3. twitter
    4. Implementing Quantlib (blog)
    5. Examples in python, blog
  10. Coursera – Wharton
    1. Quant Modeling (page)
    2. Spreadsheets & Models (page)
    3. Risk Models (page)
    4. Decision Making and Scenarios (page)
    5. Capstone (page)
  11. Cousera – Columbia
    1. Financial Engineering and Risk Management I (page)
    2. Financial Engineering and Risk Management II (page)
  12. Khan Aademy
    1. Valuation and Investing (youtube playlist)

Methods

  1. Sentiment Analysis
  2. Machine Learning
  3. Deep Learning
  4. MDPs, RL, and QL

Books

  1. Hull (Options, Futures, and Other Derivatives (9th Edition) on Amazon)
  2. R Cookbook (Amazon)

Papers

  1. ArXiv – Q-fin, Qfin recent, twitter
    1. q-fin.CP – Computational Finance (new, recent, current month)
      Computational methods, including Monte Carlo, PDE, lattice and other numerical methods with applications to financial modeling
    2. q-fin.EC – Economics (new, recent, current month)
      Economics, including micro and macro economics, international economics, theory of the firm, labor economics, and other economic topics outside of finance
    3. q-fin.GN – General Finance (new, recent, current month)
      Development of general quantitative methodologies with applications in finance
    4. q-fin.MF – Mathematical Finance (new, recent, current month)
      Mathematical and analytical methods of finance, including stochastic, probabilistic and functional analysis, algebraic, geometric and other methods
    5. q-fin.PM – Portfolio Management (new, recent, current month)
      Security selection and optimization, capital allocation, investment strategies and performance measurement
    6. q-fin.PR – Pricing of Securities (new, recent, current month)
      Valuation and hedging of financial securities, their derivatives, and structured products
    7. q-fin.RM – Risk Management (new, recent, current month)
      Measurement and management of financial risks in trading, banking, insurance, corporate and other applications
    8. q-fin.ST – Statistical Finance (new, recent, current month)
      Statistical, econometric and econophysics analyses with applications to financial markets and economic data
    9. q-fin.TR – Trading and Market Microstructure (new, recent, current month)
      Market microstructure, liquidity, exchange and auction design, automated trading, agent-based modeling and market-making

People

  1. Ipython Quant blog

Groups

  1. Reddit
    1. Algotrading Reddit
    2. Investing Reddit
    3. Stock Market Reddit
    4. Stocks Reddit
    5. Forex Reddit
    6. Finance Reddit
    7. Bitcoin Reddit
    8. Bitcoin Markets Reddit

Docker/AWS

  1. Quantlib (docker images)
  2. QSTK
    1. ipython docker image
    2. ontouchstart docker image
    3. Twisted logic docker image
  3. BitQuant docker image

Terms

  1. Mathematical Finance (Wikipedia pageWikipedia subject)
  2. Alpha
  3. Beta
  4. Sharpe Ratio
  5. Bollinger Bands
  6. Drawdown
  7. SMA (crossover)
  8. CAPM
  9. Options
  10. Futures
  11. Forex

 

 

Chinese Kids Shows, Movies, Games, and Videos for Immersion

I have been trying to learn Chinese lately and have been using a lot of Chinese kids shows, Movies, and Games for immersion.  I started with Fun Fun Elmo and have watched all the shows listed below.  Enjoy!

Good Chinese kids shows with Chinese Subtitles that match the Chinese Dialogue

  1. Pac-Man and the Ghostly Adventures – Netflix
  2. Fuller House – Netflix

Good Chinese kids shows – Chinese Subtitles don’t match dialogue or no subtitles

  1. Fun Fun Elmo (Season 1) (Season 2)

Good Movies with Chinese Subtitles that match the Chinese Dialogue

  1. Lion King (Amazon Region 6 DVD)
  2. Clone Wars (Amazon Region 6 Movie)
  3. Kiki’s Delivery Service (Amazon Region 6 DVD)
  4. Harry Potter – all movies (Amazon Region 6 Movie)

Good Movies – Chinese Subtitles don’t match dialogue or no subtitles

  1. The Grandmaster (Amazon)
  2. Hero (Amazon)
  3. Crouching Tiger Hidden Dragon (Amazon)
  4. Shaolin (Amazon)
  5. House of Flying Daggers (Amazon)
  6. The Sorcerer and the White Snake (Amazon)
  7. Dangerous Liasons (Amazon)

Youtube Channels

Youtube Playlists

Steam Chinese Search, Traditional Chinese, Simplified Chinese

Steam in Chinese

Reddit Videogame Thread

Starcraft

Good Steam games that can be switched to Chinese spoken dialog w/ Chinese subtitles

  1. The Gate of Firmament (youtube english subtitles, mandarin audio) (youtube mandarin subtitles and audio)
  2. Gaokao.Love.100Days (youtube playlist)
  3. Mayjasmine episode01 What is God?
  4. Rise of the Tomb Raider
  5. The Talos Principle
  6. Season of 12 Colors (youtube english subtitles)
  7. Romance of the Three Kingdoms 13

Good Steam games: Chinese spoken dialog w/ No Chinese subtitles

  1. Half-Life 2
  2. Portal?
  3. Portal 2?
  4. Age of Empires 2 – Simplified Chinese
  5. Age of Mythology – Traditional Chinese
  6. Oblivious Garden – Carmina Burana
  7. Eternal Senia
  8. Alan Wake
  9. Alan Wake’s American Nightmare
  10. DOTA2
  11. FEZ
  12. Unepic

Good Steam games: Chinese text only, no Chinese audio

  1. Detention
  2. Witcher 1,2,3
  3. Tomb Raider
  4. Assassin’s Creed 3
  5. Dark Souls 1,2,3
  6. Botanicula
  7. Civ 5
  8. Counter Strike:Global Offensive
  9. XCOM Enemy Unknown
  10. Borderlands 2, Pre-sequel
  11. Dead Rising 3
  12. Fable Anniversary
  13. GTA 5
  14. Resident Evil Revelations 2
  15. Trine 2, 3
  16. Fallout 4
  17. Assassin’s Creed Unity, Syndicate
  18. NOBUNAGA’S AMBITION: Souzou
  19. The Maw
  20. Insurgency
  21. Ori and the Blind Forest
  22. PewDiePie: Legend of the Brofist
  23. Heroes of Might and Magic 3
  24. Black Ops 3
  25. Might and Magic Heroes VII
  26. Lego Worlds
  27. Dragon Quest Heroes
  28. The Emerald Maiden: Symphony of Dreams
  29. Fairy Tale Mysteries: The Puppet Thief
  30. Doom (2016)
  31. Double Dragon IV

Good Nintendo Games

  1. Pokémon Moon – 3DS (youtube playlist) – Get the game!
  2. My Chinese Coach – DS

Ok Chinese shows with Chinese Subtitles that match the Dialogue

  1. Placeholder

Chinese Video Game Consoles

  1. Ique – Chinese Nintendo 64
  2. Chinese ique 3ds

IPad/Android Games/Apps

  1. Chinese Skill (youtube playlist)
  2. Anki
  3. Mindsnacks Mandarin (youtube playlist)

Doulingo wikia

Fully Convolutional Networks

Fully Convolutional Networks for Semantic Segmentation (arxiv, video, caffemodel, example, presentation, solved issue)

Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs (arxiv, DeepLab bitbucket, github, pretrained models, UCLA page)

Conditional Random Fields as Recurrent Neural Networks (arxiv, project, demo, github)

Weakly- and Semi-Supervised Learning of a DCNN for Semantic Image Segmentation (arxiv)

Constrained Convolutional Neural Networks for Weakly Supervised Segmentation (arxiv)

BoxSup: Exploiting Bounding Boxes to Supervise Convolutional Networks for Semantic Segmentation (arxiv)