Jupyter Notebook

Installation #

Jupyter notebook #

# BY PIP
pip install --upgrade pip
pip install --upgrade ipython jupyter

# BY CONDA
conda install ipython jupyter

Or read more in this note.

If you meet error OSError: [Errno 99] Cannot assign requested address, try

jupyter notebook --ip=127.0.0.1 --port=8080
# or
jupyter notebook --ip=127.0.0.1 --port=8080 --allow-root

Setting up a password #

# create a juputer notebook config file
# it can be used for other settings
# https://jupyter-notebook.readthedocs.io/en/stable/public_server.html#prerequisite-a-notebook-configuration-file
jupyter notebook --generate-config

# create a new password
# note: sha1 cannot be reverted!!
jupyter notebook password

Inside notebook:

from notebook.auth import passwd
passwd()

With docker

# create a sha1 password
# download file create_sha1.py from https://github.com/dinhanhthi/scripts
# run ./create_sha1.py

# docker-compose.yml
environment:
  - PASSWD='sha1:d03968479249:319e92302e68d601392918f011d6c9334493023f'

# Dockerfile
CMD /bin/bash -c 'jupyter lab --no-browser --allow-root --ip=0.0.0.0 --NotebookApp.password="$PASSWD" "[email protected]"'

R with jupyter notebook #

Read more here.

# install jupyter
sudo apt-get install libzmq3-dev libcurl4-openssl-dev libssl-dev jupyter-core jupyter-client

# install R on linux
sudo apt install r-base

# R kernel for Jupyter Notebook
R # enter R environnement
# install R kernel
install.packages(c('repr', 'IRdisplay', 'IRkernel'), type = 'source')
# or
install.packages(c('repr', 'IRkernel'), type = 'source')
# make jupyter see r kernel
IRkernel::installspec() # current user
IRkernel::installspec(user = FALSE) # global

# embedded R
# use by cell magic %%R
pip install rpy2

# in a notebook
%load_ext rpy2.ipython

# then use
%%R
# R's codes

Other tips #

• Running 2 tasks in the same cell TAKE LONGER TIME than running each on different cells.
• Download a folder in jupyter notebook:

  • Inside notebook, use:

    %%bash
    tar -czf archive.tar.gz foldername

  • Or using nbzip (only working on current server).

Check the info #

# function's info
?<func-name>

# function's shortcode
??<func-name>

# get the list of current variables
whos

Check where command executed from (in your $path)?

!type python

python is /Users/thi/anaconda/envs/python3.6/bin/python

Multiline commands #

# Using '\'
df.columns = df.columns.str.replace('.', ' ')\
                       .str.replace('\s+', ' ')\
                       .str.strip().str.upper()

You CANNOT put # comments at the end of each line break!

Hotkeys / Shortcuts #

There are 2 modes: command mode (pres ESC to activate) and edit mode (Enter to activate). Below are the most useful ones (for me).

You can edit / add more shortcuts in Help > Edit Keyboard Shortcuts.

Jupyter notebook on remote server #

Open jupyter notebook in local browser but the backend-server is on remote.

• If jupyter server is already running on remote at http://192.168.0.155:9889,

  ssh -N -L localhost:9888:192.168.0.155:9899 <username-remote>@<remote-host> -p <port>
  # if there is no port, remove `-p <port>`

  Open browser: http://localhost:9888 (type password if needed).

• If jupyter server is not running on remote yet,

  # connect to remote
  ssh <username-remote>@<remote-host> -p <port>
  # if there is no port, remove `-p <port>`

  On remote,

  # run juputer with custom port
  jupyter notebook --no-browser --port=9899
  
  # if there is error `OSError: [Errno 99] Cannot assign requested address`
  jupyter notebook --ip=0.0.0.0 --no-browser --port=9899
  
  # if there is error `Running as root is not recommended`
  jupyter notebook --ip=0.0.0.0 --no-browser --port=9899 --alow-root

  It's running and there are somethings like that,

  http://127.0.0.1:9889/?token=717d9d276f0537a9...831793df6319ad389accd

  Open another terminal window and type,

  ssh -N -L localhost:9888:localhost:9889 <username-remote>@<remote-host> -p <port>
  # if there is no port, remove `-p <port>`
  # there is nothing but it's running

  Open browser:

  http://localhost:9888/?token=717d9d276f0537a9...831793df6319ad389accd

You can choose any port number you wanna instead of 9888 and 9889 (they can be the same), note that, you need to use a port number GREATER THAN 8000!

Install new python package inside Jupyter Notebook #

Using conda^[ref],

# Install a conda package in the current Jupyter kernel
import sys
!conda install --yes --prefix {sys.prefix} numpy

# DON'T DO THIS
!conda install --yes numpy

Using pip,

# Install a pip package in the current Jupyter kernel
import sys
!{sys.executable} -m pip install numpy

# DON'T DO THIS
!pip install numpy

Check version and update/upgrade,

!pip show pandas

Display dataframes side-by-side #

from IPython.display import display_html
def display_side_by_side(*args):
    html_str=''
    for df in args:
        html_str+=df.to_html()
    display_html(html_str.replace('table','table style="display:inline; margin-right: 5px;"'),raw=True)

display_side_by_side(df1,df2,df1)

Get previous outputs #

_ # previous output
__ # second-to-last output
___ # third-to-last output

Display 2 figures side-by-side markdown cell #

Put below codes in the markdown cell of Jupyter Notebook.

<tr>
  <td> <img src="Nordic_trails.jpg" alt="Drawing" style="width: 250px;"/> </td>
  <td> <img src="Nordic_trails.jpg" alt="Drawing" style="width: 250px;"/> </td>
</tr>

Magic Functions #

• Check the full list (in examples) here or their docs here.
• You can define your custom magic functions here.

Auto update the new updated modules (put at the beginning of the notebook)

%load_ext autoreload
%autoreload 2 # Reload all modules every time before executing

%autoreload 0 # disable autoreloader

Check more settings of %autoreload here.

Show the plots inside the notebook:

%matplotlib inline

Get the commands from 1 to 4:

%history -n 1-4 # get commands 1 to 4

Extensions #

Table of contents #

1. Install npm and nodejs.
2. Install this extension.
3. Enable in jupyter lab view.
4. Refresh the page.

# errors
# UnicodeDecodeError: 'ascii' codec can't decode byte 0xf0 in position 23: ordinal not in range(128)
npm config set unicode false

Debugger #

1. Install xeus-python, jupyterlab

pip install xeus-python
pip install jupyterlab

2. Install this extension.
3. Refresh the page, you have to choose kernel xpython (instead of Python 3) to use the debugger.