- URLs
- Colab & Github
- Hotkeys / Shortcuts
- Import libraries
- Upgrade/Switch TensorFlow versions
- Git with Colab
- Keep Colab awake
- Change to current working directory
- Upload a file to Colab[ref]
- Install
- Basic command lines
- Create a screen + list of command lines
- Hotkeys
- Errors
- Reference
- PyTorch with GPU
- Install 7zip reader, GraphViz, PyDot, cartopy
- Save as HTML
- Other functions
URLs #
- Main site.
- Welcome site.
- SeedBank by Google: Collection of Interactive Machine Learning Examples.
Colab & Github #
"Open with Colab" any Jupyter Notebook file (.ipynb
) in Github. For example, the file's URL is:
https://github.com/dinhanhthi/dataquest-aio/blob/master/file-name.ipynb
You can open with colab with the URL:
https://colab.research.google.com/github/dinhanhthi/dataquest-aio/blob/master/file-name.ipynb
In the case you wanna import dataset (.csv
) from Github. First, open this .csv
file as RAW. Its URL may be
https://raw.githubusercontent.com/dinhanhthi/dataquest-aio/master/file.csv
and then use it as the url of the data file. Note that, you cannot use open
to read this file,
import csv
# We can use on localhost
opened_file = open(dataset_url, encoding="utf8")
read_file = csv.reader(opened_file)
# But we CAN'T use this `open` for the link from Github, we use:
from urllib.request import urlopen
opened_file = urlopen(dataset_url).read().decode('utf-8')
read_file = csv.reader(opened_file.splitlines())
Hotkeys / Shortcuts #
Check the command shortcuts in Tools > Keyboard shortcuts (Ctrl + M H), below are the most popular ones:
- Ctrl + S: save the notebook.
- Ctrl + Enter: run a cell in place.
- Shift + Enter: to run the cell and move focus to the next cell (adding one if none exists).
- Alt + Enter: run the cell and insert a new code cell immediately below it.
- Ctrl + M Y: convert a cell to a code cell.
- Ctrl + M M: convert a cell to a text cell.
- Ctrl + M D: delete current cell / selected cells.
- Ctrl + M A: insert a code cell above.
- Ctrl + M B: insert a code cell below.
- Ctrl + Alt + M: insert a comment.
- Ctrl + Space or Tab: autocomplete.
- Ctrl + H: global find/replace.
- Ctrl + G: global find next.
{:alert.alert-success}
We can use system commands in Colab with !<command>
. For example, !git clone ...
.
Import libraries #
!pip install -q matplotlib-venn
# or
!apt-get -qq install -y libfluidsynth1
Upgrade/Switch TensorFlow versions #
# To determine which version you're using:
!pip show tensorflow
# For the current version:
!pip install --upgrade tensorflow
# For a specific version:
!pip install tensorflow==1.2
# For the latest nightly build:
!pip install tf-nightly
Git with Colab #
Check out my note for Git.
# Initialize the git repository (optional)
!git init
# Set the global username and email
!git config --global user.email "[email protected]"
!git config --global user.name "Your Name"
# Add all the files
!git add -A
# or
!git add .
# Commit
!git commit -m "Comment for that commit"
# Pass your Github credentials
!git remote rm origin # in the case you meet "fatal: remote origin already exists"
!git remote add origin https://<github-username>:<github-password>@github.com/<github-username>/<repository-name>.git
# Push to origin
!git push -u origin master
If you don't want to use your username andd password, you can use "Personal access tokens" on Github. Create one here and then use,
!git git remote add origin https://<username>:<access-token>@github.com/<username>/<repo>.git
Keep Colab awake #
F12 then Console and type,
function ClickConnect(){
console.log("Working");
document.querySelector("colab-connect-button").shadowRoot.getElementById("connect").click()
}
setInterval(ClickConnect,60000)
Change to current working directory #
By default, the working directory is /content/
. One can use below command to change to another place,
%cd /content/data-science-learning
From that point, we are working on /content/data-science-learning
.
Upload a file to Colab[ref] #
Each user has a "machine" in /content/
.
Install #
sudo apt install screen # ubuntu
Basic command lines #
# check screen version
screen -v
# start new session (with name)
screen -S <name>
# list running sessions
screen -ls
# attach to a running session (without name)
screen -x
# attach to a running session (with name)
screen -rx <name>
# -x for an interactive (scrolling)
# detach a running session
screen -d <name> # or Ctrl + A, D
# kill a session
screen -X -S <name> quit
Delete sessions #
- Reattach first:
screen -r <name>
- Ctrl + A, K then Y
# kill ALL auto-created sesssions
screen -ls | grep pts | cut -d. -f1 | awk '{print $1}' | xargs kill
# kill all detached sessions
screen -ls | grep Detached | cut -d. -f1 | awk '{print $1}' | xargs kill
Create a screen + list of command lines #
screen -S 'dat' -dm bash -c 'cd /jekyll-site; bundle exec jekyll serve -I; exec sh'
Hotkeys #
- Detach: Ctrl + A, D
- Reattach: Ctrl + A, R
- Kill current session: Ctrl + A, K then Y
Errors #
# Cannot make directory '/run/screen': Permission denied
mkdir ~/.screen && chmod 700 ~/.screen
export SCREENDIR=$HOME/.screen
# also add this line to ~/.zshrc or ~/.bashrc
Reference #
Create a new cell and paste,
from google.colab import files
uploaded = files.upload()
for fn in uploaded.keys():
print('User uploaded file "{name}" with length {length} bytes'.format(
name=fn, length=len(uploaded[fn])))
Run 2 times this cell, at the 2nd time, you can choose your file.
Using Google Drive #
Run a cell containing following codes,
from google.colab import drive
drive.mount('/content/drive')
and then follow the guide on the screen. In order to access to the drive,
with open('/content/drive/My Drive/foo.txt', 'w') as f:
f.write('Hello Google Drive!')
Clone a repo from Github #
!git clone https://github.com/dinhanhthi/data-science-learning.git
The cloned folder are stored in /content/
. If you wanna pull
requests, use,
%cd /content/data-science-learning
!git pull
PyTorch with GPU #
To enable GPU backend for your notebook, go to Edit → Notebook Settings and set Hardware accelerator to GPU.[ref]
Install 7zip reader, GraphViz, PyDot, cartopy #
# https://pypi.python.org/pypi/libarchive
!apt-get -qq install -y libarchive-dev && pip install -q -U libarchive
import libarchive
# https://pypi.python.org/pypi/pydot
!apt-get -qq install -y graphviz && pip install -q pydot
import pydot
!apt-get -qq install python-cartopy python3-cartopy
import cartopy
Save as HTML #
Jupyter Notebook has an option to 'Download as' HTML (or other) format. Google Colaboratory does not.
Save your Colab notebook.
In your terminal:
jupyter nbconvert --to <output format> <filename.ipynb>
# jupyter nbconvert --to html mynotebook.ipynb
Other functions #
- Interrupt a long running python process: Runtime > Interrupt execution (Alt + M I).
- Support Jupyter magic commands, check full list here.