Google Colab

Last modified 10 months ago / Edit on Github


Colab & Github

👉 Check this section too.

Open with Colab from Github URL

"Open with Colab" any Jupyter Notebook file (.ipynb) in Github. For example, the file's URL is:

You can open with colab with the URL:
In the case you wanna import dataset (.csv) from Github.

First, open this .csv file as RAW. Its URL may be

and then use it as the url of the data file. Note that, you cannot use open to read this file,

import csv

# We can use on localhost
opened_file = open(dataset_url, encoding="utf8")
read_file = csv.reader(opened_file)

# But we CAN'T use this `open` for the link from Github, we use:
from urllib.request import urlopen
opened_file = urlopen(dataset_url).read().decode('utf-8')
read_file = csv.reader(opened_file.splitlines())
Save a notebook to Github

For example, I wanna save current notebook (eg. abc.ipynb to repository data-science-learning/playground).

From the notebook, File > Save a copy in Github > Choose a repository > Type playground/ before File path > OK.

Hotkeys / Shortcuts

Check the command shortcuts in Tools > Keyboard shortcuts (Ctrl + M H), below are the most popular ones (If you use MacOS, replace Ctrl with cmd):

Show the list
  • Ctrl + S: save the notebook.
  • Ctrl + Enter: run a cell in place.
  • Shift + Enter: to run the cell and move focus to the next cell (adding one if none exists).
  • Alt + Enter: run the cell and insert a new code cell immediately below it.
  • Ctrl + M Y: convert a cell to a code cell.
  • Ctrl + M M: convert a cell to a text cell.
  • Ctrl + M D: delete current cell / selected cells.
  • Ctrl + M A: insert a code cell above.
  • Ctrl + M B: insert a code cell below.
  • Ctrl + Alt + M: insert a comment.
  • Ctrl + Space or Tab: autocomplete.
  • Ctrl + H: global find/replace.
  • Ctrl + G: global find next.
Idea icon

We can use system commands in Colab with !<command>. For example, !git clone ....

Import libraries

!pip install -q matplotlib-venn
# or
!apt-get -qq install -y libfluidsynth1

Install permanently

Install with gsfuse (source)

Install gcsfuse,

!echo "deb gcsfuse-bionic main" > /etc/apt/sources.list.d/gcsfuse.list
!curl | apt-key add -
!apt -qq update
!apt -qq install gcsfuse
!mkdir googleBucketFolder
!gcsfuse --implicit-dirs colab-connect-bucket googleBucketFolder
Do these before continuing
%%writefile /key.json
"type": "service_account",
"project_id": "kora-id",
"private_key_id": "xxxxxxx",
"private_key": "-----BEGIN PRIVATE KEY-----\nxxxxxxx==\n-----END PRIVATE KEY-----\n",
"client_email": "[email protected]",
"client_id": "100380920993833371482",
"auth_uri": "",
"token_uri": "",
"auth_provider_x509_cert_url": "",
"client_x509_cert_url": "...."
import sys
nb_path = 'googleBucketFolder'
sys.path.insert(0, nb_path)

Whenever you want to install a package,

!pip install --target=$nb_path transformers
Install to Google Drive (My choice) (source)
import os, sys
from google.colab import drive
nb_path = '/content/notebooks'
# You may need to create INSTALLED_LIBS folder before following step
os.symlink('/content/gdrive/My Drive/Colab Notebooks/INSTALLED_LIBS', nb_path)
sys.path.insert(0, nb_path)
!pip install --target=$nb_path transformers

From now, everytime you open a new notebook,

import sys
sys.path.append('/content/gdrive/My Drive/Colab Notebooks/INSTALLED_LIBS')

from google.colab import drive
# Try?
!pip install --target=$nb_path transformers

Upgrade/Switch TensorFlow versions

# To determine which version you're using:
!pip show tensorflow
# For the current version:
!pip install --upgrade tensorflow
# For a specific version:
!pip install tensorflow==1.2
# For the latest nightly build:
!pip install tf-nightly

Git with Colab

Check out my note for Git.

# Initialize the git repository (optional)
!git init
# Set the global username and email
!git config --global "[email protected]"
!git config --global "Your Name"
# Add all the files
!git add -A
# or
!git add .
# Commit
!git commit -m "Comment for that commit"
# Pass your Github credentials
!git remote rm origin # in the case you meet "fatal: remote origin already exists"
!git remote add origin https://<github-username>:<github-password><github-username>/<repository-name>.git
# Push to origin
!git push -u origin master

If you don't want to use your username andd password, you can use "Personal access tokens" on Github. Create one here and then use,

!git git remote add origin https://<username>:<access-token><username>/<repo>.git

Keep Colab awake

F12 then Console and type,

function ClickConnect(){

Change to current working directory

By default, the working directory is /content/. One can use below command to change to another place,

%cd /content/data-science-learning

From that point, we are working on /content/data-science-learning.

Upload a file to Colab[ref]

Each user has a "machine" in /content/.

Create a new cell and paste,

from google.colab import files

uploaded = files.upload()

for fn in uploaded.keys():
print('User uploaded file "{name}" with length {length} bytes'.format(
name=fn, length=len(uploaded[fn])))

Run 2 times this cell, at the 2nd time, you can choose your file.

Using Google Drive

Run a cell containing following codes,

from google.colab import drive

and then follow the guide on the screen. In order to access to the drive,

with open('/content/drive/My Drive/foo.txt', 'w') as f:
f.write('Hello Google Drive!')

Clone a repo from Github

!git clone

The cloned folder are stored in /content/. If you wanna pull requests, use,

%cd /content/data-science-learning
!git pull

PyTorch with GPU

To enable GPU backend for your notebook, go to EditNotebook Settings and set Hardware accelerator to GPU.[ref]

Install 7zip reader, GraphViz, PyDot, cartopy

!apt-get -qq install -y libarchive-dev && pip install -q -U libarchive
import libarchive
!apt-get -qq install -y graphviz && pip install -q pydot
import pydot
!apt-get -qq install python-cartopy python3-cartopy
import cartopy

Save as HTML

Jupyter Notebook has an option to 'Download as' HTML (or other) format. Google Colaboratory does not.

  1. Install the nbconvert package.

  2. Save your Colab notebook.

  3. In your terminal:

    jupyter nbconvert --to <output format> <filename.ipynb>
    # jupyter nbconvert --to html mynotebook.ipynb

Other functions

  • Interrupt a long running python process: Runtime > Interrupt execution (Alt + M I).
  • Support Jupyter magic commands, check full list here.

💬 Comments

Support Thi Support Thi