Python Installation

23-02-2021 / Edit on Github

Windows #

Update 11/Sep/20: Install python on WSL2 using Miniconda.

Other options (directly install)

Download and install Anaconda or smaller version Miniconda.

# default installed dir
C:\ProgramData\Anaconda3

Add to System Environment Variables:

C:\ProgramData\Anaconda3
C:\ProgramData\Anaconda3\Scripts

If you don't wanna use Anaconda and install it by yourself, add this:

C:\Users\<user>\AppData\Roaming\Python\Python36\Scripts

(You can find C:\...\Roaming by typing %appdata% in the Windows Explorer's navigation bar)

App to run: cmder (use this setting file).

MacOS #

If you use Apple M1 (updated on 21/02/2021)?
Cannot use docker with AI things! Pytorch is not available for M1!

πŸ‘‰ Instructions to install TensorFlow in a Conda Environment

conda init # after installing
# restart terminal after installing
# check if ok
which python # should return "/Users/thi/miniforge3/bin/python"

# manually install package
conda install <pkg name>
conda install -y <pkg> # auto accept

If you use conda (miniforge3), use the instructions in previous link to install tensorflow (not about the link to "newest" version of tensorflow-macos). If you wanna use python on your mac (using venv), check below link.

πŸ‘‰ Mac-optimized TensorFlow and TensorFlow Addons

Remark: After installing, check the version of tensorflow-macos (not tensorflow)

pip show tensorflow-macos

Remark: If you wanna install some package, use conda install -y <pkg>, then check its version by pip show <pkg>.

πŸ‘‰ Using my personal requirement file.

conda install --file requirement_m1.txt

By default, Python 2 is already installed on MacOS, you can check this by

python --version
# to be sure, check if python3 is installed?
python3 --version

Linux (Ubuntu) #

Python is already installed on Ubuntu. You would like to install Anaconda, download and install it.

Wanna install Miniconda instead? πŸ‘‰ Download .sh file and install inside Linux environement (including WSL2).

Miniconda or Conda? (ref)

Note that Conda is the package manager (e.g. conda list displays all installed packages in the environment), whereas Anaconda and Miniconda are distributions.

Choose Anaconda if you:

  • Are new to conda or Python
  • Like the convenience of having Python and over 1500 scientific packages automatically installed at once
  • Have the time and disk space (a few minutes and 3 GB), and/or
  • Don’t want to install each of the packages you want to use individually.

Choose Miniconda if you:

  • Do not mind installing each of the packages you want to use individually.
  • Do not have time or disk space to install over 1500 packages at once, and/or
  • Just want fast access to Python and the conda commands, and wish to sort out the other programs later.
Add conda to $PATH
# ADD CONDA TO $PATH

nano ~/.profile
# find where conda is installed and then
export PATH=/home/<user>/anaconda3/bin:$PATH
source ~/.profile

# check
which python
# should return: /home/<user>/anaconda3/bin/python

# check version
conda --version
Make a right version
alias python=python3
alias pip=pip3
# for ubuntu >=20.04
sudo apt install python-is-python3
# prevent Python 2 from being installed as a dependency of something
sudo apt-mark hold python2 python2-minimal python2.7 python2.7-minimal libpython2-stdlib libpython2.7-minimal libpython2.7-stdlib

Jupyer Notebook #

πŸ‘‰ Note: Jupyter notebook.

Anaconda contains JN in it, no need to install it again. cd to the folder you wanna work on and run

# RUN (after installing Anaconda)
python -m notebook
# If `ImportError: DLL load failed`
active base # active env "base" in anaconda
jupyter notebook

The -m option allows you to execute a module or package as a script[ref].

# If `import _ssl`, `ImportError`
set CONDA_DLL_SEARCH_MODIFICATION_ENABLE=1
python -m notebook

Check GPU #

πŸ‘‰ Read more on note of pytorch.

# with pytorch
import torch
print('cuda is available? ', torch.cuda.is_available())
print('device_count: ', torch.cuda.device_count())
print('current device: ', torch.cuda.current_device())
print('device name: ', torch.cuda.get_device_name(0))
# with tensorflow
import tensorflow as tf
print("Num GPUs Available: ", len(tf.config.experimental.list_physical_devices('GPU')))

pip #

Update pip #

# Check pip version
pip -V
# update pip
easy_install -U pip
Problem?
  • If you meet AttributeError: 'NoneType' object has no attribute 'bytes' when updating pip, check the version and make sure that there is only 1 pip on your computer and then use easy_install -U pip (don't forget to activate)
  • If there is a problem with python -m pip install --upgrade pip, use easy_install!

Install packages with pip #

Install pip (It's actually installed with Anaconda). If you wanna upgrade it to the latest version:

python -m pip install --user --upgrade pip # install for current user only
python -m pip install --upgrade pip # need to run cmder as administrator
AttributeError: 'NoneType' object

First, activate <env> and then using easy_install -U pip. You can check the version of pip by pip -V.

# LIST ALL INSTALLED PACKAGES
pip freeze
# INSTALL A PACKAGE
pip install <package> # admin <-- SHOULDN'T!!!
pip install --user <package> # current user only
# REMOVE
pip uninstall <package>
pip uninstall --user <package>
# CHECK VERSION OF A PACKAGE
pip show <package>
# version <=
pip3 install -U "pillow<7"
# Install a package from a git repository
pip install git+https://github.com/TimeSynth/TimeSynth.git

If install packages with pip, they are installed in which environment of conda? Where pip is executed from.

which python
which pip

conda info --envs
# or
# conda env list
/c/ProgramData/Anaconda3/python
/c/ProgramData/Anaconda3/Scripts/pip

# conda environments:
base                  *  C:\ProgramData\Anaconda3
fastai                   C:\Users\thi\.conda\envs\fastai

Install packages with requirement file,

pip install -r requirements.txt

An example of requirement file,

geopandas==0.4.1
grpcio==1.27.1
grpcio-tools==1.27.1
h5py==2.10.0
isodate==0.6.0
PyYAML==5.3.1

pip vs conda? #

Differences:[ref]

  • pip installs python packages in any environment.
  • conda installs any package in conda environments.

Which one to be used?[ref]

  • If you installed Python using Anaconda or Miniconda, then use conda to install Python packages. If conda tells you the package you want doesn't exist, then use pip (or try conda-forge, which has more packages available than the default conda channel).
  • If you installed Python any other way (from source, using pyenv, virtualenv, etc.), then use pip to install Python packages

Python virtual environnement #

Main guide is here.

sudo apt-get install python3-venv

# cd to <DIR> where python venv stored
python3 -m venv <DIR>

# activate
tutorial-env\Scripts\activate.bat # windows
source <DIR>/bin/activate # linux

# deactivate
deactivate

To detele, just remove the corresponding folder, i.e., <DIR>.

Conda #

Install / Update conda #

πŸ‘‰ Read more on this section.

# INSTALL CONDA BY PIP (without Anaconda)
pip install conda
# UPDATE CONDA
conda --version # check version
conda update -n base -c defaults conda

If there is an error TypeError: LoadLibrary() argument 1 must be str, not None at the end of the log, try to activate the environment base before running above line.

activate base # on Windows
source activate base # on MacOS

Install packages with conda #

# INSTALL
activate <env> # you need to activate an environment first
conda install <package> # install for <env> only
# UPDATE
acctivate <env> # choose an env first
conda update <package> # ud package in that env
# LIST ALL INSTALLED PACKAGES
conda list
# check version of a package
pip show <pkg>
# Update packages listed in an env file to current env,
conda env update -n <env> -f /path/to/<file>.yml
# example of yml file
name: stats
dependencies:
- python=3.6
- geopandas==0.4.1
# install packages with requirements.txt
conda install --file requirements.txt

Environment #

πŸ‘‰ Check an official doc or this useful post.

Create a new environment with python version 3.7:

conda create -n <env-name> python=3.7 anaconda
# created in /home/thi/miniconda3/envs/<env-name>/
source activate <env-name> # activate this env
# The same python version with current shell's Python interpreter
conda create -n <env-name> python
# with addtional packages (python will be automatically installed)
conda create -n <env-name> <package1> <package2>
# with version
conda create -n <env-name> <package1>=1.16 <package2>
# in different directory
conda create --prefix /path/to/<env-name>
# create from file <file>.yml
conda env create -n <env> -f /path/to/<file>.yml
# Clone from another env
conda create --name <cloned-env> --clone <env>

Most of below commands are assumed to be run in an environment named env which is already activated. If you don't activate any environment before, use an alternative instead. For example,

conda update pandas             # <env> activated
conda update -n <env> pandas # <env> isn't activated
conda update -p /path/to/<env> # <env> isn't in the default directory of conda
conda env export -f <file>.yml                       # <env> activated & current folder
conda env export -n <env> -f /path/to/<file>.yml # <env> isn't activated & different folder
conda update -p /path/to/<env> -f /path/to/<file>.yml # <env> isn't in the default directory of conda & different folder
# Activate an env
activate <env> # windows
source activate <env> # linux / macos
# DEACTIVE AN ENV
conda deactivate # Linux
deactivate # Windows
source deactivate # MacOS
# REMOVE AN ENV
conda remove -n <env> --all
# SHOW LIST OF CURRENT ENV
conda info --envs
# or
conda env list
# EXPORT TO A ENV FILE
conda env export -f <file>.yml
# Update packages listed in an env file to current env,
conda env update -n <env> -f /path/to/<file>.yml --prune
Show activated env in zsh
# run this first in your current shell
conda init zsh
# other options than zsh: bash, powershell, ...
source ~/.zshrc

If you wanna hide (base) in prompt,

conda config --set changeps1 false
# don't forget to reset the terminal

Kernel 2 & 3 for Jupyter Notebook #

Check if nb_conda_kernels is installed by conda list. If not, install it by:

conda install nb_conda_kernels

If you are using Python 2 and you wanna separate Python 3,

conda create -n py37 python=3.7 ipykernel # "py37" is a custom name

If you are using Python 3 and you wanna separate Python 2,

conda create -n py27 python=2.7 ipykernel # "py37" is a custom name

Restart the Jupyter Notebook, the list of kernels is available under New.

Conda Revisions #

# Check revisions
conda list --revisions
# Go back to revision `1`,
conda install --revision 1

Error? #

# UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 975: ordinal not
# solution: instead of
pip3 install sesd
# use
LC_ALL=C.UTF-8 pip3 install sesd
# conda: The following packages are not available from current channels:
# Solution 1: One can use pip in this case (the same env with conda)
# Solution 2:
conda install -c anaconda <package>
# The following required packages can not be built: * freetype (from matplotlib)
# try to use conda to install matplotlib
conda install matplotlib
# it actually install the same thing as pip does on the same env
# dtaidistance: C-library is not available
pip install -vvv --upgrade --force-reinstall dtaidistance
# zsh: command not found: conda
# Make sure your installation folder is already
# in the $PATH
export PATH="/home/thi/miniconda3/bin:$PATH"