TABLE OF CONTENTS (HIDE)

Python

IDEs and Tools

Python IDEs and Debuggers

Picking a IDE with a powerful graphic debugger is CRITICAL in program development!!!

IDLE (Recommended for Beginners)

Python IDLE (Interactive DeveLopment Environment) is a simple IDE with features such as syntax highlighting, automatic code indentation and visual debugger.

Installing/Launching IDLE

For Ubuntu: To install Python IDLE for Python 3:

# Install IDLE for Python 3
$ sudo apt-get install idle3

# Verify the installation
$ which idle3
/usr/bin/idle3
$ ll /usr/bin/idle*
-rwxr-xr-x 1 root root 92 xxx xx  xxxx /usr/bin/idle3*
-rwxr-xr-x 1 root root 94 xxx xx  xxxx /usr/bin/idle-python3.5*
$ dpkg --status idle3
Version: 3.5.1-3
......

To launch IDLE for Python 3:

$ idle3

For Windows: IDLE is bundled in the installation. Click the START button ⇒ Python ⇒ IDLE (Python GUI). To exit, choose "File" menu ⇒ Exit. IDLE is written in Python and is kept under "Lib\idlelib". You can also use "idle.bat", " idle.py", "idle.pyw" to start the IDLE.
For Mac OS X: Follow this instructions

Using IDLE

In Python IDLE, you can enter Python statements interactively, similar to interactive Python command-line shell. E.g.,

$ idle

Python 3.7.10 (default, Feb 26 2021, 13:06:18) [MSC v.1916 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> print('Hello, world')
Hello, world
>>> print(2 ** 88)
309485009821345068724781056
>>> print(8.01234567890123456789)
8.012345678901234
>>> print((1+2j) * (3-4j))
(-5+10j)
>>>

Writing Python Script in IDLE

To write a Python script under IDLE, choose "File" menu ⇒ "New File". Enter the above script and save as "hello.py" in a directory of your choice (the Python script must be saved with the ".py" extension). To run the script, choose "Run" Menu ⇒ "Run Module". You shall see the outputs on the IDLE console.

Notes:

You can use Alt-P/Alt-N to retrieve the previous/next command in the command history.
You can read the Python Manual via the "Help" menu.

Debugging Python Script in IDLE

In the main IDLE console, choose "Debug" ⇒ "Debugger" to pop out the "Debug Control" and turn on the debugging mode.
In the script's edit window, choose "Run" ⇒ "Run Module" to start debugging the script. Tick "Source".
You can set a breakpoint by right-click the source line, choose "Set Breakpoint" (or "Clear Breakpoint").
From the "Debug Control", You can step over (Over), step into (Step), step out (Out), etc.

Spyder

Spyder (Scientific Python Development Environment) ( https://www.spyder-ide.org/) is an IDE that has similar features to the IDE of Matlab.

"Spyder is a powerful scientific environment written in Python, for Python, and designed by and for scientists, engineers and data analysts. It features a unique combination of the advanced editing, analysis, debugging, and profiling functionality of a comprehensive development tool with the data exploration, interactive execution, deep inspection, and beautiful visualization capabilities of a scientific package. Furthermore, Spyder offers built-in integration with many popular scientific packages, including NumPy, SciPy, Pandas, IPython, QtConsole, Matplotlib, SymPy, and more."

Spyder is bundled in Python's Anaconda distribution, can be installed together with Jupyter Notebook.

Getting Started

Spyder IDE contains an Editor and a IPython Console. You can use the editor to edit and run a Python script, or use the IPython Console to issue a Python statement. The editor support syntax highlighting, content-assit, and many productivity features. You can call out the online help.

Debugging

The editor supports debugging. You can set breakpoints by double-clicking on the left margin, step through or run to the statements, and watch the variables and outputs.

Jupyter Notebook (Recommended for Data Science Projects)

The Jupyter Notebook (http://jupyter.org/) is an open-source web application that allows you to create and share notebooks. According to Wiki, "A notebook is a book or binder of paper pages, often ruled, used for purposes such as recording notes or memoranda, writing, drawing or scrapbooking." A Jupyter Notebook (just like a paper notebook) integrates code and its output into a single document that combines visualizations, narrative text, mathematical equations, and other rich media.

Jupyter Notebook is an easy-to-use data science tool for beginners and educators, in particular for data cleaning and transformation, numerical simulation, statistical modeling, data visualization, machine learning. It supports many programming languages, of which Python is the most commonly-used.

Jupyter Notebook is a successor to the earlier iPython Notebook, started in 2010.

Installation

Follow the instructions at the Jupyter mother site (http://jupyter.org/). Download and install "Anaconda" (Python 3 version), which is the most widely used Python distribution for data science, bundled with Jupyter Notebook and Python data science libraries such as NumPy, pandas and Matplotlib.

Getting Started

References: Read "Jupyter Notebook for Beginners: A Tutorial" https://www.dataquest.io/blog/jupyter-notebook-tutorial/, which has a simple data science example.

Launch "Jupyter Notebook", which is a webapp that opens a new tab in your default web browser, showing the so-called "Jupyter Dashboard", with URL http://localhost:8888/tree listing the default start-up directory.
The default start-up directory is probably not your desired working directory. To change the start-up directory, shutdown the previous app (Press Ctrl-C twice at the console). Launch the Anaconda Prompt from the Start menu (Windows). Change directory to your desired start-up directory and run "jupyter notebook".
To create a new notebook for Python, click "New" and select "Python". A new notebook is opened in a new tab. It is saved as "Untitled.ipynb" (switch back to the Dashboard to see this new file in the directory). Each .ipynb file is a text file that describes the contents of your notebook in JSON format. You can check its contents later after performing some operations.
Jupyter's code is organized in fragments called cells. You can create/remove cells and re-arrange them to create your desired notebook.
Type "print('Hello World!')" into the active cell labeled In[ ], and click "run" (or press Ctrl+Enter). The label changes from In[] to In[1]. The number 1 indicates when the cell was executed on the kernel. Run the cell again, it will change to In[2]. Take note of the printed output below the code.
Create a new cell by choosing menu "Insert" ⇒ "Insert cell below", or click the '+' icon. Enter "print(2 ** 88)" and Run.
Create a new new cell. Enter the following code. Run and take note of the "Out[n]" box, which contains the return value of the code.
```
def say_hello(who):
    return 'Hello, {}!'.format(who)

say_hello('Peter')
```
Create a new cell. Enter the following code.
```
import time
time.sleep(5)
```
Run and observe that In[ ] changes to In[*] (indicating the cell is currently running the sleep()), and then In[n] when completed.
To run a Python script, enter either:
1. %run -i script_name (without the .py)
2. exec(open('script_name.py').read())
Jupyter runs in two modes: edit mode (with green left-border for the active cell and a "pen" icon displayed on top-right corner) and command mode (with blue left-border). You can switch between these 2 modes by clicking inside/outside the active cell.
In "command mode", you can issue these commands (with their respective keyboard shortcuts):
- Keys 'a' and 'b' to insert a cell above and below the active cell.
- Keys 'd'+'d' ('d' twice) to delete the active cell, and 'z' to undo the delete.
- Shift+Up/Down (or Shift+Click the margin) to select multiple cells, and Shift+'m' to merge the cells.
In "edit mode", you can issue these commands (with their respective keyboard shortcuts):
- Ctrl+Shift+'-' to split the active cell at the cursor.
To rename a notebook, you need to go to the Dashboard. Select the notebook, choose "Shutdown" to shutdown the kernel. Select the notebook again, then "Rename".
To shutdown Jupyter Notebook, press Ctrl-C twice at the console.

Cells

A cell contains program code or documentation text. Jupyter Notebook supports two types of cells:

Code Cell: contains program code to be executed in the kernel. The output is displayed below the cell.
Markdown Cell: contains text formatted using Markdown (a lightweight version of HTML - to be explained below). The output is displayed in-place when it is run.

Kernel

A kernel is a computational engine that executes the code contained in a notebook. Each notebook runs a kernel, which maintains the state of a notebook's computations. When you run a code cell, that code is run inside the kernel and the output is returned back to the cell to be displayed. The kernel's state persists over time and between cells. A variable defined or a moduled imported in one cell is available to another cell. Take note that cells are ordered which corresponds to the state of the kernel. From the "Kernel" menu, you can:

Restart: restart the kernel. Clearing all the variables.
Restart & Clear Output: Also clear the output displayed below the code cells.
Restart & Run All: Also run all your codes from top to bottom.
Interrput: If your kernel is stuck on a computation and you wish to stop it.
Shutdown:

Jupyter Notebook support many types of kernel. Beside the various version of Python, it supports over 100 languages including Java, C, R, Julia, MatLab, and SoS multi-language kernel.

Formatting Text with Markdown

Markdown cells are meant for including formatted documentation for your notebook. Markdown is a lightweight (or simplified or subset) of HTML, with very few markup tags for formatting documents, e.g., line beginning with # for <h1>, ## for <h2>, **text** for <strong>, __text__ for <em>, etc.

For example, insert a new cell, select "Markdown" from the pull-down menu, and enter the following:

# This is a level 1 heading or h1 in HTML
## This is a level 2 heading or h2 in HTML
### This is a level 3 heading or h3 in HTML
This is a paragraph or p in HTML.
Mark text as strong via **strong** and __strong__, or em via *emphasis* and _emphasis_.

Separate paragraphs by an empty line.

Below is an unordered list (marked by a beginning star):
* level 1
 * level 2 (indented)

Below is an order list (marked by numbers):
1. Item 1
2. Item 2

Markdown cell supports normal HTML as well.

<h1>Level 1 Heading</h1>
<p>A paragraph with <strong>strong</strong> and <em>emphasis</em>.</p>

To show the actual layout, run the cell or press Ctrl-Enter.

Take note that the Markdown cells support the normal HTML as well, which many people are more familiar and comfortable.

Debugging

Because is hard, use Spyder or IDLE instead, which supports breakpoint and stepping.

Python's Anaconda Distribution (Recommended for any application)

Besides the Official Python Distribution from python.org, there are many Python distributions available. Among them, the open-source Anaconda Python Distribution (https://www.anaconda.com/) is probably the most popular for data science, by bundling:

IDEs: Jupyter Notebook, Spyder
Analytics & Scientific Computing Python Packages: NumPy, Matplotlib, Pandas, SciPy
Machine Learning Python Packages: scikit-learn

It is available on Linux, Windows, and Mac OS X.

Installation

Goto Anaconda's mother site https://www.anaconda.com/. Download "Anaconda Individual Edition". Follow the instructions to install.

Getting Started

The Anaconda Distribution comes with:

Anaconda Prompt: a Command-line shell (CMD, Terminal, Bash Shell) for launching the Python Interactive Interpreter.

(base) xxx> python
Python 3.7.10 (default, Feb 26 2021, 13:06:18) [MSC v.1916 64 bit (AMD64)] :: Anaconda, Inc. on win32
Type "help", "copyright", "credits" or "license" for more information.
>>>

Jupyter Notebook: a web-based IDE suitable for data analytics by combining code, documentation, and visual into notebook.
Spyder: a MatLab-like IDE.
Anaconda Navigator

Explore "Get Started" section, under https://www.anaconda.com/download/.

Conda Package Management System

Anaconda uses the open-source conda package management system for managing packages and their dependencies (install and update), and the environment.

You need to run Conda under the "Anaconda Prompt". To install or update pacakges, you may need the administrator/superuser right:

For Windows, right-click and "run as administrator"
For Linux and Mac OS X, run the commands with "sudo".

Using Conda:

Conda's commands:

# Show help menu
$ conda help
# Show help for command
$ conda command --help

# Clean up the conda installation - remove cache, outdated packages, zip files
$ conda clean --all

# Show info about Anaconda
$ conda info

# Show all packages
$ conda list

# Install a package
$ conda install package-name

# Update a package
$ conda update package-name

To install a package under Anaconda, start a Anaconda Prompt (as administrator or with sudo) and issue:

$ conda install --help

$ conda install package_name
# For examples,
$ conda install numpy
$ conda install pandas

To update a package, start a Anaconda Prompt (as administrator or with sudo) and issue:

$ conda update --help

# Update Anaconda distribution
$ conda update anaconda

# Update a Python package
$ conda update numpy
$ conda update pandas
$ conda update matplotlib
$ conda update scikit-learn
$ conda update jupyter

(For Windows) You can change the start-up directory of "Anaconda Prompt", by right-right on the program ⇒ Properties ⇒ Shortcut ⇒ change the "start-in".

Eclipse PyDev (Recommended for Web Projects)

There are several Eclipse plug-ins for Python - notably PyDev @ http://www.pydev.org/.

Installing PyDev

To install PyDev plug-in for Eclipse: Launch Eclipse ⇒ Help ⇒ Install New Software ⇒ In "Work With", enter "http://pydev.org/updates" ⇒ Add ⇒ Select "PyDev".

Caution: When you are prompted for PyDev certificate, you MUST manually select (check) the certificate, before pressing OK. Otherwise, the installation aborts without any warning.

Configuring PyDev

After the installation, you need to configure Python Interpreter: Launch Eclipse ⇒ Window ⇒ Preference ⇒ PyDev ⇒ Expand "Interpreters" node ⇒ Python Interpreter ⇒ New to configure your Python Interpreter. For examples, in Ubuntu, /usr/bin/python for Python 2.7 or /usr/bin/python3 for Python 3.5; in Windows, python.exe. You can configure many interpreters and choose the desire one for each of your project.

Writing Python Script using PyDev

To start a new Python project: Launch Eclipse ⇒ File ⇒ New ⇒ Project ⇒ PyDev ⇒ PyDev Project ⇒ Enter "Project name", e.g., "HelloPython" ⇒ Choose the "Interpreter" (configured earlier) and "Grammar Version" ⇒ "Finish".

To write a Python script: Right-click on the project ⇒ New ⇒ PyDev Module ⇒ Leave the "Package" empty ⇒ In "Name", enter "hello" ⇒ Key in the following script:

"""First Python module to say Hello"""
def sayHello(name):
    return "Hello, " + name

print(sayHello('Peter'))

Running Python Script

To run the script, right-click on the script ⇒ Run As ⇒ Python Run.

Debugging Using PyDev

To debug a script, set a breakpoint by double-clicking on the left-margin of the the desired line ⇒ right-click on the script ⇒ Debug as ⇒ Python run.

You can then trace the execution via "Step Over (current statement) (F5)", "Step Into (function) (F6)", "Step Out (function) (F7)", "Resume (to next breakpoint) (F8)", "Terminate", and etc.

Python Tools

python-dev - Python development tools

Many of the Python packages require python-dev (Python development tools) to install and build.

(Ubuntu) Installing python-dev

# Python 3
$ sudo apt-get install python3-dev
......
The following NEW packages will be installed:
  libpython3-dev libpython3.5-dev python3-dev python3.5-dev
......

$ dpkg --status python3-dev
Version: 3.5.1-3
......

pip - Python Package Manager

References

pip https://pypi.python.org/pypi/pip.
pip documentation https://pip.readthedocs.org/en/stable.

pip (Python Package Manager) is used for downloading, installing/un-installing, and managing Python packages. It is a replacement for an earlier tool called easy_install.

(Ubuntu) Installing pip

# (For Python 3) Install 'pip3'
$ sudo apt-get install python3-pip
......
$ which pip3
/usr/bin/pip3
$ ll /usr/bin/pip*
-rwxr-xr-x 1 root root 307 Nov 18 20:28 /usr/bin/pip3*
$ pip3 --version
pip 18.1 from /usr/local/lib/python3.6/dist-packages/pip (python 3.6)

Upgrading pip

The apt-get install may not install the latest pip version. You could upgrade pip using pip itself, as follows:

# Python 3
$ sudo pip3 install --upgrade pip
$ pip3 --version
......

Note: If you use 'sudo' in the above commands, the upgrades will be installed in /usr/lib, which will be available system-wide and for all users. On the other hand, if you did not use 'sudo', the upgrades were installed in /home/username/.local/lib, which is meant for the current user.

Using pip

Installing a package:

# Syntax: pip install <package-name>
$ pip install virtualenv
Downloading/unpacking virtualenv
  Downloading virtualenv-13.1.2-py2.py3-none-any.whl (1.7MB): 1.7MB downloaded
Installing collected packages: virtualenv
Successfully installed virtualenv
Cleaning up...

Showing what files were installed and where:

# Syntax: pip show --files <package-name>
$ pip show --files virtualenv
Name: virtualenv
Version: 15.1.0
Location: /usr/local/lib/python2.7/dist-packages
......

Installing a package of a specific or minimum version:

# Syntax for installing specific version: pip install <package-name>==<version>
# Syntax for installing minimum version: pip install <package-name>>=<version>
$ pip install virtualenv==13.1.1  # specific version
$ pip install virtualenv>=13.1.1  # minimum version

Installing a package from a URL:

Syntax: pip install <url>
$ pip install https://github.com/pypa/virtualenv/archive/develop.tar.gz

Listing all/out-dated packages:

# List all installed packages
$ pip list

# List out-dated packages
$ pip list --outdated
requests (Current: 2.2.1 Latest: 2.8.1)
......

Upgrading a package:

# Syntax: pip install --upgrade <package-name>
$ pip install --upgrade requests

Un-Installing a package:

# Syntax: pip uninstall <package-name>
$ pip uninstall requests

Search for a package:

# Syntax: pip search <package-name>
$ pip search requests

Installing package for a user:

# Syntax: pip install --user <package-name>
$ pip install --user requests
   # The package will be installed in $HOME/.local/lib/python2.7/site-packages

REFERENCES & RESOURCES

The Python's mother site www.python.org; "The Python Documentation" https://www.python.org/doc/; "The Python Tutorial" https://docs.python.org/tutorial/; "The Python Language Reference" https://docs.python.org/reference/.

Latest version tested: Python (Ubuntu, Windows, Cygwin, Jupyter Notebook) 3.7.1 and 2.7.14
Last modified: December, 2018

TABLE OF CONTENTS (HIDE)

Python

IDEs and Tools

Python IDEs and Debuggers

IDLE (Recommended for Beginners)

Installing/Launching IDLE

Using IDLE

Writing Python Script in IDLE

Debugging Python Script in IDLE

Spyder

Getting Started

Debugging

Jupyter Notebook (Recommended for Data Science Projects)

Installation

Getting Started

Cells

Kernel

Formatting Text with Markdown

Debugging

Python's Anaconda Distribution (Recommended for any application)

Installation

Getting Started

Conda Package Management System

Eclipse PyDev (Recommended for Web Projects)

Installing PyDev

Configuring PyDev

Writing Python Script using PyDev

Running Python Script

Debugging Using PyDev

PyCharm (Recommended for Web Projects)

Visual Studio Code

Atom

Sublime Text

Notepad++

Python Tools

python-dev - Python development tools

(Ubuntu) Installing python-dev

pip - Python Package Manager

References

(Ubuntu) Installing pip

Upgrading pip

Using pip