Functions

4. Functions#

4.1. Overview#

Functions are an extremely useful construct provided by almost all programming.

We have already met several functions, such as the built-in print() function.

In this lecture we’ll

treat functions systematically and cover syntax and use-cases, and
learn to do is build our own user-defined functions.

4.2. Function Basics#

A function is a named section of a program that implements a specific task.

Many functions exist already and we can use them as is.

First we review these functions and then discuss how we can build our own.

Python functions are very flexible. In particular

Any number of functions can be defined in a given file.
Functions can be (and often are) defined inside other functions.
Any object can be passed to a function as an argument, including other functions.
A function can return any kind of object, including functions.

User-defined functions are important for improving the clarity of your code by

separating different strands of logic
facilitating code reuse

(Writing the same thing twice is almost always a bad idea)

4.2.1. Built-In Functions#

Python has a number of built-in functions that are available without import.

We have already met some

max(19, 20)

print('foobar')

foobar

str(22)

'22'

type(22)

int

The full list of Python built-ins is here.

4.2.2. Third Party Functions#

If the built-in functions don’t cover what we need, we either need to import functions or create our own.

Example: tests whether a given year is a leap year inside the module calendar:

import calendar
calendar.isleap(2024)

True

4.3. Defining Functions#

In Python, you define a function via the keyword def followed by the function name, the parameter list, the doc-string and the function body. Inside the function body, you can use a return statement to return a value to the caller.

The syntax is:

def function_name(arg1, arg2, ...):
    """Function doc-string"""    # Can be retrieved via function_name.__doc__
    body_block
    return return-value

Here’s a very simple Python function, that implements the mathematical function \(f(x) = 2 x + 1\)

def f(x):
    return 2 * x + 1

Now that we’ve defined this function, let’s call it and check whether it does what we expect:

f(1)   

f(10)

Another example of function that computes the absolute value of a given number: (Such a function already exists as a built-in, but let’s write our own for the example.)

def new_abs_function(x):
    if x < 0:
        abs_value = -x
    else:
        abs_value = x
    return abs_value

Let’s review the syntax here.

def is a Python keyword used to start function definitions.
def new_abs_function(x): indicates that the function is called new_abs_function and that it has a single argument x.
The indented code is a code block called the function body.
The return keyword indicates that abs_value is the object that should be returned to the calling code.

This whole function definition is read by the Python interpreter and stored in memory.

Let’s call it to check that it works:

print(new_abs_function(3))
print(new_abs_function(-3))

3
3

Note that a function can have arbitrarily many return statements (including zero).

Execution of the function terminates when the first return is hit, allowing code like the following example:

def f(x):
    if x < 0:
        return 'negative'
    return 'nonnegative'

(Writing functions with multiple return statements is typically discouraged, as it can make logic hard to follow.)

Functions without a return statement automatically return the special Python object None.

4.3.1. The pass statement#

The pass statement does nothing. It is sometimes needed as a dummy statement placeholder to ensure correct syntax, e.g.,

def my_fun():
    pass      # To be defined later, but syntax error if empty

4.3.2. Examples#

>>> def my_square(x):
        """Return the square of the given number"""
        return x * x

# Invoke the function defined earlier
>>> my_square(8)
64
>>> my_square(1.8)
3.24
>>> my_square('hello')
TypeError: can't multiply sequence by non-int of type 'str'
>>> my_square
<function my_square at 0x7fa57ec54bf8>
>>> type(my_square)
<class 'function'>
>>> my_square.__doc__  # Show function doc-string
'Return the square of the given number'
>>> help(my_square)    # Show documentation
my_square(x)
    Return the square of the given number
>>> dir(my_square)     # Show attribute list
[......]

Take note that you need to define the function before using it, because Python is interpretative.

# Define a function (need to define before using the function)
def fibon(n):
    """Print the first n Fibonacci numbers, where f(n)=f(n-1)+f(n-2) and f(1)=f(2)=1"""
    a, b = 1, 1   # pair-wise assignment
    for count in range(n): # count = 0, 1, 2, ..., n-1
        print(a, end=' ')  # print a space instead of a default newline at the end
        a, b = b, a+b
    print()   # print a newline

# Invoke the function
fibon(20)

1 1 2 3 5 8 13 21 34 55 89 144 233 377 610 987 1597 2584 4181 6765

Example: Function doc-string

def my_cube(x):
    """
    (number) -> (number)
    Return the cube of the given number.

    :param x: number
    :return: number

    Examples (can be used by doctest for unit testing):
    >>> my_cube(5)
    125
    >>> my_cube(-5)
    -125
    >>> my_cube(0)
    0
    """
    return x*x*x

# Test the function
print(my_cube(8))    # 512
print(my_cube(-8))   # -512
print(my_cube(0))    # 0

512
-512
0

This example elaborates on the function’s doc-string:

The first line “(number) -> (number)” specifies the type of the argument and return value. Python does not perform type check on function, and this line merely serves as documentation.
The second line gives a description.

Example of a function that return two values:

def derivatives(f,x,h=0.0001): # h has a default value
    """function that computes the first two derivatives
    of f (x) by finite differences"""
    df =(f(x+h) - f(x-h))/(2.0*h)
    ddf =(f(x+h) - 2.0*f(x) + f(x-h))/h**2
    return df,ddf

# Test the function
from math import atan
df,ddf = derivatives(atan,0.5) # Uses default value of h
print('Firstderivative=',df)
print('Secondderivative=',ddf)

Firstderivative= 0.7999999995730867
Secondderivative= -0.6399999918915711

Example of a function that operate over a list:

def squares(a):
    """a list is passed to a function where it is modified
    """
    for i in range(len(a)):
        a[i] = a[i]**2

# Test the function
a = [1, 2, 3, 4]
squares(a)
print(a) # 'a' now contains 'a**2'

[1, 4, 9, 16]

4.3.3. Function Parameters and Arguments#

There is a subtle difference between function parameters and arguments. Function parameters are named variables listed in a function definition. Parameter variables are used to import arguments into functions. For example,

>>> def foo(parameter):
        print(parameter)

>>> argument = 'hello'
>>> foo(argument)
hello

Note the differences:

Function parameters are the names listed in the function’s definition.
Function arguments are the real values passed to the function.
Parameters are initialized to the values of the arguments supplied.

4.3.3.1. Passing Arguments by-Value vs. by-Reference#

In Python:

Immutable arguments (such as integers, floats, strings and tuples) are passed by value. That is, a copy is cloned and passed into the function. The original cannot be modified inside the function. Pass-by-value ensures that immutable arguments cannot be modified inside the function.
Mutable arguments (such as lists, dictionaries, sets and instances of classes) are passed by reference. That is, the pointer (or reference) of the object is passed into the function, and the object’s contents can be modified inside the function via the pointer.

For examples,

# Immutable arguments are passed-by-value
>>> def increment_int(num):
        num += 1
>>> num = 5
>>> increment_int(num)
>>> num
5     # no change

# Mutable arguments are passed-by-reference
>>> def increment_list(lst):
        for i in range(len(lst)):
            lst[i] += lst[i]
>>> lst = [1, 2, 3, 4, 5]
>>> increment_list(lst)
>>> lst
[2, 4, 6, 8, 10]   # changed

4.3.3.2. Function Parameters with Default Values#

You can assign a default value to the trailing function parameters. These trailing parameters having default values are optional during invocation. For examples,

>>> def foo(n1, n2 = 4, n3 = 5):  # n1 is required, n2 and n3 having default are optional
        """Return the sum of all the arguments"""
        return n1 + n2 + n3

>>> print(foo(1, 2, 3))
6
>>> print(foo(1, 2))    # n3 defaults
8
>>> print(foo(1))       # n2 and n3 default
10
>>> print(foo())
TypeError: foo() takes at least 1 argument (0 given)
>>> print(foo(1, 2, 3, 4))
TypeError: foo() takes at most 3 arguments (4 given)

Another example,

def greet(name):
    return 'hello, ' + name
    
greet('Peter')
#'hello, Peter'

'hello, Peter'

Instead of hard-coding the ‘hello, ‘, it is more flexible to use a parameter with a default value, as follows:

def greet(name, prefix='hello'):  # 'name' is required, 'prefix' is optional
    return prefix + ', ' + name
    
greet('Peter')                    #'hello, Peter'
greet('Peter', 'hi')              #'hi, Peter'
greet('Peter', prefix='hi')       #'hi, Peter'
greet(name='Peter', prefix='hi')  #'hi, Peter'

'hi, Peter'

4.3.3.3. Positional and Keyword Arguments#

Python functions support both positional and keyword (or named) arguments.

Normally, Python passes the arguments by position from left to right, i.e., positional. Python also allows you to pass arguments by keyword (or name) in the form of kwarg=value. For example,

def foo(n1, n2 = 4, n3 = 5):
    """Return the sum of all the arguments"""
    return n1 + n2 + n3

print(foo(n2 = 2, n1 = 1, n3 = 3)) # Keyword arguments need not follow their positional order
print(foo(n2 = 2, n1 = 1))         # n3 defaults
print(foo(n1 = 1))                 # n2 and n3 default
print(foo(1, n3 = 3))              # n2 default. Place positional arguments before keyword arguments
print(foo(n2 = 2))                 # TypeError, n1 missing

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
Cell In[19], line 9
      7 print(foo(n1 = 1))                 # n2 and n3 default
      8 print(foo(1, n3 = 3))              # n2 default. Place positional arguments before keyword arguments
----> 9 print(foo(n2 = 2))                 # TypeError, n1 missing

TypeError: foo() missing 1 required positional argument: 'n1'

You can also mix the positional arguments and keyword arguments, but you need to place the positional arguments first, as shown in the above examples.

4.3.3.4. Variable Number of Positional Arguments (*args)#

Python supports variable (arbitrary) number of arguments. In the function definition, you can use * to pack all the remaining positional arguments into a tuple. For example,

def foo(a, *args):  # Accept one positional argument, followed by arbitrary number of arguments pack into tuple
    """Return the sum of all the arguments (one or more)"""
    sum = a
    print('args is:', args)  # for testing
    for item in args:  # args is a tuple
        sum += item
    return sum

print(foo(1))           #args is: ()
print(foo(1, 2))        #args is: (2,)
print(foo(1, 2, 3))     #args is: (2, 3)
print(foo(1, 2, 3, 4))  #args is: (2, 3, 4)

Python supports placing *args in the middle of the parameter list. However, all the arguments after *args must be passed by keyword to avoid ambiguity. For example

def foo(a, *args, b):
    """Return the sum of all the arguments"""
    sum = a
    print('args is:', args)
    for item in args:
        sum += item
    sum += b
    return sum

print(foo(1, 2, 3, 4))      #TypeError: my_sum() missing 1 required keyword-only argument: 'b'
print(foo(1, 2, 3, 4, b=5)) #args is: (2, 3, 4)

4.3.3.5. Unpacking List/Tuple into Positional Arguments (lst, tuple)#

In the reverse situation when the arguments are already in a list/tuple, you can also use * to unpack the list/tuple as separate positional arguments. For example,

>>> def foo(a, b, c):
        """Return the sum of all the arguments"""
        return a+b+c

>>> lst = [11, 22, 33]
>>> foo(lst)
TypeError: foo() missing 2 required positional arguments: 'b' and 'c'
>>> foo(*lst)   # Unpack the list into separate positional arguments
66

>>> lst = [44, 55]
>>> my_sum(*lst)
TypeError: my_sum() missing 1 required positional argument: 'c'

>>> def foo(*args):  # Variable number of positional arguments
        sum = 0
        for item in args: sum += item
        return sum
>>> foo(11, 22, 33)  # positional arguments
66
>>> lst = [44, 55, 66]
>>> foo(*lst)   # Unpack the list into positional arguments
165
>>> tup = (7, 8, 9, 10)
>>> foo(*tup)   # Unpack the tuple into positional arguments
34

4.3.3.6. Variable Number of Keyword Arguments (**kwargs)#

For keyword parameters, you can use ** to pack them into a dictionary. For example,

>>> def my_print_kwargs(msg, **kwargs):  # Accept variable number of keyword arguments, pack into dictionary
        print(msg)
        for key, value in kwargs.items():  # kwargs is a dictionary
            print('{}: {}'.format(key, value))

>>> my_print_kwargs('hello', name='Peter', age=24)
hello
name: Peter
age: 24

4.3.3.7. Unpacking Dictionary into Keyword Arguments (**dict)#

Similarly, you can also use ** to unpack a dictionary into individual keyword arguments

>>> def my_print_kwargs(msg, **kwargs):  # Accept variable number of keyword arguments, pack into dictionary
        print(msg)
        for key, value in kwargs.items():  # kwargs is a dictionary
            print('{}: {}'.format(key, value))

>>> dict = {'k1':'v1', 'k2':'v2', 'k3':'v3'}
>>> my_print_kwargs('hello', **dict)  # Use ** to unpack dictionary into separate keyword arguments k1=v1, k2=v2
hello
k1: v1
k2: v2
k3: v3

4.3.3.8. Using both *args and **kwargs#

You can use both *args and **kwargs in your function definition. Place *args before **kwargs. For example,

>>> def my_print_all_args(*args, **kwargs):   # Place *args before **kwargs
        for item in args:  # args is a tuple
            print(item)
        for key, value in kwargs.items():  # kwargs is a dictionary
            print('%s: %s' % (key, value))

>>> my_print_all_args('a', 'b', 'c', name='Peter', age=24)
a
b
c
name: Peter
age: 24

>>> lst = [1, 2, 3]
>>> dict = {'name': 'peter'}
>>> my_print_all_args(*lst, **dict)  # Unpack
1
2
3
name: peter

4.3.4. Function Overloading#

Python does NOT support Function Overloading because Python does not have data type in the function parameters. If a function is defined twice, the new version will replace the older one with the same name.

>>> def foo(a, b):
        print('version 1')

>>> def foo(a, b, c):
        print('version 2')

>>> foo(1, 2, 3)
version 2
>>> foo(1, 2)
TypeError: foo() missing 1 required positional argument: 'c'

You can archive function overloading by defining the datatype as a parameter (e.g., datatype=’int’, datatype=’str’), *args or param=None (parameter with default of None).

4.3.5. Function Return Values#

You can return multiple values from a Python function, e.g.,

>>> def foo():
       return 1, 'a', 'hello'  # Return a tuple

>>> x, y, z = foo()  # Chain assignment
>>> z
'hello'
>>> foo()  # Return a tuple
(1, 'a', 'hello')

4.3.6. Types Hints via Function Annotations#

From Python 3.5, you can provide type hints via function annotations in the form of:

def say_hello(name:str) -> str:  # Type hints for parameter and return value
    return 'hello, ' + name

say_hello('Peter')

The type hints annotations are ignored, and merely serves as documentation. But there are external library that can perform the type check.

The Python runtime does not enforce function and variable type annotations. They can be used by third party tools such as type checkers, IDEs, linters, etc.

Read: PEP 484 – Type Hints.

4.4. Modules, Import-Statement and Packages#

4.4.1. Modules#

A Python module is a file containing Python codes - including statements, variables, functions and classes. It shall be saved with file extension of “.py”. The module name is the filename, i.e., a module shall be saved as “<module_name>.py”.

By convention, modules names shall be short and all-lowercase (optionally joined with underscores if it improves readability).

A module typically begins with a triple-double-quoted documentation string (doc-string) (which is available in <module_name>.doc), followed by variables, functions and class definitions.

Example: The greet Module

Create a module called greet and save as “greet.py” as follows:

"""
greet
-----
This module contains the greeting message 'msg' and greeting function 'greet()'.
"""

msg = 'Hello'      # Global Variable
 
def greet(name):   # Function
    """Do greeting"""
    print('{}, {}'.format(msg, name))

This greet module defines a variable msg and a function greet().

4.4.2. The `import` statement#

To use an external module in your script, use the import statement:

import <module_name>                          # import one module
import <module_name_1>, <module_name_2>, ...  # import many modules, separated by commas
import <module_name> as <name>                # To reference the imported module as <name>

Once imported, you can reference the module’s attributes as <module_name>.<attribute_name>. You can use the import-as to assign an alternate name to avoid module name conflict.

For example, to use the greet module created earlier:

$ cd /path/to/target-module
$ python3
>>> import greet
>>> greet.greet('Peter')  # <module_name>.<function_name>
Hello, Peter
>>> print(greet.msg)      # <module_name>.<var_name>
Hello

>>> greet.__doc__         # module's doc-string
'greet.py: the greet module with attributes msg and greet()'
>>> greet.__name__        # module's name
'greet'

>>> dir(greet)            # List all attributes defined in the module
['__built-ins__', '__cached__', '__doc__', '__file__', '__loader__', '__name__',
 '__package__', '__spec__', 'greet', 'msg']

>>> help(greet)           # Show module's name, functions, data, ...
Help on module greet:
NAME
    greet
DESCRIPTION
    ...doc-string...
FUNCTIONS
    greet(name)
DATA
    msg = 'Hello'
FILE
    /path/to/greet.py

>>> import greet as salute  # Reference the 'greet' module as 'salute'
>>> salute.greet('Paul')
Hello, Paul

The import statements should be grouped in this order:

Python’s standard library
Third party libraries
Local application libraries

4.4.3. The `from-import` Statement#

The syntax is:

from <module_name> import <attr_name>              # import one attribute
from <module_name> import <attr_name_1>, <attr_name_2>, ...   # import selected attributes
from <module_name> import *                        # import ALL attributes (NOT recommended)
from <module_name> import <attr_name> as <name>    # import attribute as the given name

With the from-import statement, you can reference the imported attributes using <attr_name> directly, without qualifying with the <module_name>.

For example,

>>> from greet import greet, msg as message
>>> greet('Peter')  # Reference without the 'module_name'
Hello, Peter
>>> message
'Hello'
>>> msg
NameError: name 'msg' is not defined

4.4.4. import vs. from-import#

The from-import statement actually loads the entire module (like import statement); and NOT just the imported attributes. But it exposes ONLY the imported attributes to the namespace. Furthermore, you can reference them directly without qualifying with the module name.

For example, let create the following module called imtest.py for testing import vs. from-import:

"""
imtest

For testing import vs. from-import “”” x = 1 y = 2

print(‘x is: {}’.format(x))

def foo(): print(‘y is: {}’.format(y))

def bar(): foo()

Let's try out import:

```python
$ python3
>>> import imtest
x is: 1
>>> imtest.y  # All attributes are available, qualifying by the module name
2
>>> imtest.bar()
y is: 2

Now, try the from-import and note that the entire module is loaded, just like the import statement.

$ python3
>>> from imtest import x, bar
x is: 1
>>> x  # Can reference directly, without qualifying with the module name
1
>>> bar()
y is: 2
>>> foo()  # Only the imported attributes are available
NameError: name 'foo' is not defined

4.4.5. Conditional Import#

Python supports conditional import too. For example,

if ....:   # E.g., check the version number
   import xxx
else:
   import yyy

4.4.6. `sys.path` and `PYTHONPATH/PATH` environment variables#

The environment variable PATH shall include the path to Python Interpreter “python3”.

The Python module search path is maintained in a Python variable path of the sys module, i.e., sys.path. The sys.path is initialized from the environment variable PYTHONPATH, plus an installation-dependent default. The environment variable PYTHONPATH is empty by default.

For example,

>>> import sys
>>> sys.path
['', '/usr/lib/python3.5', '/usr/local/lib/python3.5/dist-packages', 
 '/usr/lib/python3.5/dist-packages', ...]

sys.path default includes the current working directory (denoted by an empty string), the standard Python directories, plus the extension directories in dist-packages.

The imported modules must be available in one of the sys.path entries.

>>> import some_mod
ImportError: No module named 'some_mod'
>>> some_mod.var
NameError: name 'some_mod' is not defined

To show the PATH and PYTHONPATH environment variables, use one of these commands:

# Windows
> echo %PATH%
> set PATH
> PATH
> echo %PYTHONPATH%
> set PYTHONPATH

# macOS / Ubuntu
$ echo $PATH
$ printenv PATH
$ echo $PYTHONPATH
$ printenv PYTHONPATH

4.4.7. Template for Python Standalone Module#

The following is a template of standalone module for performing a specific task:

"""
<package_name>.<module_name>
----------------------------
A description which can be long and explain the complete
functionality of this module even with indented code examples.
Class/Function however should not be documented here.

:author: <author-name>
:version: x.y.z (version.release.modification)
:copyright: ......
:license: ......
"""
import <standard_library_modules>
import <third_party_library_modules>
import <application_modules>

# Define global variables
#......

# Define helper functions
#......

# Define the entry 'main' function
def main():
    """The main function doc-string"""
    #.......

# Run main() if this script is executed directly by the Python interpreter
if __name__ == '__main__':
    main()

Note

When you execute a Python module (via the Python Interpreter), the name is set to ‘main’. On the other hand, when a module is imported, its name is set to the module name. Hence, the above module will be executed if it is loaded by the Python interpreter, but not imported by another module.

4.4.8. Packages#

A module contains attributes (such as variables, functions and classes). Relevant modules (kept in the same directory) can be grouped into a package. Python also supports sub-packages (in sub-directories). Packages and sub-packages are a way of organizing Python’s module namespace by using “dotted names” notation, in the form of ‘<pack_name>.<sub_pack_name>.<sub_sub_pack_name>.<module_name>.<attr_name>’.

To create a Python package:

Create a directory and named it your package’s name.
Put your modules in it.
Create a ‘init.py’ file in the directory.

The ‘init.py’ marks the directory as a package. For example, suppose that you have this directory/file structure:

myapp/                 # This directory is in the 'sys.path'
   |
   + mypack1/          # A directory of relevant modules
   |    |
   |    + __init__.py  # Mark this directory as a package called 'mypack1'
   |    + mymod1_1.py  # Reference as 'mypack1.mymod1_1'
   |    + mymod1_2.py  # Reference as 'mypack1.mymod1_2'
   |
   + mypack2/          # A directory of relevant modules
        |
        + __init__.py  # Mark this directory as a package called 'mypack2'
        + mymod2_1.py  # Reference as 'mypack2.mymod2_1'
        + mymod2_2.py  # Reference as 'mypack2.mymod2_2'

If ‘myapp’ is in your ‘sys.path’, you can import ‘mymod1_1’ as:

import mypack1.mymod1_1       # Reference 'attr1_1_1' as 'mypack1.mymod1_1.attr1_1_1'
from mypack1 import mymod1_1  # Reference 'attr1_1_1' as 'mymod1_1.attr1_1_1'

Without the ‘init.py’, Python will NOT search the ‘mypack1’ directory for ‘mymod1_1’. Moreover, you cannot reference modules in the ‘mypack1’ directory directly (e.g., ‘import mymod1_1’) as it is not in the ‘sys.path’.

4.4.8.1. Attributes in ‘init.py’#

The ‘init.py’ file is usually empty, but it can be used to initialize the package such as exporting selected portions of the package under more convenient name, hold convenience functions, etc.

The attributes of the ‘init.py’ module can be accessed via the package name directly (i.e., ‘.’ instead of ‘.<init>.’). For example,

import mypack1               # Reference 'myattr1' in '__init__.py' as 'mypack1.myattr1'
from mypack1 import myattr1  # Reference 'myattr1' in '__init__.py' as 'myattr1'

4.4.9. Function Variables (Variables of Function Object)#

Function is an object in Python.

In Python, a variable takes a value or object (such as int, str). It can also take a function. For example,

>>> def square(n): return n * n

>>> square(5)
25
>>> sq = square   # Assign a function to a variable. No parameters.
>>> sq(5)
25
>>> type(square)
<class 'function'>
>>> type(sq)
<class 'function'>
>>> square
<function square at 0x7f0ba7040f28>
>>> sq
<function square at 0x7f0ba7040f28>  # Exactly the same reference as square

4.4.10. Nested Functions#

Python supports nested functions, i.e., defining a function inside a function. For example,

def outer(a):      # Outer function
    print('outer() begins with arg =', a)
    x = 1  # Local variable of outer function

    # Define an inner function
    # Outer has a local variable holding a function object
    def inner(b):  
        print('inner() begins with arg =', b)
        y = 2  # Local variable of the inner function
        print('a = {}, x = {}, y = {}'.format(a, x, y))
            # Have read access to outer function's attributes
        print('inner() ends')

    # Call inner function defined earlier
    inner('bbb')

    print('outer() ends')

# Call outer function, which in turn calls the inner function 
outer('aaa')

The expected output is:

outer begins with arg = aaa
inner begins with arg = bbb
a = aaa, x = 1, y = 2
inner ends
outer ends

Take note that the inner function has read-access to all the attributes of the enclosing outer function, and the global variable of this module.

4.4.11. Lambda Function (Anonymous Function)#

Lambda functions are anonymous function or un-named function. They are used to inline a function definition, or to defer execution of certain codes. The syntax is:

lambda arg1, arg2, ...: return_expression

For example,

# Define an ordinary function using def and name
>>> def f1(a, b, c): return a + b + c

>>> f1(1, 2, 3)
6
>>> type(f1)
<class 'function'>

# Define a Lambda function (without name) and assign to a variable
>>> f2 = lambda a, b, c: a + b + c  # No need for return keyword, similar to evaluating an expression

>>> f2(1, 2, 3)  # Invoke function
6
>>> type(f2)
<class 'function'>

f1 and f2 do the same thing. Take note that return keyword is NOT needed inside the lambda function. Instead, it is similar to evaluating an expression to obtain a value.

Lambda function, like ordinary function, can have default values for its parameters.

>>> f3 = lambda a, b=2, c=3: a + b + c
>>> f3(1, 2, 3)
6
>>> f3(8)
13

Take note that the body of a lambda function is a one-liner return_expression. In other words, you cannot place multiple statements inside the body of a lambda function. You need to use a regular function for multiple statements.

The definitions

def f(x):
    return x**3

and

f = lambda x: x**3

are entirely equivalent.

4.4.12. Assertion and Exception Handling#

4.4.12.1. `assert` Statement#

You can use assert statement to test a certain assertion (or constraint). For example, if x is supposed to be 0 in a certain part of the program, you can use the assert statement to test this constraint. An AssertionError will be raised if x is not zero.

Syntax

The syntax for assert is:

assert test, error-message

If the test if True, nothing happens; otherwise, an AssertionError will be raised with the error-message.

For example,

>>> x = 0
>>> assert x == 0, 'x is not zero?!'  # Assertion true, no output
 
>>> x = 1
>>> assert x == 0, 'x is not zero?!'  # Assertion false, raise AssertionError with the message
......
AssertionError: x is not zero?!

4.4.12.2. Exceptions#

In Python, errors detected during execution are called exceptions. For example,

>>> 1/0        # Divide by 0
ZeroDivisionError: division by zero
>>> zzz        # Variable not defined
NameError: name 'zzz' is not defined
>>> '1' + 1    # Cannot concatenate string and int
TypeError: Can't convert 'int' object to str implicitly

>>> lst = [0, 1, 2]
>>> lst[3]        # Index out of range
IndexError: list index out of range
>>> lst.index(8)  # Item is not in the list
ValueError: 8 is not in list

>>> int('abc')    # Cannot parse this string into int
ValueError: invalid literal for int() with base 10: 'abc'

>>> tup = (1, 2, 3)
>>> tup[0] = 11    # Tuple is immutable
TypeError: 'tuple' object does not support item assignment

Whenever an exception is raised, the program terminates abruptly.

4.4.12.3. try-except-else-finally#

You can use try-except-else-finally exception handling facility to prevent the program from terminating abruptly.

The exception handling process for try-except-else-finally is:

Python runs the statements in the try-block.
If no exception is raised in all the statements of the try-block, all the except-blocks are skipped, and the program continues to the next statement after the try-except statement.
However, if an exception is raised in one of the statements in the try-block, the rest of try-block will be skipped. The exception is matched with the except-blocks. The first matched except-block will be executed. The program then continues to the next statement after the try-except statement, instead of terminates abruptly. Nevertheless, if none of the except-blocks is matched, the program terminates abruptly.
The else-block will be executable if no exception is raised.
The finally-block is always executed for doing house-keeping tasks such as closing the file and releasing the resources, regardless of whether an exception has been raised.

Syntax

The syntax for try-except-else-finally is:

try:
    statements
except exception_1:                # Catch one exception
    statements
except (exception_2, exception_3): # Catch multiple exceptions
    statements
except exception_4 as var_name:    # Retrieve the exception instance
    statements
except:         # For (other) exceptions
    statements
else:
    statements   # Run if no exception raised
finally:
    statements   # Always run regardless of whether exception raised

The try-block (mandatory) must follow by at least one except or finally block. The rests are optional.

Example 1: Handling Index out-of-range for List Access

def get_item(seq, index):
    """Return the indexed item of the given sequences."""
    try:
        result = seq[index]   # may raise IndexError
        print('try succeed')      
    except IndexError:
        result = 0
        print('Index out of range')
    except:        # run if other exception is raised
        result = 0
        print('other exception')
    else:          # run if no exception raised
        print('no exception raised')
    finally:       # always run regardless of whether exception is raised
        print('run finally')

    # Continue into the next statement after try-except-finally instead of abruptly terminated.
    print('continue after try-except')
    return result
 
print(get_item([0, 1, 2, 3], 1))  # Index within the range
print('-----------')
print(get_item([0, 1, 2, 3], 4))  # Index out of range

The expected outputs are:

try succeed
no exception raised
run finally
continue after try-except
1
-----------
Index out of range
run finally
continue after try-except
0

Example 2: Input Validation

>>> while True:
       try:
           x = int(input('Enter an integer: '))  # Raise ValueError if input cannot be parsed into int
           break                                 # Break out while-loop
       except ValueError:
           print('Invalid input! Try again...')    # Repeat while-loop

Enter an integer: abc
Wrong input! Try again...
Enter an integer: 11.22
Wrong input! Try again...
Enter an integer: 123

4.4.12.4. `raise` Statement#

You can manually raise an exception via the raise statement. The syntax is:

raise exception_class_name     # E.g. raise IndexError
raise exception_instance_name  # E.g. raise IndexError('out of range')
raise                          # Re-raise the most recent exception for propagation

For example:

>>> raise IndexError('out-of-range')
IndexError: out-of-range

A raise without argument in the except block re-raise the exception to the outer block, e.g.,

try:
    ......
except:
    raise   # re-raise the exception (for the outer try)

Built-in Exceptions

Some exceptions are:

BaseException, Exception, StandardError: base classes
ArithmeticError: for OverflowError, ZeroDivisionError, FloatingPointError.
BufferError:
LookupError: for IndexError, KeyError.
Environment: for IOError, OSError.

User-defined Exception

You can defined your own exception by sub-classing the Exception class.

Example:

class MyCustomError(Exception):  # Sub-classing Exception base class (to be explained in OOP)
    """My custom exception"""

    def __init__(self, value):
        """Constructor"""
        self.value = value

    def __str__(self):
        return repr(self.value)

# Test the exception defined
try:
    raise MyCustomError('an error occurs')
    print('after exception')
except MyCustomError as e:
    print('MyCustomError: ', e.value)
else:
    print('running the else block')
finally:
    print('always run the finally block')

4.4.12.5. with-as Statement and Context Managers#

The syntax of the with-as statement is as follows:

with ... as ...:
    statements
   
# More than one items
with ... as ..., ... as ..., ...:
    statements

Python’s with statement supports the concept of a runtime context defined by a context manager. In programming, context can be seen as a bucket to pass information around, i.e., the state at a point in time. Context Managers are a way of allocating and releasing resources in the context.

Example 1

with open('test.log', 'r') as infile:  # automatically close the file at the end of with
    for line in infile:
        print(line)

This is equivalent to:

infile = open('test.log', 'r')
try:
    for line in infile:
        print(line)
finally:
    infile.close()

The with-statement’s context manager acquires, uses, and releases the context (of the file) cleanly, and eliminate a bit of boilerplate.

However, the with-as statement is applicable to certain objects only, such as file; while try-finally can be applied to all.

Example 2:

# Copy a file
with open('in.txt', 'r') as infile, open('out.txt', 'w') as outfile:
    for line in infile:
        outfile.write(line)

4.4.13. Local Variables vs. Global Variables#

Local-Scope: Names created inside a function (i.e., within def statement) are local to the function and are a vailable inside the function only.

Module-Scope: Names created outside all functions are global to that particular module (or file), but not available to the other modules. Global variables are available inside all the functions defined in the module. Global-scope in Python is equivalent to module-scope or file-scope. There is NO all-module-scope in Python.

For example,

x = 'global'     # x is a global variable for THIS module (module-scope)
 
def foo(arg):    # arg is a local variable for this function
    y = 'local'  # y is also a local variable
    
    # Function can access both local and global variables
    print(x)
    print(y)
    print(arg)
 
foo('abc')
print(x)
#print(y)   # locals are not visible outside the function
#print(arg)

4.4.14. Functions are Objects#

In Python, functions are objects. Like any object,

a function can be assigned to a variable;
a function can be passed into a function as an argument; and
a function can be the return value of a function, i.e., a function can return a function.

4.4.14.1. Example: Passing a Function Object as a Function Argument#

A function name is an object reference that can be passed into another function as argument.

def my_add(x, y): return x + y

def my_sub(x, y): return x - y

# This function takes a function object as its first argument
def my_apply(func, x, y):
    # Invoke the function received
    return func(x, y)

print(my_apply(my_add, 3, 2))  #5
print(my_apply(my_sub, 3, 2))  #1

# We can also pass an anonymous (Lambda) function as argument
print(my_apply(lambda x, y: x * y, 3, 2))  #6

4.4.14.2. Example: Returning an Inner Function object from an Outer Function#

# Define an outer function
def my_outer():
    # Outer has a function local variable
    def my_inner():  
        print('hello from inner')

    # Outer returns the inner function defined earlier
    return my_inner

result = my_outer()  # Invoke outer function, which returns a function object
result()             # Invoke the return function.
#'hello from inner'
print(result)
#'<function inner at 0x7fa939fed410>'

4.4.14.3. Example: Returning a Lambda Function#

def increase_by(n):
    return lambda x: x + n  # Return a one-argument anonymous function object

plus_8 = increase_by(8)    # Return a specific invocation of the function,
                           # which is also a function that takes one argument
type(plus_8)
#<class 'function'>
print(plus_8(1))    # Run the function with one argument.
#9

plus_88 = increase_by(88)
print(plus_88(1))
# 89

# Same as above with anonymous references
print(increase_by(8)(1))
#9
print(increase_by(88)(1))
#89

4.4.15. Function Closure#

In the above example, n is not local to the lambda function. Instead, n is obtained from the outer function.

When we assign increase_by(8) to plus_8, n takes on the value of 8 during the invocation. But we expect n to go out of scope after the outer function terminates. If this is the case, calling plus_8(1) would encounter an non-existent n?

This problem is resolved via so called Function Closure. A closure is an inner function that is passed outside the enclosing function, to be used elsewhere. In brief, the inner function creates a closure (enclosure) for its enclosing namespaces at definition time. Hence, in plus_8, an enclosure with n=8 is created; while in plus_88, an enclosure with n=88 is created. Take note that Python only allows the read access to the outer scope, but not write access. You can inspect the enclosure via function_name.func_closure, e.g.,

4.4.15.1. Functional Programming: Using Lambda Function in filter(), map(), reduce() and Comprehension#

Instead of using a for-in loop to iterate through all the items in an iterable (sequence), you can use the following functions to apply an operation to all the items. This is known as functional programming or expression-oriented programming. Filter-map-reduce is popular in big data analysis (or data science).

filter(func, iterable): Return an iterator yielding those items of iterable for which func(item) is True. For example,

>>> lst = [11, 22, 33, 44, 55]
>>> filter(lambda x: x % 2 == 0, lst)  # even number
<filter object at 0x7fc46f72b8d0>
>>> list(filter(lambda x: x % 2 == 0, lst))  # Convert filter object to list
[22, 44]
>>> for item in filter(lambda x: x % 2 == 0, lst): print(item, end=' ')
22 44
>>> print(filter(lambda x: x % 2 == 0, lst))  # Cannot print() directly
<filter object at 0x6ffffe797b8>

map(func, iterable): Apply (or Map or Transform) the function func on each item of the iterable. For example,

>>> lst = [11, 22, 33, 44, 55]
>>> map(lambda x: x*x, lst)   # square
<map object at 0x7fc46f72b908>
>>> list(map(lambda x: x*x, lst))  # Convert map object to list
[121, 484, 1089, 1936, 3025]
>>> for item in map(lambda x: x*x, lst): print(item, end=' ')
121 484 1089 1936 3025
>>> print(map(lambda x: x*x, lst))  # Cannot print() directly?
<map object at 0x6ffffe79a90>

reduce(func, iterable): (in module functools): Apply the function of two arguments cumulatively to the items of a sequence, from left to right, so as to reduce the sequence to a single value, also known as aggregation. For example,

>>> lst = [11, 22, 33, 44, 55]
>>> from functools import reduce
>>> reduce(lambda x,y: x+y, lst)  # aggregate into sum
165    # (((11 + 22) + 33) + 44) + 55

filter-map-reduce: used frequently in big data analysis to obtain an aggregate value.

# Combining filter-map to produce a new sub-list
>>> new_lst = list(map(lambda x: x*x, filter(lambda x: x % 2 == 0, lst)))
>>> new_lst
[4, 36]

# Combining filter-map-reduce to obtain an aggregate value
>>> from functools import reduce
>>> reduce(lambda x, y: x+y, map(lambda x: x*x, filter(lambda x: x % 2 == 0, lst)))
40

List comprehension: a one-liner to generate a list as discussed in the earlier section. e.g.,

>>> lst = [3, 2, 6, 5]
>>> new_lst = [x*x for x in lst if x % 2 == 0]
>>> new_lst
[4, 36]

# Using Lambda function
>>> f = lambda x: x*x    # define a lambda function and assign to a variable
>>> new_lst = [f(x) for x in lst if x % 2 == 0]  # Invoke on each element
>>> new_lst
[4, 36]
>>> new_lst = [(lambda x: x*x)(x) for x in lst if x % 2 == 0]  # inline lambda function
>>> new_lst
[4, 36]

These mechanisms replace the traditional for-loop, and express their functionality in simple function calls. It is called functional programming, i.e., applying a series of functions (filter-map-reduce) over a collection.

4.4.16. Decorators#

In Python, a decorator is a callable (function) that takes a function as an argument and returns a replacement function. Recall that functions are objects in Python, i.e., you can pass a function as argument, and a function can return an inner function. A decorator is a transformation of a function. It can be used to pre-process the function arguments before passing them into the actual function; or extending the behavior of functions that you don’t want to modify.

Example: Decorating an 1-argument Function

def clamp_range(func):  # Take a 1-argument function as argument
    """Decorator to clamp the value of the argument to [0,100]"""
    def _wrapper(x):    # Applicable to functions of 1-argument only
        if x < 0:
            x = 0
        elif x > 100:
            x = 100
        return func(x)  # Run the original 1-argument function with clamped argument
    return _wrapper

def square(x): return x*x

# Invoke clamp_range() with square()
print(clamp_range(square)(5))    # 25
print(clamp_range(square)(111))  # 10000 (argument clamped to 100)
print(clamp_range(square)(-5))   # 0 (argument clamped to 0)

# Transforming the square() function by replacing it with a decorated version
square = clamp_range(square)  # Assign the decorated function back to the original
print(square(50))    # Output: 2500
print(square(-1))    # Output: 0
print(square(101))   # Output: 10000

Notes:

The decorator clamp_range() takes a 1-argument function as its argument, and returns an replacement 1-argument function _wrapper(x), with its argument x clamped to [0,100], before applying the original function.
In ‘square=clamp_range(square)’, we decorate the square() function and assign the decorated (replacement) function to the same function name (confusing?!). After the decoration, the square() takes on a new decorated life!

Example: Using the @ symbol

Using ‘square=clamp_range(square)’ to decorate a function can be messy. Instead, Python uses the @ symbol to denote the replacement. For example,

def clamp_range(func):
    """Decorator to clamp the value of the argument to [0,100]"""
    def _wrapper(x):
        if x < 0:
            x = 0
        elif x > 100:
            x = 100
        return func(x)  # Run the original 1-arg function with clamped argument
    return _wrapper

# Use the decorator @ symbol
# Same as cube = clamp_range(cube)
@clamp_range
def cube(x): return x**3

print(cube(50))    # Output: 12500
print(cube(-1))    # Output: 0
print(cube(101))   # Output: 1000000

Example: Decorator with an Arbitrary Number of Function Arguments

The above example only work for one-argument function. You can use *args and/or **kwargs to handle variable number of arguments. For example, the following decorator log all the arguments before the actual processing.

def logger(func):
    """log all the function arguments"""
    def _wrapper(*args, **kwargs):
        print('The arguments are: {}, {}'.format(args, kwargs))
        return func(*args, **kwargs)  # Run the original function
    return _wrapper

@logger
def myfun(a, b, c=3, d=4):
    pass   # Python syntax needs a dummy statement here

myfun(1, 2, c=33, d=44)  # Output: The arguments are: (1, 2), {'c': 33, 'd': 44}
myfun(1, 2, c=33)        # Output: The arguments are: (1, 2), {'c': 33}

We can also modify our earlier clamp_range() to handle an arbitrary number of arguments:

def clamp_range(func):
    """Decorator to clamp the value of ALL arguments to [0,100]"""
    def _wrapper(*args):
        newargs = []
        for item in args:
            if item < 0:
                newargs.append(0)
            elif item > 100:
                newargs.append(100)
            else:
                newargs.append(item)
        return func(*newargs)  # Run the original function with clamped arguments
    return _wrapper

@clamp_range
def my_add(x, y, z): return x + y + z

print(my_add(1, 2, 3))     # Output: 6
print(my_add(-1, 5, 109))  # Output: 105

Example: Passing Arguments into Decorators

Let’s modify the earlier clamp_range decorator to take two arguments - min and max of the range.

from functools import wraps

def clamp_range(min, max):    # Take the desired arguments instead of func
    """Decorator to clamp the value of ALL arguments to [min,max]"""
    def _decorator(func):     # Take func as argument
        @wraps(func)          # For proper __name__, __doc__
        def _wrapper(*args):  # Decorate the original function here
            newargs = []
            for item in args:
                if item < min:
                    newargs.append(min)
                elif item > max:
                    newargs.append(max)
                else:
                    newargs.append(item)
            return func(*newargs)  # Run the original function with clamped arguments
        return _wrapper
    return _decorator

@clamp_range(1, 10)
def my_add(x, y, z):
    """Clamped Add"""
    return x + y + z
# Same as
# my_add = clamp_range(min, max)(my_add)
# 'clamp_range(min, max)' returns '_decorator(func)'; apply 'my_add' as 'func'

print(my_add(1, 2, 3))     # Output: 6
print(my_add(-1, 5, 109))  # Output: 16 (1+5+10)
print(my_add.__name__)     # Output: add
print(my_add.__doc__)      # Output: Clamped Add

The decorator clamp_range takes the desired arguments and returns a wrapper function which takes a function argument (for the function to be decorated).

4.4.17. Iterable and Iterator: iter() and next()#

An iterator in Python is an object that is used to iterate over iterable objects like lists, tuples, dicts, and sets. The Python iterators object is initialized using the iter() method. It uses the next() method for iteration.

Python’s next() function returns the next item of an iterator.

Python iterators are supported by two magic member methods: iter(self) and next(self).

The Iterable object (such as list) shall implement the iter(self) member method to return an iterator object. This method can be invoked explicitly via “iterable.iter()”, or implicitly via “iter(iterable)” or “for item in iterable” loop.
The returned iterator object shall implement the next(self) method to return the next item, or raise StopIeration if there is no more item. This method can be invoked explicitly via “iterator.next()”, or implicitly via “next(iterator)” or within the “for item in iterable” loop.

Example 1:

A list is an iterable that supports iterator.

# Using iter() and next() built-in functions
>>> lst_itr = iter([11, 22, 33])  # Get an iterator from a list
>>> lst_itr
<list_iterator object at 0x7f945e438550>
>>> next(lst_itr)
11
>>> next(lst_itr)
22
>>> next(lst_itr)
33
>>> next(lst_itr)  # No more item, raise StopIteration
...... 
StopIteration

# Using __iter__() and __next__() member methods
>>> lst_itr2 = [44, 55].__iter__()
>>> lst_itr2
<list_iterator object at 0x7f945e4385f8>
>>> lst_itr2.__next__()
44
>>> lst_itr2.__next__()
55
>>> lst_itr2.__next__()
StopIteration

# The "for each in iterable" loop uses iterator implicitly
>>> for item in [11, 22, 33]: 
        print(item)

4.4.18. Generator and yield#

A generator function is a function that can produce a sequence of results instead of a single value. A generator function returns a generator iterator object, which is a special type of iterator where you can obtain the next elements via next().

A generator function is like an ordinary function, but instead of using return to return a value and exit, it uses yield to produce a new result. A function which contains yield is automatically a generator function.

Generators are useful to create iterators.

Example 1: A Simple Generator

>>> def my_simple_generator():
        yield(11)
        yield(22)
        yield(33)
   
>>> g1 = my_simple_generator()
>>> g1
<generator object my_simple_generator at 0x7f945e441990>
>>> next(g1)
11
>>> next(g1)
22
>>> next(g1)
33
>>> next(g1)
......
StopIteration
>>> for item in my_simple_generator(): print(item, end=' ')
11 22 33

Example 2

The following generator function range_down(min, max) implements the count-down version of range(min, max+1).

>>> def range_down(min, max):
   """A generator function contains yield statement and creates a generator iterator object"""
   current = max
   while current >= min:
       yield current  # Produce a result each time it is run
       current -= 1   # Count down

>>> range_down(5, 8)
<generator object range_down at 0x7f5e34fafc18>  # A generator function returns a generator object

# Using the generator in the for-in loop
>>> for i in range_down(5, 8):
   print(i, end=" ")  # 8 7 6 5

# Using iter() and next()
>>> itr = range_down(2, 4)  # or iter(range_down(2, 4))
>>> itr
<generator object range_down at 0x7f230d53a168>
>>> next(itr)
4
>>> next(itr)
3
>>> next(itr)
2
>>> next(itr)
StopIteration

# Using __iter__() and __next__()
>>> itr2 = range_down(5, 6).__iter__()
>>> itr2
<generator object range_down at 0x7f230d53a120>
>>> itr2.__next__()
6
>>> itr2.__next__()
5
>>> itr2.__next__()
StopIteration

Each time the yield statement is run, it produce a new value, and updates the state of the generator iterator object.

Example 3

We can have generators which produces infinite value.

from math import sqrt, ceil

def gen_primes(number):
    """A generator function to generate prime numbers, starting from number"""
    while True:   # No upperbound!
        if is_prime(number):
            yield number
        number += 1


def is_prime(number:int) -> int:
    if number <= 1:
        return False

    factor = 2
    while (factor <= ceil(sqrt(number))):
        if number % factor == 0: return False
        factor += 1

    return True


if __name__ == '__main__':
    g = gen_primes(8)     # From 8
    for i in range(100):  # Generate 100 prime numbers
        print(next(g))

Generator Expression

A generator expression has a similar syntax as a list/dictionary comprehension (for generating a list/dictionary), but surrounded by braces and produce a generator iterator object. (Note: braces are used by tuples, but they are immutable and thus cannot be comprehended.) For example,

>>> a = (x*x for x in range(1,5))
>>> a
<generator object <genexpr> at 0x7f230d53a2d0>
>>> for item in a: print(item, end=' ')
1 4 9 16 
>>> sum(a)  # Applicable to functions that consume iterable
30
>>> b = (x*x for x in range(1, 10) if x*x % 2 == 0)
>>> for item in b: print(item, end=' ')
4 16 36 64 

# Compare with list/dictionary comprehension for generating list/dictionary
>>> lst = [x*x for x in range(1, 10)]
>>> lst
[1, 4, 9, 16, 25, 36, 49, 64, 81]
>>> dct = {x:x*x for x in range(1, 10)}
>>> dct
{1: 1, 2: 4, 3: 9, 4: 16, 5: 25, 6: 36, 7: 49, 8: 64, 9: 81}

4.4.19. Recursive Function Calls#

Basically, a recursive function is a function that calls itself.

Advantages of using recursion:

A complicated function can be split down into smaller sub-problems utilizing recursion.
Sequence creation is simpler through recursion than utilizing any nested iteration.
Recursive functions render the code look simple and effective.

Disadvantages of using recursion:

A lot of memory and time is taken through recursive calls which makes it expensive for use.
Recursive functions are challenging to debug.
The reasoning behind recursion can sometimes be tough to think through.

Sintax:

def func(): <--
              |
              | (recursive call)
              |
    func() ----

Example 1: The Fibonacci sequence is defined as \(F_n=F_{n-1}+F_{n-2}, \quad F_1=1, \quad F_2=1\). In words, each term of the Fibonacci sequence is the sum of the previous two terms. The first several terms of the sequence are 1,1,2,3,5,8,13,21,34,55,89,144,233,377,610,987,…

# Program to print the fibonacci series upto n_terms

# Recursive function
def recursive_fibonacci(n):
  if n <= 1:
      return n
  else:
      return(recursive_fibonacci(n-1) + recursive_fibonacci(n-2))

n_terms = 15

# check if the number of terms is valid
if n_terms <= 0:
  print("Invalid input ! Please input a positive value")
else:
  print("Fibonacci series:")
for i in range(1,n_terms+1):
    print(recursive_fibonacci(i),end=',')

Example 2: The factorial for a positive number is defined as \(n! = n \cdot (n-1)!\).

# Program to print factorial of a number
# recursively.

# Recursive function
def recursive_factorial(n):
  if n == 1:
      return n
  else:
      return n * recursive_factorial(n-1)

# user input
num = int(input('Enter a positive number: ')

# check if the input is valid or not
if num < 0:
  print("Invalid input ! Please enter a positive number.")
elif num == 0:
  print("Factorial of number 0 is 1")
else:
  print("Factorial of number", num, "=", recursive_factorial(num))

Example 3: Consider the problem of computing \(x_t\) for some t when

(4.1)#\[x_{t+1} = 2 x_t, \quad x_0 = 1\]

Obviously the answer is \(2^t\).

We can compute this easily enough with a loop

def x_loop(t):
    x = 1
    for i in range(t):
        x = 2 * x
    return x

We can also use a recursive solution, as follows

def x(t):
    if t == 0:
        return 1
    else:
        return 2 * x(t-1)

Example 4: Solve the Tower of Hanoi problem.

Tower of Hanoi is a mathematical puzzle where we have three rods and n disks. The objective of the puzzle is to move the entire stack to another rod, obeying the following simple rules:

Only one disk can be moved at a time.
Each move consists of taking the upper disk from one of the stacks and placing it on top of another stack i.e. a disk can only be moved if it is the uppermost disk on a stack.
No disk may be placed on top of a smaller disk.

To solve the Tower of Hanoi using recursion we employ a function that recursively breaks down the problem of moving n disks into smaller problems of moving n-1 disks. It alternates the roles of the rods (source, destination, auxiliary) in each recursive call to facilitate the step-by-step transfer of disks according to the rules of the Tower of Hanoi puzzle.

# Recursive Python function to solve the tower of hanoi

def TowerOfHanoi(n , source, destination, auxiliary):
    if n==1:
        print ("Move disk 1 from source",source,"to destination",destination)
        return
    TowerOfHanoi(n-1, source, auxiliary, destination)
    print ("Move disk",n,"from source",source,"to destination",destination)
    TowerOfHanoi(n-1, auxiliary, destination, source)
        
# Driver code
n = int(input("Input the number of disks "))
TowerOfHanoi(n,'A','C','B') 

4.5. Frequently-Used Python Standard Library Modules#

Remember that a module is a collection of functions and data with a common theme. Python contains a number of native modules that come with every installation of Python. The next table lists a few common examples, but there are certainly many others worth exploring. This section will introduce a few useful modules with some examples of their uses.

Python provides a set of standard library and many non-standard libraries are provided by third party.

Table Some Python Modules

Name	Description
`math`	Mathematical functions on floating-point numbers
`cmath`	Mathematical functions on complex numbers
`statistics`	Statistics functions
`random`	Functions for pseudorandom number generation
`sys`	Provides functions and variables of the Python Runtime Environment
`os`	Provides access to your computer file system
`time`	Times the execution of code
`datetime`	Handling of date and time information
`timeit`	Times the execution of code
`pickle`	Preserves Python objects on the file system
`itertools`	Iterator and combinatorics tools
`csv`	For writing and reading CSV files
`audioop`	Tools for reading and working with audio files

To use a module, use ‘import <module_name>’ or ‘from <module_name> import <attribute_name>’ to import the entire module or a selected attribute. You can use ‘dir(<module_name>)’ to list all the attributes of the module, ‘help(<module_name>)’ or ‘help(<attribute_name>)’ to read the documentation page. For example,

>>> import math   # import an external module
>>> dir(math)     # List all attributes
['e', 'pi', 'sin', 'cos', 'tan', 'tan2', ....]
>>> help(math)    # Show the documentation page for the module
......
>>> help(math.atan2)  # Show the documentation page for a specific attribute
......
>>> math.atan2(3, 0)
1.5707963267948966
>>> math.sin(math.pi / 2)
1.0
>>> math.cos(math.pi / 2)
6.123233995736766e-17

>>> from math import pi  # import an attribute from a module
>>> pi
3.141592653589793

4.5.1. `math` and `cmath` Modules#

The math module provides access to the mathematical functions defined by the C language standard. The commonly-used attributes are:

Constants: pi, e.
Power and exponent: pow(x,y), sqrt(x), exp(x), log(x), log2(x), log10(x)
Converting float to int: ceil(x), floor(x), trunc(x).
float operations: fabs(x), fmod(x)
hypot(x,y) (=sqrt(xx + yy))
Conversion between degrees and radians: degrees(x), radians(x).
Trigonometric functions: sin(x), cos(x), tan(x), acos(x), asin(x), atan(x), atan2(x,y).
Hyperbolic functions: sinh(x), cosh(x), tanh(x), asinh(x), acosh(x), atanh(x).

For examples,

>>> import math
>>> dir(math)
......
>>> help(math)
......
>>> help(math.trunc)
......

# Test floor(), ceil() and trunc()
>>> x = 1.5
>>> type(x)
<class 'float'>
>>> math.floor(x)
1
>>> type(math.floor(x))
<class 'int'>
>>> math.ceil(x)
2
>>> math.trunc(x)
1
>>> math.floor(-1.5)
-2
>>> math.ceil(-1.5)
-1
>>> math.trunc(-1.5)
-1

4.5.2. `statistics` Module#

The statistics module computes the basic statistical properties such as mean, median, variance, and etc. For examples,

>>> import statistics
>>> dir(statistics)
['mean', 'median', 'median_grouped', 'median_high', 'median_low', 'mode', 'pstdev', 'pvariance', 'stdev', 'variance', ...]
>>> help(statistics)
......
>>> help(statistics.pstdev)
......

>>> data = [5, 7, 8, 3, 5, 6, 1, 3]
>>> statistics.mean(data)
4.75
>>> statistics.median(data)
5.0
>>> statistics.stdev(data)
2.3145502494313788
>>> statistics.variance(data)
5.357142857142857
>>> statistics.mode(data)
statistics.StatisticsError: no unique mode; found 2 equally common values

4.5.3. `random` Module#

The module random can be used to generate various pseudo-random numbers.

>>> import random
>>> dir(random)
......
>>> help(random)
......
>>> help(random.random)
......

>>> random.random()       # float in [0,1)
0.7259532743815786
>>> random.random()
0.9282534690123855
>>> random.randint(1, 6)  # int in [1,6]
3
>>> random.randrange(6)   # From range(6), i.e., 0 to 5
0
>>> random.choice(['apple', 'orange', 'banana'])  # Pick from the given list
'apple'

4.5.4. `sys` Module#

The module sys (for system) provides system-specific parameters and functions. The commonly-used are:

sys.exit([exit_status=0]): exit the program by raising the SystemExit exception. If used inside a try, the finally clause is honored. The optional argument exit_status can be an integer (default to 0 for normal termination, or non-zero for abnormal termination); or any object (e.g., sys.exit(‘an error message’)).
sys.path: A list of module search-paths. Initialized from the environment variable PYTHONPATH, plus installation-dependent default entries.
sys.stdin, sys.stdout, sys.stderr: standard input, output and error stream.
sys.argv: A list of command-line arguments passed into the Python script. argv[0] is the script name. See example below.

Example: Command-Line Arguments

The command-line arguments are kept in sys.argv as a list. For example, create the following script called “test_argv.py”:

import sys
print(sys.argv)       # Print command-line argument list
print(len(sys.argv))  # Print length of list

Run the script:

$ python3 test_argv.py
['test_argv.py']   # sys.argv[0] is the script name
1
 
$ python3 test_argv.py hello 1 2 3 apple orange
['test_argv.py', 'hello', '1', '2', '3', 'apple', 'orange']   # list of strings
7

4.5.5. `time` Module#

The module time allows to work with time. It allows functionality like getting the current time, pausing the Program from executing, etc.

Example: Measure the time taken between lines of code.

The steps to use the time module to calculate the program’s execution time:

Import time module. Import it using the import statement.
Store the start time. To do this, we will use the time() function to get the current time and store it in a start_time variable before the first line of the program.
Store the end time. Again, we will use the time() function to get the current time and store it in the end_time variable before the last line of the program.
Calculate the execution time. The difference between the end time and start time is the execution time.

import time

# get the start time
st = time.time()
# main program
# find sum to first 10 million numbers
sum_x = 0
for i in range(10000000):
    sum_x += i
print('Sum of first 10 million numbers is:', sum_x)
# get the end time
et = time.time()
# get the execution time
elapsed_time = et - st
print('Execution time:', elapsed_time, 'seconds')

4.5.6. `os` Module#

The os module provides access to the files and directories (i.e., folders) on your computer. If you want to open a file somewhere else on your computer or open multiple files, this module is particularly useful.

Module Functions:

Function	Description
`os.chdir()`	Changes the current working directory to the path provide
`os.getcwd()`	Returns the current working directory path
`os.listdir()`	Returns a list of all files in the current or indicated directory

>>> import os
>>> dir(os)          # List all attributes
......
>>> help(os)         # Show man page
......
>>> help(os.getcwd)  # Show man page for specific function
......

>>> os.getcwd()                   # Get current working directory
... current working directory ...
>>> os.listdir()                  # List the contents of the current directory
... contents of current directory ...
>>> os.chdir('test-python')       # Change directory
>>> exec(open('hello.py').read()) # Run a Python script
>>> os.system('ls -l')            # Run shell command
>>> os.name                       # Name of OS
'posix'
>>> os.makedir('sub_dir')            # Create sub-directory
>>> os.makedirs('/path/to/sub_dir')  # Create sub-directory and the intermediate directories
>>> os.remove('filename')            # Remove file
>>> os.rename('oldFile', 'newFile')  # Rename file

Functions

Contents

4. Functions#

4.1. Overview#

4.2. Function Basics#

4.2.1. Built-In Functions#

4.2.2. Third Party Functions#

4.3. Defining Functions#

4.3.1. The pass statement#

4.3.2. Examples#

4.3.3. Function Parameters and Arguments#

4.3.3.1. Passing Arguments by-Value vs. by-Reference#

4.3.3.2. Function Parameters with Default Values#

4.3.3.3. Positional and Keyword Arguments#

4.3.3.4. Variable Number of Positional Arguments (*args)#

4.3.3.5. Unpacking List/Tuple into Positional Arguments (*lst, *tuple)#

4.3.3.6. Variable Number of Keyword Arguments (**kwargs)#

4.3.3.7. Unpacking Dictionary into Keyword Arguments (**dict)#

4.3.3.8. Using both *args and **kwargs#

4.3.4. Function Overloading#

4.3.5. Function Return Values#

4.3.6. Types Hints via Function Annotations#

4.4. Modules, Import-Statement and Packages#

4.4.1. Modules#

4.4.2. The import statement#

4.4.3. The from-import Statement#

4.4.4. import vs. from-import#

4.4.5. Conditional Import#

4.4.6. sys.path and PYTHONPATH/PATH environment variables#

4.4.7. Template for Python Standalone Module#

4.4.8. Packages#

4.4.8.1. Attributes in ‘init.py’#

4.4.9. Function Variables (Variables of Function Object)#

4.4.10. Nested Functions#

4.4.11. Lambda Function (Anonymous Function)#

4.4.12. Assertion and Exception Handling#

4.4.12.1. assert Statement#

4.4.12.2. Exceptions#

4.4.12.3. try-except-else-finally#

4.4.12.4. raise Statement#

4.4.12.5. with-as Statement and Context Managers#

4.4.13. Local Variables vs. Global Variables#

4.4.14. Functions are Objects#

4.4.14.1. Example: Passing a Function Object as a Function Argument#

4.4.14.2. Example: Returning an Inner Function object from an Outer Function#

4.4.14.3. Example: Returning a Lambda Function#

4.4.15. Function Closure#

4.4.15.1. Functional Programming: Using Lambda Function in filter(), map(), reduce() and Comprehension#

4.4.16. Decorators#

4.4.17. Iterable and Iterator: iter() and next()#

4.4.18. Generator and yield#

4.4.19. Recursive Function Calls#

4.5. Frequently-Used Python Standard Library Modules#

4.5.1. math and cmath Modules#

4.5.2. statistics Module#

4.5.3. random Module#

4.5.4. sys Module#

4.5.5. time Module#

4.5.6. os Module#

4.3.3.5. Unpacking List/Tuple into Positional Arguments (lst, tuple)#

4.4.2. The `import` statement#

4.4.3. The `from-import` Statement#

4.4.6. `sys.path` and `PYTHONPATH/PATH` environment variables#

4.4.12.1. `assert` Statement#

4.4.12.4. `raise` Statement#

4.5.1. `math` and `cmath` Modules#

4.5.2. `statistics` Module#

4.5.3. `random` Module#

4.5.4. `sys` Module#

4.5.5. `time` Module#

4.5.6. `os` Module#