Lecture 8¶

Object Oriented Programming IV¶

Tuesday, October 1st 2020¶

Last Time¶

• Dunder methods
• The Python Data Model

Today¶

• Class methods, static methods, instance methods
• Modules and packages

If we have time...

• "Privacy" in Python
• More details on Polymorphism

Building out our class: instances and classmethods¶

At this point, you should feel comfortable with classes, special methods, and the python data model.

We will take a short excursion to enhance our classes using classmethods. We will also see staticmethods and regular instance methods.

A Favorite Example¶

make_complex is a class method. See how its signature is different above. It is a factory to produce instances.

The take-away¶

• A classmethod has access to the actual class, but not the instance of the class.
• Use it to provide alternative ways of constructing an object of your class.
  • The original client only needed to create complex numbers from the real and imaginary parts.
  • A new client needs to create complex numbers from the polar form (radius and angle).
  • Provide a classmethod that allows users to construct complex number objects from the polar form!

Breakout Room¶

• Recall your favorite number. The speaker for this room will be the person with the largest favorite number.
• Think about some situations where you might need to create a classmethod.
  • Think of any class that you want to create.
  • Think about some different scenarios that may require different constructors.

Static Methods, Class Methods, Instance Methods¶

What's really going on under the hood here?

PythonTutor Example

Class variables and instance variables¶

PythonTutor Example

Class variables are shared between all instances of the class.

DRY Principle

Practical Comments on class methods¶

• Act as a factory to produce objects that are configured the way you want.
  • This can make life easier: Instead of defining a new object every time, just get a pre-defined one.
  • See Factory Method Design Pattern
• Pre-define commonly used objects.
• These objects all still use the same constructor.

Practical Comments on static methods¶

• Python doesn't need to instantiate a bound method for each object.
  • This saves on cost.
• Might improve code readability.
  • You know right away that the method doesn't depend on the state.

Going Deeper¶

Code and Data for objects¶

dir for classes contains the names of its attributes and recursively of the attributes of its bases.

vars on an object gets the contents of a special attribute called __dict__.

Let's make an instance of A.

dir again:

vars again:

There is some kind of a table implementation for Python objects (it's written in C).

This implementation allows us to look for attributes and methods, and if not found look elsewhere.

The exact details are complex, using descriptors and other lookups, and we'll tackle them in more detail later (hopefully).

But currently it suffices for us to know that lookup first happens in the instance table, followed by the class table (methods) and if not there somewhere up in the inheritance hierarchy.

Creating Packages from Python Code¶

Now that you know how to write classes, it's time to figure out how to combine them into a package.

A package is a collection of Python modules.

Let's start by reviewing how to import a Python module.

Module Recap¶

• Import a module with the import statement

  import mymod
  

Here's how Python searches for a module once it's imported:

1. The interpreter searches for a built-in module with that name.
2. If no built-in module exists with that name, then the interpreter searches for the name in the list of directories in the sys.path variable.
3. If the requested name can't be found, an ImportError exception is thrown.

The Many Ways to Import¶

Suppose your module contains some methods called myf1, myf2, and so on.

There are a variety of ways to import the module and its methods. Here are a few along with their uses:

Comments on Importing¶

• Generally a very bad idea to do from mymod import *. Can lead to name clashes!
• from mymod import myf1 is also dangerous if you're not careful.

• Recommendation: Just do import mymod or import mymod as new_name unless you have a very good reason for doing otherwise.

Where to put the import statements in a module? A common convention is:

• After the module's documentation.

What order to import libraries?

• First import standard library modules.
• Then import third-party library modules.
• Then import your own modules.

Modules and Packages¶

• For larger projects, you will have multiple modules.

• A collection of multiple modules is called a package.

Why multiple modules?¶

Having multiple modules helps with code organization.

physics_code/
             __init__.py  
             preprocessing/ # This is a (sub)package
                           __init__.py 
                           parse_xml_inputs.py # This is a module
                           parse_txt_inputs.py # and this is a module
                           ...
             solvers/
                     __init__.py
                     time_integrators.py
                     discretization.py
                     linear_solvers.py
                     ...
             postprocessing/
                            __init__.py
                            write_hdf5.py
                            write_txt.py
                            ...
                            stat_utils/
                                       __init__.py
                                       ...
                            viz/
                                __init__.py
                                line_plots.py
                                ...
             tests/
                   ...

What is __init__.py?¶

• Used for package initialization-time actions.

• Generates a module namespace for a directory.
  • In Python 3.3+, empty __init__.py is not required: Packages
  • Still use for package initialization

• Implements the from * behavior.
  • This is done using __all__ lists.
  • e.g. include the line __all__ = ["mod1", "mod2", ..., "modN"]

More Practical Comments on __init__.py¶

• Empty __init__.py files no longer necessary

• They help prevent directories with common names from hiding true modules

• The first time Python imports through a directory, it runs the code in __init__.py.

Working With Packages¶

• Once you have your directory structure set up (with the __init_.py files), you are ready to use the package.

Let's talk about __name__¶

• You may have seen the code snippet:

  if __name__ == "__main__":
    # Do some things
  

• The variable __name__ is created whenever a .py file is run and is set to the string "__main__".
• However, when a module is imported, __name__ is set to the module's name.
• Hence, if the module is not being run as a Python script, the if statement will not be executed.

Additional Information¶

As with most things Python, you can simply consult the excellent documentation: Python Modules.

• Absolute vs. Relative imports
• Compiled Python files

Illustrative Example¶

Consider the following directory structure:

dir1\
     __init__.py
     dir2/
          __init__.py
          mymod.py

Here is what is in each file:

# dir1/__init__.py
print("Initializing dir1/")

# dir2/__init__.py
print("Initializing dir2/")

# dir2/mymod.py
my_name = "David"

Outputs¶

• If I work from the command line in the container of dir1, I can see various things happen.

>>> import dir1.dir2.mymod
Initializing dir1/
Initializing dir2/
>>> dir1.dir2.mymod.my_name
'David'
>>> import dir1.dir2.mymod as mod
>>> mod.my_name
'David'

Some Practical Comments¶

• It's annoying to write all those paths manually.
• You can make the functions and classes available in __init__.py and then use the direct import statement

Consider the directory structure:

example.py
package\
     __init__.py
     mod.py

Now the import can be achieved with:

# __init__.py
from .mod import myclass
from .mod import myfunc

# mod.py
class myclass():
    def __init__(self):
        ...
    ...

def myfunc():
    ...

# example.py
import package
C = package.myclass()

Breakout Room¶

• Think of a package that you may want to create.
• Think of the subpackages and modules that you will include. How will the package be organized?
• What would you like to include in __init__.py?

Creating and Distributing Packages¶

At this point, you know how to create packages in Python and the basics of how things fit together.

Ultimately, you want to be able to distribute your package to other people.

There are a number of ways to do this...brace yourself.

• PyScaffold
  • It sets up the entire infrastructure for you.
  • That's great, but you might not understand all the details.
• Packaging and Distributing Projects
  • Fantastic documentation covering the meaning of everything.
• How To Package Your Python Code
  • Excellent tutorial on packaging.
• Tensor Basis Neural Network
  • Easy-to-follow real world example.
• How to package a python application to make it pip-installable
  • Bare-bones example.
• Submitting a Python package with GitHub and PyPI
  • Slightly out-of-date, but still useful.

There Are So Many Options!¶

• As you can see, you have many options on how to set up and distribute your package.

• I will give you broad freedom in how you do this, but your project must be easily installable.

What does "easiliy installable" mean?¶

• Using pip is great! This would be the easiest for the user.
• You are also welcome to host your project on GitHub and have the user manually install and test with setup.py.
• Either way, your package should be installable and the user should be able to run the tests.

Privacy in Python¶

• Python does not have private names
• It can "localize" some names in classes
• This localization is handled by "name mangling"
• Name mangling does not prevent access by code outside the class!
• Name mangling is intended to help avoid namespace collisions

Therefore, we say that Python has the notion of pseudoprivate names.

Pseudoprivacy and Name Mangling¶

Names inside a class that begin with two underscores are expanded to include the name of the enclosing class.

For example, suppose you have a class called Universes and a name in that class called __our_universe.

Python changes the name __our_universe to _Universes__our_universe.

Now if there is another class in the hierarchy containing an attribute name our_universe then the two names will not clash.

If you know the name of the enclosing class, you can still access the "private" attributes.

By the way...¶

This simple example allows us to say something about the Python method resolution order.

We've been dancing around this idea: This is the order that Python looks for a method in the hierarchy.

Let's see what this looks like for our example:

Some details: Private Variables.

A note on single underscores:

a name prefixed with an underscore (e.g. _spam) should be treated as a non-public part of the API (whether it is a function, a method or a data member).