Sunday, April 3, 2016

Private variables in Python (single and double underscore)

Private variables and methods do not exist in Python. There is no way to strictly restrict access to variables or methods from outside the classes where they were defined. If you come from another object-oriented language such as Java or C++ this may sound strange, to say the least. However, this behavior is intended and there are good reasons for it which, for the sake of brevity, this post will not address.

There are, however, ways to emulate private variables and methods in Python. You can hamper access from outside their classes by applying the following conventions outlined in PEP 8:

Single leading underscore "_" 

Prefixing a variable or method name with "_" (single underscore) indicates its intended for internal use and should not be accessed from outside the class in which it was defined.

class MyClass(variable):

    def __init__(self):
        self._meditation = 'zazen'  # Notice the single leading underscore

This is just a convention and will not actually stop us from accessing the variable or method from outside. However, it's advisable to follow this convention as ignoring it can cause a lot of problems.

>>> obj = MyClass()
>>> obj._meditation

Notice that the _meditation attribute is accessible because it was only hidden by convention.

  • Functions that provide internationalization and localization, used to prefix strings that need translation, are frequently named "_". Avoid confusing this with the "private" variables discussed in this post.

Double leading underscore "__"

Python has a feature called name mangling. When variables are prefixed with "__" (double underscore), their names are automatically changed into _<className><variableName>, so they are not easily accessible from outside their classes.

class MyClass(variable):
    def __init__(self):
        self.__meditation = 'zazen'  # Notice the double leading underscore

Now let's try to access it from outside the class:

>>> obj = MyClass()
>>> obj.__meditation
AttributeError: 'MyClass' variable has no attribute '__meditation'

t.__meditation was not found because its name was automatically changed (mangled) to _MyClass__meditation. We could still access it with the syntax below:

>>> obj._MyClass__meditation

The example above is used to illustrate how "private" variables or methods can be accessed outside their classes. This does not mean we should do it as it will most likely wreak all sorts of havoc.

Another example:

class Glass:
    def __init__(self):
        self.__status = None  # Notice the double leading underscore

    def fill(self):
        self.__status = 'filled'

    def empty(self):
        self.__status = 'empty'

    def print(self):

In the example above, the __status attribute of the Glass class can only be set by the empty() and fill() methods. Let's test it:

>>> g = Glass()
>>> g.print()
>>> g.fill()
>>> g.print()
>>> g.empty()
>>> g.print()

In real code, "private" variables like these are used mostly to avoid clashes of names defined within classes with names defined within their subclasses.

  • Python's "magic" variables or attributes are also prefixed with double underscore (e.g., __init__, __import__ or __file__). In this context, the double underscore prefix has no "privatizing" effect. Avoid confusing this with the "private" variables discussed in this post.
Thank you for reading.

Saturday, April 2, 2016

Python function arguments, *args and **kwargs

If you are new to Python, you have probably stumbled upon *args and **kwargs in someone else's code, on a forum or tutorial. This post explains what they are.

Types of function arguments in Python

Python functions accept 2 kinds of arguments: positional arguments and keyword arguments. With a few exceptions (explained below), when you define a function, you don't specify whether the arguments are positional or keyword. This is defined at the time the function is called, depending on the syntax used to pass the arguments.

Positional arguments

These arguments are identified based on the position they are supplied when calling the function.
def divide(dividend, divisor):
  return dividend / divisor

When called, the above function outputs:

It knows 10 is the dividend and 2 is the divisor because of the order the arguments were provided.

Keyword arguments

Now let's suppose we want to call the above function inverting the argument order. We can do that using keyword arguments.
divide(divisor=2, dividend=10)
Notice that the argument order was inverted but they were passed by name, so they were correctly identified.

In Python 3, by inserting "*" as a first argument, we force all following arguments to be called by name (as keyword arguments).
def divide(*, dividend, divisor):
  return dividend / divisor
The function above outputs:
TypeError: divide() takes 0 positional arguments but 2 were given

divide(dividend=10, divisor=2)

What are *args and **kwargs?

*args and **kwargs allows you to pass an arbitrary number of arguments to a function without knowing them in advance.

The relevant syntax is the * and **. The names "args" and "kwargs" are only by convention. You could use *foo and **bar and have the same result. However, sticking to conventions makes it easier for us to read each other's code.

*args example

def print_args(*args):
  for arg in args:
Running the above function with an arbitrary number of arguments results in:
print_args('foo', 'bar', 5)
There is no limit to the number of positional arguments you can provide.

**kwargs example

def print_kwargs(**kwargs):
  for name, value in kwargs.items():
    print('{0} = {1}'.format(name, value))
If you are not familiar with the items() method, it's explained in this post.

Running the above function with an arbitrary number of arguments results in:
print_kwargs(meditation = 'zazen', minutes = 40)
meditation = zazen
minutes = 40

Combining fixed arguments, *args and **kwargs

def print_args(fixed_arg, *args, **kwargs):
Running the above code outputs:
print_args('foo', 'bar', 'baz', meditation = 'zazen', minutes = 40)
('bar', 'baz')
{'meditation': 'zazen', 'minutes': 40}
Notice that:
  • fixed_arg is a fixed positional argument 
  • args is a tuple of positional arguments 
  • kwargs is a dictionary of keyword arguments

Friday, April 1, 2016

Quick Reference: How to iterate over keys and values at once on a Python 3 dictionary

On Python 3, we use the items() method to iterate over a dictionary's keys and values simultaneously.
dictionary = {"name": "John", "age": "32"}

for key, val in dictionary.items():
    print(key, val)
When the above code is executed, it produces the following result:
age 32
name John

Thursday, March 31, 2016

Python Closures and Decorators

What are decorators?

Python has a great feature called decorators. In short, a decorator allows us to inject or modify the code in other functions or classes. There are function decorators and class decorators. This article is focused on function decorators.

Please be aware that the terms "outer function" and "parent function" are used interchangeably in this post, as are "inner function" and "child function".

Functions are first-class objects

In Python, everything is an object, including functions. Functions are first-class objects and, just like any other object (integers, strings, lists), they can be passed to other functions as arguments, assigned to variables, even returned from other functions.

Nested functions

We can also define functions inside functions. Nested functions are useful for encapsulation purposes (isolating inner functions from the global scope).

def parent():
     """Outer function"""
     print("Hello from parent() function.")

     def child():
         """Inner function"""
         return "Hello from child() function."


Inner functions are only visible inside the outer function. If we call the inner function from outside the outer function, we get a NameError.


In order to understand decorators, we must first understand closures. A closure is an inner function that remembers values set in the the outer function scope.
def outer(msg):
     """Outer function"""

     def inner():
         """Inner function"""

When the above code is executed, the following happens:

Notice that the inner function was able to access the value of the msg variable, which is defined in the outer function.

In the example above, the inner function is called on the last line of the outer function. What happens if, instead of calling the inner function, the outer function returned it?

def outer(msg):
     """Outer function"""

     def inner():
         """Inner function"""

     return inner()

When the above code is executed, the following happens:
my_function = outer("Hello from the inner function")
Hello from the inner function

The outer() function is a factory function
Basically, we have a closure when:
  • There are nested functions (a function inside another function).
  • The inner function refers to a value defined in the outer function.
  • The outer function returns the inner function.

Function decorators

A decorator is a function that:
  1. Takes other function as an argument.
  2. Adds to or modifies the function's code.
  3. Returns the modified function.
The preferred syntax for using decorators is:
def my_decorator(func):

    def wrapper():
        print("Run something before func() is called.")
        print("Run something after func() is called.")

    return wrapper

def my_function():
    print('Hello from my_function.')
The "@" in the line above a function indicates the application of a decorator. In the above example, my_decorator is applied to my_function(). The output will be:
Run something before func() is called
Hello from my_function.
Run something after func() is called.

Chaining decorators

It's possible to apply multiple decorators to a function. 
def my_function():
    print('Hello from my_function.')
The bottommost decorator (the one closest to the function) is executed first, then the function returned by this decorator is passed to the decorator above it and so on.

A future post will talk about class decorators.