Python Useful Wrappers/Decorators

Attention

Source Code: GH/pydry

PyDRY (Python’s “Don’t Repeat Yourself” Tools) is a software development principle that encourages developers (like me) to avoid duplication of code in a system. To effectively understand and use this principle in python coding context one might need the use of advanced powerful tools that makes the code cleaner, more resuable and easier to manage. The pydry is module is developed to simulate such a scenario. Below are some of the use cases that were designed to be simple and efficient and provide extensions to other developed functions and code-blocks.

Python Decorators

A decorator in python is a powerful tool that allows to modify the behavior of a function or a class by wrapping another method that extends its behavior without permanently modifying it.

Time Decorators

A Set of Decorators related to Timing and Profiling

timer_decorator.timeit(func)

A Mimic of the iPython Magic %timeit for Python Function

The built-in iPython magic function %timeit displays the executed time of a function, or like the one from command line:

python -m timeit "function()"

The function creates a decorator which can be used to time and analyze the execution time for any function by:

@timeit
def some_function():
    # do some work

some_function() # execute the function with the decorator
> Run time for some_function: ****

The function is meant for general python function, for more robus operation and using the same for pandas use the pandaswizard.wrappers.timeit - more information is available here.

The wrappers are a great way that provides information that can be used for time and memory optimization. However, python does not support inline decorator definations - more information is available at SO28245471.

A context manager (with) can serve as a bridge between the decorators and in-line capabilities of function execution GH#32.

Context Managers

An advanced python tool - a context manager defines the runtime context (or behavior) for executing a code within a with statement. Typical example of a context manager is:

with open(filename, "r") as f:
  ... # do something

In the above statement, we use the context manager open to interact with a file and auto close the file without the need of explicit .close() method on the open file (more information).

Time Context Manegers

An Interface of Common Functions that Extends the Wrappers

The wrappers are a great way that provides information that can be used for time and memory optimization. However, python does not support inline decorator definations - more information is available at SO28245471.

A context manager (with) can serve as a bridge between the decorators and in-line capabilities of function execution GH#32.

class context_managers.CatchTime(fmt: str = 'Executed in {:,.3f} secs.')

Bases: object

A Context Manager Interface for In-Line Time Calculation

The wrappers provided under pandaswizard are great way to print execution time of a function - however, this lacks the capability to calculate time for in-line python functions like for example:

import pandas as pd

# using wrapper approach to read csv and timeit
@pdw.wrappers.timeit
def read_csv_(*args, **kwargs) -> pd.DataFrame:
    return pd.read_csv(*args, **kwargs)

However, this can be further enhanced and thus reducing time to develop wrappers interfaces by using context managers lile:

with CatchTime() as T:
    data = pd.read_csv(...)

# another use case, using the context manager for apply
with CatchTime():
    data["calculation"] = data["column"].apply(...)

The context manager thus can act as a bridge for in-line function application and print time.

Parameters:: fmt (str) – Output format string with prefix is in-built, this is a new addition to control the format from outside the scope of the context manager.

property elapsed: float

Returns the Current Elapsed Time from the Function Start

The .elapsed property returns the in-between elapsed time from the context manager’s scope or can be used to store the time at certain levels for time optimization.

When executed from the context manager’s scope the function returns the intermediate time, or returns the final executed time when the context manager is closed.

with CatchTime() as t:
    # do something
    print("Elapsed Time: {t.elapsed:.3f}")
    # do something else
    print("Elapsed Time: {t.elapsed:.3f}")

# the final execution time can be stored or printed
total_exec_time = t.elapsed