Callable

A Callable is the base class representing a function available in a Pipeline.

Definition

Constructor

class controller.callable.Callable(self, type: CallableType, id: str, params: CallableParameters)

The base abstract class representing a function available for execution in a Pipeline.

Parameters:

  • type (CallableType) – The type of this Callable.

  • id (str) – The unique identifier of this Callable. Also referred to as callable_id in documentations for specific Callables.

  • params (CallableParameters) – The parameters of this Callable. Refer to the documentation for a specific Callable for details.

Methods

abstract controller.callable.Callable.check_params(self, params: CallableParameters) → boolean

Function parameters validation logic. This function should contain code that ensures all passed parameters are valid values. Overriden by subclasses to implement the actual parameter validation logic.

abstract controller.callable.Callable.exec(self, data: pd.DataFrame, **kwargs) → pd.DataFrame

Function call logic. This function should contain the actual execution logic that will result in some transformed data. Overriden by subclasses to implement the actual execution logic.

Parameters:

  • data (pd.DataFrame) – Known as the target DataFrame, which contains data to be transformed. All operations in the callable implementation are done on this DataFrame. This is injected at runtime and is not exposed during Callable instantiation.

  • kwargs – Additional keyword arguments that may be passed to the Callable. This is injected at runtime as well using entries in the instance variable params.

Returns:

Known as the output DataFrame, which is the result of the execution.

Usage

To implement new Callables, refer to the documentation for specific Callable types listed below.

Output conventions

The output of a Callable is the return value of its exec() method.

All Callables must return a DataFrame, known as the output DataFrame, that contains all data required for subsequent Callables to use.

  • The DataFrame is headered (i.e. columns are named).

    • The column name must start with the id followed by a period. That is, the column name must start with <callable_id>.

    • Further restrictions on column names may be specified in individual Callables. Consult their documentations for more information.

  • The DataFrame is single-indexed. The pipeline runner does not support aggregation of other types of DataFrames, such as multi-index DataFrames.

    • It is up to individual Callables to specify how multi-level DataFrames returned from functions are flattened such that the output DataFrame is single-indexed.

Types of Callables

Callables are separated into different types. Each type specifies the nature of what operations a Callable is responsible for and governs the format of output DataFrame of that Callable.

List of Callables

Note

Callable instances are not supposed to be instantiated using the constructor, but rather through methods provided in the Pipeline API. As a result, the Definition section for a specific Callable does not document the constructor or the internal implementation. Rather, it describes the parameters that are required to configure the Callable when it is instantiated. The signature in the documentation may be considered to have unpacked the CallableParameters object into keyword parameters. This is intentional since the generic CallableParameters object (inherited from the base Callable class) is only unpacked and passed to the exec method when the Callable is executed.

In short, you may refer to both the Definition and Parameters sections, when instantiating a Callable via the Pipeline API. Do not treat the documentation to be describing the Callable constructor.

Extractors

Transformers

Outputters

Other Callable objects