Operation Node

An OperationNode represents an operation that executes in SystemDS. Most methods are overloaded for OperationNode. This means that they return an OperationNode.

To get the result from an OperationNode you simply call .compute() on it, thereby getting the numpy or pandas equivalent result.

class systemds.operator.OperationNode(sds_context: SystemDSContext, operation: str, unnamed_input_nodes: str | Iterable[DAGNode | str | int | float | bool] = None, named_input_nodes: Dict[str, DAGNode | str | int | float | bool] = None, is_python_local_data: bool = False, brackets: bool = False, is_datatype_unknown: bool = False, is_datatype_none: bool = True)

A Node representing an operation in SystemDS

__init__(sds_context: SystemDSContext, operation: str, unnamed_input_nodes: str | Iterable[DAGNode | str | int | float | bool] = None, named_input_nodes: Dict[str, DAGNode | str | int | float | bool] = None, is_python_local_data: bool = False, brackets: bool = False, is_datatype_unknown: bool = False, is_datatype_none: bool = True)

Create general OperationNode

Parameters:

• sds_context – The SystemDS context for performing the operations

• operation – The name of the DML function to execute

• unnamed_input_nodes – inputs identified by their position, not name

• named_input_nodes – inputs with their respective parameter name

• is_python_local_data – if the data is local in python e.g. Numpy arrays that this operation node returns multiple values. If set remember to set the output_types value as well.

code_line(var_name: str, unnamed_input_vars: Sequence[str], named_input_vars: Dict[str, str]) → str

Generates the DML code line equal to the intended action of this node.

Parameters:

• var_name – Name of DML-variable this nodes result should be saved in

• unnamed_input_vars – all strings representing the unnamed parameters

• named_input_vars – all strings representing the named parameters (name value pairs)

Returns:

the DML code line that is equal to this operation

compute(verbose: bool = False, lineage: bool = False) → float | array | Tuple[float | array, str]

Get result of this operation. Builds the dml script and executes it in SystemDS, before this method is called all operations are only building the DAG without actually executing (lazy evaluation).

Parameters:

• verbose – Can be activated to print additional information such as created DML-Script

• lineage – Can be activated to print lineage trace till this node

Returns:

the output as an python builtin data type or numpy array

get_lineage_trace() → str

Get the lineage trace for this node.

Returns:

Lineage trace

pass_python_data_to_prepared_script(jvm: JVMView, var_name: str, prepared_script: JavaObject) → None

Passes data from python to the prepared script object.

Parameters:

• jvm – the java virtual machine object

• var_name – the variable name the data should get in java

• prepared_script – the prepared script

print(**kwargs: Dict[str, DAGNode | str | int | float | bool]) → OperationNode

Prints the given Operation Node. There is no return on calling. To get the returned string look at the stdout of SystemDSContext.

write(destination: str, format: str = 'binary', **kwargs: Dict[str, DAGNode | str | int | float | bool]) → OperationNode

Write input to disk. The written format is easily read by SystemDSContext.read(). There is no return on write.

Parameters:

• destination – The location which the file is stored. Defaulting to HDFS paths if available.

• format – The format which the file is saved in. Default is binary to improve SystemDS reading times.

• kwargs – Contains multiple extra specific arguments, can be seen at http://apache.github.io/systemds/site/dml-language-reference#readwrite-built-in-functions