Modification in Action Delete API

StackStorm offers functionality to delete actions/workflows by invoking API. Prior to 3.6 release this API was only de-registering actions from database. In the 3.6 release, with backward compatibility retained this API was modified to optionally delete related action files from disk as well.

From CLI st2 action delete <pack>.<action> will delete only action database entry. From CLI st2 action delete --remove-files <pack>.<action> or st2 action delete -r <pack>.<action> will delete action database entry along with files from disk.

API action DELETE method with {"remove_files": true} argument in json body will remove database entry of action along with files from disk. API action DELETE method with {"remove_files": false} or no additional argument in json body will remove only action database entry.

• Usage:

st2 action delete [-h] [-t TOKEN] [--api-key API_KEY] [-j] [-y] [-r]
                  ref-or-id

• Positional arguments:

# Reference or ID of the action
ref-or-id

• Optional arguments:

-h, --help            show this help message and exit
-t TOKEN, --token TOKEN
                      Access token for user authentication. Get
                      ST2_AUTH_TOKEN from the environment variables by
                      default
--api-key API_KEY     Api Key for user authentication. Get ST2_API_KEY from
                      the environment variables by default
-j, --json            Print output in JSON format
-y, --yaml            Print output in YAML format
-r, --remove-files    Delete action files from disk

Cloning an Action

Clone operation clones action from source pack to destination pack and renames them appropriately. This operation makes changes in destination action metadata files like action name, pack name, entry point name etc. Also, this operation registers the newly cloned action to the database.

API endpoint /api/v1/actions/{ref_or_id}/clone takes ref_or_id of source action. Request method body {"dest_pack": "dest_pack_name", "dest_action": "dest_action_name"} takes destination pack and action name. Request method body also takes optional paramater overwrite which is of type boolean and false by default. Overwrite flag should be true (overwrite: true) in case of destination action already exists and to be overwritten.

From CLI st2 action clone <ref_or_id> <dest_pack> <dest_action> takes source ref_or_id, destination pack name and destination action name as mandatory arguments. In case destionation already exists then command takes optional arugument -f or --force to overwrite destination action.

• Usage:

st2 action clone [-h] [-t TOKEN] [--api-key API_KEY] [-j] [-y] [-f]
                 source-ref-or-id dest-pack-name dest-action-name

• Positional arguments:

# Reference or ID of the source action
source-ref-or-id

# Destination pack name
dest-pack-name

# Destination action name
dest-action-name

• Optional arguments:

-h, --help            Show this help message and exit
-t TOKEN, --token TOKEN
                      Access token for user authentication. Get
                      ST2_AUTH_TOKEN from the environment variables by default
--api-key API_KEY     Api Key for user authentication. Get ST2_API_KEY from
                      the environment variables by default
-j, --json            Print output in JSON format
-y, --yaml            Print output in YAML format
-f, --force           Overwrite action files on disk if destination exists.

Available Runners

The environment in which the action runs is specified by the runner. Currently the system provides the following runners:

1. local-shell-cmd - This is the local runner. This runner executes a Linux command on the host where StackStorm is running.

2. local-shell-script - This is the local runner. Actions are implemented as scripts. They are executed on the hosts where StackStorm is running.

3. remote-shell-cmd - This is a remote runner. This runner executes a Linux command on one or more remote hosts provided by the user.

4. remote-shell-script - This is a remote runner. Actions are implemented as scripts. They run on one or more remote hosts provided by the user.

5. python-script - This is a Python runner. Actions are implemented as Python classes with a run() method. They run locally on the same machine where StackStorm components are running. The return value from the action run() method is either a tuple of success status flag and the result object respectively or it is just the result object. For more information, please refer to the Action Runners section in the documentation.

6. http-request - HTTP client which performs HTTP requests for running HTTP actions.

7. action-chain - This runner supports executing simple linear work-flows. For more information, please refer to the Workflows and ActionChain documentation.

8. inquirer - This runner provides the core logic of the Inquiries feature.

   Note: This runner is an implementation detail for the core.ask action, and in most cases should not be referenced in other actions.

9. winrm-cmd - The WinRM command runner allows you to run the command-line interpreter (cmd) commands on Windows hosts using the WinRM protocol.

10. winrm-ps-cmd - The WinRM PowerShell command runner allows you to run the PowerShell commands on Windows hosts using the WinRM protocol.

11. winrm-ps-script - WinRM PowerShell script runner allows you to run PowerShell scripts on Windows hosts.

12. orquesta - This runner supports executing complex work-flows. For more information, please refer to the Workflows and Orquesta documentation.

Runners come with their own set of input parameters. When an action is executed, it inherits the runner parameters, in addition to its own parameters. The built-in parameters can be over-ridden on a per-action basis.

For a complete list of Runners and their parameters, see Action Runners.

Action Metadata

Action metadata is used to describe the action and is defined as YAML. These attributes can be present in the metadata file:

• name - Name of the action.

• runner_type - The type of runner to execute the action.

• enabled - Action cannot be invoked when disabled.

• entry_point - Location of the action launch script relative to the /opt/stackstorm/packs/${pack_name}/actions/ directory.

• parameters - A dictionary of parameters and optional metadata describing type and default. The metadata is structured data following the JSON Schema specification draft 4. The common parameter types allowed are string, boolean, number (whole numbers and decimal numbers - e.g. 1.0, 1, 3.3333, etc.), object, integer (whole numbers only - 1, 1000, etc.) and array. If metadata is provided, input args are validated on action execution. Otherwise, validation is skipped.

• tags - An array with tags for this actions for the purpose of providing supplemental information

This is a sample metadata file for a Python action which sends an SMS via the Twilio web service:

---
name: "send_sms"
runner_type: "python-script"
description: "This sends an SMS using twilio."
enabled: true
entry_point: "send_sms.py"
parameters:
    from_number:
        type: "string"
        description: "Your twilio 'from' number in E.164 format. Example +14151234567."
        required: true
        position: 0
    to_number:
        type: "string"
        description: "Recipient number in E.164 format. Example +14151234567."
        required: true
        position: 1
        secret: true
    body:
        type: "string"
        description: "Body of the message."
        required: true
        position: 2
        default: "Hello {% if system.user %} {{ st2kv.system.user }} {% else %} dude {% endif %}!"

This action is using a Python runner (python-script). The class that implements a run method is contained in a file called send_sms.py which is located in the same directory as the metadata file. The action takes three parameters (from_number, to_number, body).

In the example above, the to_number parameter contains the attribute secret with value: true. If an attribute is marked as a secret, the value of that attribute will be masked in the StackStorm service logs.

Output Schema

StackStorm supports modeling the output of an action or runner via the output_schema key in the action definition. Action developers can create explicit, typed outputs for their actions. This aids in workflow development and error handling.

For example, you have a python action that returns three keys: errors, output, and status_code. error should be a list of strings, output should be a list of floats, and status_code should be an integer. errors is optional, whilst the others must be returned. You can define the schema as follows:

---
...
output_schema:
  type: object
  properties:
    errors:
      type: array
      items:
        type: string
    output:
      required: true
      type: array
      items:
        type: number
    status_code:
      required: true
      type: integer
  additionalProperties: false

If the action output does not return the correct fields it will fail validation and the action itself will fail. This prevents propagating corrupt data to other actions in a workflow, which could lead to unpredictable results. In future this information will be used for pre-run workflow validation.

Output schema validation is disabled by default in current versions of StackStorm. To enable it, set validate_output_schema = True under [system] in /etc/st2/st2.conf.

If an action does not define any output schema, no enforcement is done. This allows you to progressively update your actions, rather than doing them all at once.

As with all other input and output schema definitions in Stackstorm, we leverage JSONschema (draft 4) to define output_schema. We extend JSONSchema with a secret parameter so that the entire output, or a single field of the output, can be marked as secret. If any part of the output is marked as a secret, the value of that secret will be masked in the StackStorm service logs.

For example, you have a python action that returns a secret token as a string. You can define the schema as follows:

---
...
output_schema:
  type: string
  secret: true

Or the python action could return the secret token as an object field:

---
...
output_schema:
  type: object
  properties:
    super-awesome-token:
      type: string
      secret: true
  additionalProperties: false

Parameters in Actions

In the previous example, you probably noticed how you can access parameters from the key-value store by using the st2kv.system prefix in the template. You can also get access to variables from the context of the execution. For example:

parameters:
  user:
    type: "string"
    description: "User of this action."
    required: true
    default: "{{action_context.api_user}}"

The prefix action_context is used to refer to variables in action context. Depending on how the execution is executed and nature of action (simple vs workflow), variables in action_context change.

A simple execution via the API will only contain the variables user and pack. An execution triggered via ChatOps will contain variables such as api_user, user, pack, and source_channel. In the ChatOps case, api_user is the user who entered the ChatOps command from the Chat client and user is the StackStorm user configured in hubot. source_channel is the channel in which the ChatOps command was entered.

In addition to action_context you can also access config_context which contains the key-value contents of your pack configuration. The example below shows how you could use this for the default value for a parameter:

---
name: "send_sms"
runner_type: "python-script"
description: "This sends an SMS using twilio."
enabled: true
entry_point: "send_sms.py"
parameters:
    from_number:
        type: "string"
        description: "Your twilio 'from' number in E.164 format. Example +14151234567."
        required: false
        position: 0
        default: "{{config_context.from_number}}"
    to_number:
        type: "string"
        description: "Recipient number in E.164 format. Example +14151234567."
        required: true
        position: 1
        secret: true
    body:
        type: "string"
        description: "Body of the message."
        required: true
        position: 2
        default: "Hello {% if system.user %} {{ st2kv.system.user }} {% else %} dude {% endif %}!"

In ActionChains and workflows (see Workflow), every task in the workflow can access the parent’s execution_id. For example, a task in an action chain is shown below:

...
-
  name: "c2"
  ref: "core.local"
  parameters:
    cmd: "echo \"c2: parent exec is {{action_context.parent.execution_id}}.\""
  on-success: "c3"
  on-failure: "c4"
...

Action Registration

To register a new action:

1. Place it into the pack content location.

2. Tell the system that the action is available.

The actions are grouped in packs and located at /opt/stackstorm/packs

For hacking one-off actions, the convention is to use the default pack - just create your action in /opt/stackstorm/packs/default/actions. Once you have tested it out, you should move it to a dedicated pack.

Register an individual action by calling st2 action create my_action_metadata.yaml. To reload all actions, use st2ctl reload --register-actions

1. Make sure the script conforms to conventions

You should make sure that the script exits with a zero status code on success and non-zero on error. This is important since the exit code is used by StackStorm to determine if the script has finished successfully.

2. Create a metadata file

You need to create a metadata file which describes the script name, description, entry point, which runner to use and script parameters (if any).

When converting an existing script, you will want to use either the local-shell-script or remote-shell-script runner.

2. Update argument parsing in the script

Local and remote script runners recognize two types of parameters:

1. named - those parameters do not include the position attribute.

2. positional - those parameters include the position attribute.

All parameters are passed to the script via command-line arguments.

Named argument are passed to the script in the following format:

script.sh --param1=value --param2=value --param3=value

By default, each parameter is prefixed with two dashes (--). If you want to use a single dash (-), some other prefix or no prefix at all, you can configure that using the kwarg_op parameter in the metadata file.

For example:

---
name: "my_script"
runner_type: "remote-shell-script"
description: "Script which prints arguments to stdout."
enabled: true
entry_point: "script.sh"
parameters:
    key1:
        type: "string"
        required: true
    key2:
        type: "string"
        required: true
    key3:
        type: "string"
        required: true
    kwarg_op:
        type: "string"
        immutable: true
        default: "-"

In this case, arguments are passed to the script in the following format:

script.sh -key1=value1 -key2=value2 -key3=value3

Positional arguments are passed to the script ordered by the position value in the following format:

script.sh value2 value1 value3

If your script only uses positional arguments (which is often the case for existing scripts), you simply need to declare parameters with the correct value for the position attribute in the metadata file. Positional arguments are serialized based on the simple rules described below:

1. string, integer, float - Serialized as a string.

2. boolean - Serialized as a string 1 (true) or 0 (false).

3. array - Serialized as a comma delimited string (e.g. foo,bar,baz).

4. object - Serialized as JSON.

Using this simple serialization format allows users to easily utilize those values in their scripts by using standard Bash functionality (-z for check if a value is provided, -eq for comparison to 1/0 and IFS for splitting a string into an array). For working with objects, you can use a tool such as jq.

If no value is provided for a particular positional parameter, StackStorm will pass an empty string "" as a value for that parameter to the script.

For example, if a second positional parameter is optional and user provides no value, the script will be called like this:

script.sh value1 "" value3

The immutable value defines whether the default value of a parameter can be overridden. This is particularly important if you expose commands via ChatOps and do not like security related parameters to be manipulated by user input.

Example 1 - existing Bash script with positional arguments

Let’s say we have a simple Bash script named send_to_syslog.sh which writes the message provided via the command line argument to syslog.

The script takes two arguments:

1. The address of the syslog server

2. The message to write

#!/usr/bin/env bash

SERVER=$1
MESSAGE=$2
logger -n ${SERVER} ${MESSAGE}

Since this script is using positional arguments, you only need to define them in the metadata file:

---
name: "send_to_syslog.log"
runner_type: "remote-shell-script"
description: "Send a message to a provided syslog server."
enabled: true
entry_point: "send_to_syslog.sh"
parameters:
    server:
        type: "string"
        description: "Address of the syslog server"
        required: true
        position: 0
    message:
        type: "string"
        description: "Message to write"
        required: true
        position: 1

As you can see above, we declare two parameters - server and message. Both of them declare a position attribute (0 for server and 1 for message), which means they will be passed to the action script as positional arguments, so your script doesn’t require any changes.

Sample Python Action

Metadata file (my_echo_action.yaml):

---
name: "echo_action"
runner_type: "python-script"
description: "Print message to standard output."
enabled: true
entry_point: "my_echo_action.py"
parameters:
    message:
        type: "string"
        description: "Message to print."
        required: true
        position: 0

Action script file (my_echo_action.py):

import sys

from st2common.runners.base_action import Action

class MyEchoAction(Action):
    def run(self, message):
        print(message)

        if message == 'working':
            return (True, message)
        return (False, message)

This Python action prints text provided via the message parameter to the standard output. As you can see, user-supplied action parameters are passed to the run method as keyword arguments.

If the run method finishes without exceptions, the execution is successful, and the value returned by the method (any value: boolean, string, list, dict, etc.) is considered its result. Raising an exception will report the execution as failed.

Another way to specify the status of an execution is returning a tuple with two items: the first item is a boolean indicating status, the second item is the result itself.

For example, return False will result in a successful execution with the result being False, and return (False, "Failed!") is a failed execution with "Failed!" as its result.

In the example above, if the message parameter passed to the action is working, the action will be considered as succeeded (first flag in the result indicating action status is True). If another message is passed in, action will be considered as failed (first flag in the result tuple indicating action status is False).

For a more complex example, please refer to the actions in the Libcloud pack in StackStorm Exchange.

Configuration File

Python actions can store arbitrary configuration in the configuration file which is global to the whole pack. The configuration is stored in a file named <pack_name>.yaml, in the /opt/stackstorm/configs/ directory.

The configuration file format is YAML. Configuration is automatically parsed and passed to the action class constructor via the config argument. See the pack configuration doc for more details.

Logging

All logging inside the action should be performed via the logger which is specific to this action and available via the self.logger class attribute.

This logger is a standard Python logger from the logging module so all the logger methods work as expected (e.g. logger.debug, logger.info, etc).

For example:

def run(self):
    ...
    success = call_some_method()

    if success:
        self.logger.info('Action successfully completed')
    else:
        self.logger.error('Action failed...')

Action Service

Similar to sensors, action_service is available on each action instance after instantiation.

Action service provides different services to the action via public methods. Right now it supports datastore management methods. This allows actions to utilize the datastore to store arbitrary data between executions.

The action service provides the same datastore management methods as the ones available on the sensor service. You can find more details in the sensor datastore management documentation.

Example storing a dict as JSON:

def run(self):
  data = {'somedata': 'foobar'}

  # Add a value to the datastore
  self.action_service.set_value(name='cache', value=json.dumps(data))

  # Retrieve a value
  value = self.action_service.get_value('cache')
  retrieved_data = json.loads(value)

  # Retrieve an encrypted value
  value = self.action_service.get_value('ma_password', decrypt=True)
  retrieved_data = json.loads(value)

  # Delete a value
  self.action_service.delete_value('cache')

Sharing code between Python Sensors and Actions

Refer to documentation on sharing common code between python actions and sensors.