Chain

A chain is a set of processes with its own properties like scheduling, notifications, dependencies, outputs, etc.

This is an example of a basic chain with one process:

{
  "chains": [
    {
      "id": "EXAMPLE-CHAIN",
      "name": "Sample chain name",
      "namespace": "SAMPLES",
      "triggers": [],
      "depends_chains": [],
      "processes": [
        {
          "id": "PROCESS-ONE",
          "name": "First chain process",
          "exec": {
            "id": "shell_default",
            "command": "echo 'Hello world'"
          }
        }
      ]
    }
  ]
}

Runnerty has a default unattended point of view for chains and processes managment. This means that unless you specify a modification propetie for a process or chain it will try to run the next process or the chain until the end. More details below in the action on failed chain section here.

Identification

A chain has two identification fields:

id -> Which identifies the chain. With this identifier Runnerty will provide a global value CHAIN_ID that can be used in the whole chain. To learn more about global values have a look here.

{
  "id": "EXAMPLE-CHAIN"
}

name -> It is a description of the chain. Runnerty will also provide the global value CHAIN_NAME

{
  "id": "EXAMPLE-CHAIN",
  "name": "Sample chain name"
}

Execution (Triggers)

A chain can be fired by using triggers. There is a bunch of different triggers, have a look at them and how to use them here

Custom values

It is possible to define and overwrite global values at chain level, setting a custom_values attribute:

{
  "id": "EXAMPLE-CHAIN",
  "name": "Sample chain name",
  "custom_values": {
    "YYYY": "1986",
    "MY_LOCAL_CHAIN_VALUE": "ABC"
  }
}

Dependencies

You can define dependencies with other chains or other chain processes through the depends_chains property. This allows us to execute a chain in these cases:

• When the chain it depends on ends.

{
  "depends_chains": ["CHAIN-ONE"]
}

The chain will be executed when ´CHAIN-ONE´ finishes

{
  "depends_chains": ["CHAIN-ONE", "CHAIN-TWO"]
}

The chain will be executed when CHAIN-ONE or CHAIN-TWO finishes

• When the process of the chain it depends on is ends. We will also receive in the input of the chain the values that are indicated in the output_share of the process. This dependency is similar to the one used in iterable chains.

{
  "depends_chains": [
    {
      "chain_id": "CHAIN-ONE",
      "process_id": "PROCESS-ONE"
    }
  ]
}

The chain will be executed when PROCESS-ONE of CHAIN-ONE finishes

This will cause the processes that depend on the process to wait for the completion of the chain or chains dependent on it. Once the process chain is finished, its flow will continue normally. It is also possible to make the reference process chain fail if the dependent chain fails, by using the fail_on_child_fail property (default is false) in the reference process.

{
  "processes": [
    {
      "id": "PROCESS-ONE",
      "name": "PROCESS-ONE of CHAIN-ONE",
      "fail_on_child_fail": true,
      //"...":"..."
    }
]

tip

If you need to avoid that two chains can be executed at the same time make use of the queues

Notifications

With the notifications property, Runnerty can be set up to emit notifications during the chain status flow, fired up by the following callbacks:

• on_start
• on_fail
• on_end
• on_retry
• on_queue

In these notifications we could notify anything using notifiers plugins.

The following example shows how to set up notifications for the different states of the chain through Telegram Notifier, publishing messages to a previously defined Telegram's chatroom:

{
  "notifications": {
    "on_start": [
      {
        "id": "telegram_default",
        "message": "THE CHAIN @GV(CHAIN_ID) HAS STARTED"
      }
    ],
    "on_fail": [
      {
        "id": "telegram_default",
        "message": "THE CHAIN @GV(CHAIN_ID) HAS FAILED"
      }
    ],
    "on_end": [
      {
        "id": "telegram_default",
        "message": "THE CHAIN @GV(CHAIN_ID) HAS FINISHED"
      }
    ],
    "on_retry": [
      {
        "id": "telegram_default",
        "message": "THE CHAIN @GV(CHAIN_ID) HAS RETRY"
      }
    ],
    "on_queue": [
      {
        "id": "telegram_default",
        "message": "THE CHAIN @GV(CHAIN_ID) HAS QUEUE"
      }
    ]
  }
}

note

The usage of the global value and function @GV(CHAIN_ID) on the previous example. This value will be replaced with the chain's id. Learn more about global values here and more about functions here

Learn more about notifiers and how to configure them here.

Processes

In the processes array property we can defined all the processes that are going to be part of the chain.

Learn more about processes and how to configure them here.

{
  "processes": [
    {
      "id": "PROCESS-ONE",
      "name": "First chain process",
      "exec": {
        "id": "shell_default",
        "command": "echo 'Hello world'"
      }
    }
  ]
}

Actions for a chain when a process fails

It is possible to define what action (abort or retry) to perform at the chain level in case a process fails. The number of retries and delay (retry_delay) settings will be set at the chain level.

{
  "id": "CHAIN_SAMPLE",
  "name": "Chain with retries",
  "namespace": "SAMPLES",
  "retries": 1,
  "retry_delay": "1 min",
  //...
  "processes": [
    {
      "id": "SAMPLE-PROCESS",
      "chain_action_on_fail": "retry"
      //...
    }
  ]
}

Abort the chain if the process fails (this action ends the chain's flow so no other processes will be executed). It is not necessary to indicate, is the default value:

{
  //...
  "processes": [
    {
      "id": "SAMPLE-PROCESS",
      "chain_action_on_fail": "abort"
      //...
    }
  ]
}

Also you can skip this option (chain_action_on_fail) so though any process failed, the chain will continue while the dependencies between processes are met (by using depends_process propertie).

Also, it is possible to indicate in the process that the execution continues even though an error occurs:

{
  //...
  "processes": [
    {
      "id": "SAMPLE-PROCESS",
      "chain_action_on_fail": "continue"
      //...
    }
  ]
}

This will cause the process error to be reported but the string continue and end without error. Additionally, it could force the chain to end with an error indicating that the process error is taken into account for the final state of the chain:

{
  //...
  "processes": [
    {
      "id": "SAMPLE-PROCESS",
      "chain_action_on_fail": "continue",
      "ignore_in_final_chain_status": false
      //...
    }
  ]
}

Delay property understands the following strings:

• x milliseconds
• x millisecond
• x msecs
• x msec
• x ms
• x seconds
• x second
• x secs
• x sec
• x s
• x minutes
• x minute
• x mins
• x min
• x m
• x hours
• x hour
• x hrs
• x hr
• x h
• x days
• x day
• x d
• x weeks
• x week
• x wks
• x wk
• x w
• x years
• x year
• x yrs
• x yr
• x y

The space after the number is optional so you can also write 1ms instead of 1 ms. In addition, it also accepts numbers and strings which only includes numbers and we assume that these are always in milliseconds.

From: [Millisecond module](https://github.com/unshiftio/millisecond)

Iterable chains

An iterale chain is a chain that is going to be executed for each object in the array previously returned by another process.

For example, if we have a process which returns one objects array we can execute an iterable chain for each object in the array.

In the following example we are going to send an email to every user of the USERS table.

First, we have the chain get-users-email.json with a process which selects all the users's email:

{
  "id": "GET-USERS-EMAIL",
  "name": "It gets all the user's names to send an email",
  "triggers": [
    {
      "id": "schedule_default",
      "schedule_interval": "1 */1 * * *"
    }
  ],
  "processes": [
    {
      "id": "GET-USER-EMAIL",
      "name": "it gets all the users email from the database",
      "exec": {
        "id": "mysql_default",
        "command": "SELECT email, name FROM USERS"
      },
      "output_iterable": "PROCESS_EXEC_DATA_OUTPUT"
    }
  ]
}

Then, we assign the returned resultset by the MySQL SELECT query as an object array with the PROCESS_EXEC_DATA_OUTPUT value, as part of the property output_iterable. This way we are announcing this process will return an iterable output.

Now we are going to define the iterable chain "send-mail-to-user"

{
  "id": "SEND-MAIL-TO-USERS",
  "name": "it sends an email to the users returned",
  "namespace": "SAMPLES",
  "retries": 1,
  "retry_delay": "1 min",
  "depends_chains": {
    "chain_id": "GET-USERS-EMAIL",
    "process_id": "GET-USER-EMAIL"
  },
  "iterable": "parallel",
  "input": [
    {
      "email": "email"
    },
    {
      "name": "name"
    }
  ],
  "processes": [
    {
      "id": "SEND-MAIL",
      "name": "sends the email to the user",
      "exec": {
        "id": "mail_default",
        "to": [":email"],
        "message": "Hello :name",
        "title": "Message set by Runnerty"
      },
      "chain_action_on_fail": "retry"
    }
  ]
}

Here we can see some properties that the chain needs to iterate. First of all we have the dependencies on the depends_chains property.

info

An iterable chain must depends on the process from the "parent chain" whom it iterates.

{
  "depends_chains": {
    "chain_id": "GET-USERS-EMAIL",
    "process_id": "GET-USER-EMAIL"
  }
}

With the iterable property we can choose if we want to iterate over in series or parallel:

{
  "depends_chains": {
    "chain_id": "GET-USERS-EMAIL",
    "process_id": "GET-USER-EMAIL"
  },
  "iterable": "parallel"
}

With the input property we can assign the properties of each object returned by de mother's process array.

{
  "depends_chains": {
    "chain_id": "GET-USERS-EMAIL",
    "process_id": "GET-USER-EMAIL"
  },
  "iterable": "parallel",
  "input": [
    {
      "email": "email"
    },
    {
      "name": "name"
    }
  ]
  //...
}

Now, we can use these values anywhere in our iterable chain:

{
  "processes": [
    {
      "id": "SEND-MAIL",
      "name": "Sends the email to the user",
      "exec": {
        "id": "mail_default",
        "to": ["@GV(email)"],
        "message": "Hello @GV(name)",
        "title": "Message send by Runnerty"
      }
    }
  ]
}

In the example :email will be replaced with the user's email and :name will be replaced with the user's name.

Default properties for processes

It is possible to define a default value for all the processes in a chain of notifications,output and chain_action_on_fail, depending on the defaults_processes property of a chain.

For example:

{
      "id": "CHAIN_SAMPLE",
      "name": "CHAIN_SAMPLE",
      "defaults_processes": {
        "notifications": {
          "on_start": [
            {
              "id": "console_default",
              "message": "PROCESS @GV(PROCESS_ID) START"
            }
          ],
          "on_fail": [
            {
              "id": "console_default",
              "message": "ERR! PROCESS @GV(PROCESS_ID) FAIL: @GV(PROCESS_EXEC_ERR_OUTPUT)"
            }
          ],
          "on_end": [
            {
              "id": "console_default",
              "message": "PROCESS @GV(PROCESS_ID) END"
            }
          ]
        },
        "output": [
          {
            "file_name": "./test.log",
            "write": [
              "@GETDATE('YYYY-MM-DD HH:mm:sS') - @GV(CHAIN_ID)/@GV(PROCESS_ID)/@GV(PROCESS_EXEC_COMMAND_EXECUTED)\n"
            ],
            "concat": true,
            "maxsize": "10mb"
          }
        ],
        "chain_action_on_fail": "abort"
      },
      "processes": [
        //...
      ]

It is also possible to overwrite the default values (defaults_processes) in each of the processes. For example, in this case the default value of the on_start event of notifications is overwritten in the PROCESS_SAMPLE process, the rest of the notifications values will be those defined by default:

{
      "id": "CHAIN_SAMPLE",
      "name": "CHAIN SAMPLE",
      "defaults_processes": {
        "notifications": {
          "on_start": [
            {
              "id": "console_default",
              "message": "PROCESS @GV(PROCESS_ID) START"
            }
          ],
          "on_fail": [
            {
              "id": "console_default",
              "message": "ERR! PROCESS @GV(PROCESS_ID) FAIL: @GV(PROCESS_EXEC_ERR_OUTPUT)"
            }
          ]
        }
      },
      "processes": [
        {
          "id": "PROCESS_SAMPLE",
          "name": "PROCESS SAMPLE",
          "exec": {
            "id": "shell_default",
            "command": "echo hello world"
          },
          "notifications": {
            "on_start": [
              {
                "id": "console_default",
                "message": "OVERRIDE: PROCESS @GV(PROCESS_ID) START"
              }
            ]
          }
        }]

Metadata

These are the metadata that we can indicate from a chain.

Metadata	Description
author	Author
team	Team to which it belongs
criticality	Criticality level: low, medium or high
contacts	Array of objects with contact data: name, email and phone
fail_comments	Comment on action in case the chain fails
relaunchable	Indicates if the chain is ready to be relaunched
extra_id	String to indicates a key value relevant to distinguishing string executions. Especially useful for distinguishing on the platform between the different executions in iterable chains or when we need to distinguish the executions by their input values.
keywords	Array of strings: list of keywords
max_expected_time	Maximum expected time. It will be used in Runnerty IO Platform to alert on chains that exceed it. The value can be specified in string format ("1000 ms", "1 s", "30 min", "1d" ...)

Example:

{
  "id": "BACK_TO_THE_FUTURE",
  "name": "Chain to back to the future",
  "namespace": "TRAVELS",
  "meta": {
    "author": "Marty McFly",
    "team": "TIMELINES",
    "criticality": "high",
    "contacts": [
      {
        "name": "Dr. Emmett Lathrop Brown",
        "email": "doc@backtothefuture.com",
        "phone": "880000000"
      }
    ],
    "fail_comments": "Check flux capacitor and relaunch!",
    "relaunchable": true,
    "keywords": ["TRAVEL", "TIME"]
  }
  //...
}