Command Templates

For security purposes, HEAppE enables the users to run only a pre-prepared set of so-called Command Templates. Each template defines a script or executable binary, that will be executed on a specific HPC cluster queue to which the template is assigned (any dependencies or third-party software it might require). There are generally two kinds of these templates:

Static Command Template

The command template contains the set of input parameters (parameters are exposed in the API) that will be passed to the executable script during the run-time. Thus, the users can execute pre-prepared command templates with the pre-defined set of input parameters. The actual value of each parameter (input from the user) can be different for each job submission. The following block presents the static command template configuration in the HEAppE Middleware part. In the HPC part, must exist the corresponding test script with one inputParam parameter located at ~/.key_scripts/test.sh.

Static command template is configurable via the HEAppE Management API endpoints.

Configuration of the static command template also can be done by specifying the following objects at the seed.njson file:
Example of static command template definition in seed.njson.
  "CommandTemplates":[
    {
      "Id": 1,
      "Name": "TestTemplate",
      "Description": "TestTemplate",
      "Code": "TestTemplate",
      "ExecutableFile": "~/.key_scripts/test.sh",
      "CommandParameters": "%%{inputParam}",
      "PreparationScript": null,
      "ClusterNodeTypeId": 2,
      "IsGeneric": false,
      "IsEnabled": true
    }
  ],
  "CommandTemplateParameters": [
    {
      "Id": 1,
      "Identifier": "inputParam",
      "Query": "",
      "Description": "inputParam",
      "CommandTemplateId": 1,
      "IsVisible": true
    }
  ]

Generic Command Template

The Generic Command Template in HEAppE Middleware is designed to facilitate prototyping and the creation of new Static Command Templates. It allows users to execute a bash script with dynamically defined parameters at runtime.

The Generic Command Template primarily exposes the userScriptPath parameter.

Other parameters are defined in the bash script used by the template and specified with #HEAPPE_PARAM.

Users must be familiar with all input parameters defined in the script they reference when creating a job using the HEAppE API.

The Robot Account submitting jobs via the Generic Command Template must have execution rights for the script referenced in userScriptPath.

The following JSON snippet shows how to configure the Generic Command Template in the HEAppE Middleware configuration file (seed.njson):
Example of generic command template definition in seed.njson.
"CommandTemplates":[
  {
    "Id": 1,
    "Name": "GenericCommandTemplate",
    "Description": "Command template for generic job.",
    "Code": "GenericCommandTemplate",
    "ExecutableFile": "~/.key_scripts/generic.sh",
    "CommandParameters": "%%{userScriptPath} %%{userScriptParametres}",
    "PreparationScript": null,
    "ClusterNodeTypeId": 2,
    "IsGeneric": true,
    "IsEnabled": true
  }
],
"CommandTemplateParameters": [
  {
    "Id": 1,
    "Identifier": "userScriptPath",
    "Query": "",
    "Description": "Path of the user script, to be executed via the generic job script",
    "CommandTemplateId": 1,
    "IsVisible": true
  },
  {
    "Id": 2,
    "Identifier": "userScriptParametres",
    "Query": "",
    "Description": "Generic parameters of the generic command template.",
    "CommandTemplateId": 1,
    "IsVisible": false
  }
]
Note

This configuration defines the Generic Command Template, allowing execution of arbitrary scripts. However, users do not need to modify this configuration. Instead, they should focus on how to submit a job using the template.

To submit a job using the Generic Command Template, users must specify the required parameters in the job specification. The primary parameters are:

Absolute path to the script (userScriptPath): The script that will be executed.

Parameters defined in the script (userScriptParametres): Parameters listed using #HEAPPE_PARAM in the referenced script.
Example of a user-defined script located at ~/.key_scripts/test_generic.sh. This script must be placed on the cluster and referenced by an absolute path in the userScriptPath parameter of the Generic Command Template in the Job Specification, and the Robot Account must have execution rights to run it. This script will be executed using the Generic Command Template.
  #!/bin/bash
  #HEAPPE_PARAM iterations
  #HEAPPE_PARAM message

  echo "Input param: ${message}"
  for i in $( eval echo "{0..${iterations}}" )
  do
      echo "Iteration: ${i}"
      sleep 30s
  done
  echo "some result" >> resultFile.txt
  exit 0
When submitting a job using this script, the job specification should look like this:
Example of a job specification using the Generic Command Template.
{
  ...
  "CommandTemplateId": 2, // Identifier of the Generic Command Template
  "ClusterNodeTypeId": 2,
   "TemplateParameterValues": [
    {
      "CommandParameterIdentifier": "userScriptPath",
      "ParameterValue": "~/.key_scripts/test_generic.sh" // User defined script path with specified #HEAPPE_PARAM(s) if needed
    },
    {
      "CommandParameterIdentifier": "message",
      "ParameterValue": "Hello World"
    },
    {
      "CommandParameterIdentifier": "iterations",
      "ParameterValue": "5"
    }
 ]
 ...
}
userScriptPath: Specifies the absolute path to the script that will be executed (test_generic.sh).

The Generic Command Template executes ~/.key_scripts/generic.sh, passing userScriptPath and other Template Parameters.

~/.key_scripts/generic.sh sources ~/.key_scripts/test_generic.sh, which then runs with the provided parameters.
Example of the generic.sh script.
#!/bin/bash
USER_SCRIPT="$1"
PARAMETERS="$2"

# Expand parameter pairs
case ${PARAMETERS} in
    (*=*) eval "${PARAMETERS}";
esac

# Source the user script to execute it with expanded parameters
. "${USER_SCRIPT}"
The Generic Command Template allows dynamic execution of bash scripts.

Users need to provide: - userScriptPath (absolute path to the script). - parameters defined in the user script (#HEAPPE_PARAM)

Job submission requires setting these parameters correctly.

To retrieve the parameters defined in the user script, you can use the following API endpoint. This allows you to dynamically fetch the parameters associated with the script you plan to execute.

POST http://localhost:5000/localhpc/heappe/ClusterInformation/RequestCommandTemplateParametersName

The request body requires the following parameters:

SessionCode: A string representing the session code.

CommandTemplateId: The ID of the command template.

ProjectId: The project ID.

UserScriptPath: The path to the user script for which you want to retrieve the parameters.

You can use the following cURL command to make the request:
curl -X 'POST' \
  'http://localhost:5000/heappe/ClusterInformation/RequestCommandTemplateParametersName' \
  -H 'accept: application/json' \
  -H 'Content-Type: application/json' \
  -d '{
  "SessionCode": "string",
  "CommandTemplateId": 0,
  "ProjectId": 0,
  "UserScriptPath": "string"
}'