Skip to main content

Stackstorm with Global View

StackStorm (ST2) integrates with Virtana Global View to enable event-driven automation within the infrastructure monitoring ecosystem. When deployed alongside Global View, StackStorm functions as the automation engine, continuously monitoring for infrastructure events generated by the platform. Upon detecting these events, it triggers predefined rules, actions, and multi-step workflows, allowing automated remediation and operational responses without requiring manual intervention.

Before you begin, ensure you have:

  • Kubernetes cluster access.

  • Helm v3 is installed locally.

  • StorageClass available that supports ReadWriteMany (RWX) for PVCs.

  • Access to a Docker registry to pull images.

  • SSL certificate files or the ability to generate self-signed certificates.

Create Docker credential secret

You have to create a Kubernetes secret with Docker registry credentials so the cluster can pull images. Replace username/password with valid registry credentials and run the commands.

kubectl create namespace virtana-stackstorm

kubectl create secret docker-registry dh-reg-cred -n virtana-stackstorm \
  --docker-server=https://index.docker.io/v2/ \
  --docker-username=username \
  --docker-password=password

This secret is referenced by the Helm values file to authenticate image pulls. Keep credentials secure.

Create SSL certificate secret

StackStorm Web (st2web) requires TLS. You can use valid certificates or generate self-signed ones. If you do not have a certificate and key, generate them below; otherwise, rename your files to st2.crt and st2.key and create the K8s secret.

You can use a self-signed certificate in two cases: when you don’t have a valid PEM pair locally, or when your valid certificate exists only in AWS Certificate Manager but isn't uploaded as files for Kubernetes secrets.

Generate self-signed certificates

Use this step when you don't have a CA-signed certificate available as local files for the StackStorm Web UI, for example, in lab, development, or air-gapped environments, or when your production certificate lives only in AWS Certificate Manager and cannot be exported as PEM files for a Kubernetes secret.

export ST2_HOSTNAME="virtana-stackstorm.example.com"

cat > st2-ssl.cnf <<- EOF
[req]
default_bits  = 2048
distinguished_name = req_distinguished_name
req_extensions = req_ext
x509_extensions = v3_req
prompt = no
[req_distinguished_name]
countryName = US
stateOrProvinceName = CA
localityName = SJ
organizationName = Self-signed certificate
commonName = 127.0.0.1: Self-signed certificate
[req_ext]
subjectAltName = @alt_names
[v3_req]
subjectAltName = @alt_names
[alt_names]
DNS.1 = $ST2_HOSTNAME
EOF

openssl req -x509 -newkey rsa:2048 -sha256 -days 7300 -nodes \
    -keyout st2.key -out st2.crt \
    -config st2-ssl.cnf

Note

You can skip generating the self-signed certificates if you have valid certificate files.

Create K8s SSL secret

If you already have valid files, ensure they are named st2.crt and st2.key before running the command below.

ls -l st2.crt st2.key
-rw-rw-r-- 1 user user 3287 Nov 26 12:10 st2.crt
-rw-rw-r-- 1 user user  165 Nov 26 13:47 st2.key

kubectl create secret generic stackstorm-ssl-certs \
  --from-file=st2.crt=./st2.crt \
  --from-file=st2.key=./st2.key \
  -n stackstorm

Note the namespace for your SSL secret. If you deploy StackStorm into virtana-stackstorm, ensure you either create the secret in the same namespace or reference it accordingly in values.

Create PersistentVolumeClaims (PVCs)

You can provision persistent storage for StackStorm packs, virtualenvs, and configs. Copy the YAML, set your StorageClass, save as stackstorm-pvcs.yaml, then apply it to the namespace.

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: pvc-st2-packs
spec:
  storageClassName: <STORAGE_CLASS>
  accessModes:
    - ReadWriteMany
  resources:
    requests:
      storage: 1Gi
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: pvc-st2-virtualenvs
spec:
  storageClassName: <STORAGE_CLASS>
  accessModes:
    - ReadWriteMany
  resources:
    requests:
      storage: 1Gi
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: pvc-st2-configs
spec:
  storageClassName: <STORAGE_CLASS>
  accessModes:
    - ReadWriteMany
  resources:
    requests:
      storage: 1Gi

To create the PVCs, run the following command:

kubectl apply -f stackstorm-pvcs.yaml -n virtana-stackstorm

Choose a StorageClass that supports ReadWriteMany (RWX), such as NFS, EFS, or a CSI driver that offers RWX. RWX is important to allow multiple pods to mount these volumes.

Prepare Helm Values for StackStorm

In your preferred editor, create stackstorm-values.yaml to customize the Helm deployment. Then use it during Helm install.

image:
  pullSecret: dh-reg-cred
global:
  storageClass: "" 
st2: 
  username: st2admin 
  password: Ch@ngeMe 
  packs:
    images:
      - repository: virtana
        name: virtana-st2-pack
        tag: latest
        pullPolicy: Always
    volumes:
      enabled: true
      packs:
        persistentVolumeClaim:
          claimName: pvc-st2-packs
      virtualenvs:
        persistentVolumeClaim:
          claimName: pvc-st2-virtualenvs
      configs:
        persistentVolumeClaim:
          claimName: pvc-st2-configs
  config: |
    [api]
    allow_origin = '*'
    [database]
    connection_timeout = 5000
    [auth]
    api_url = http://virtana-stackstorm-st2api:9101/
    mode = proxy
    backend_kwargs = {"remote_token_header": "X-Auth-Token", "remote_user_header": "X-Auth-User"}
st2web:
  replicas: 1
  service:
    type: LoadBalancer
    
    annotations:
       service.beta.kubernetes.io/aws-load-balancer-backend-protocol: https
       service.beta.kubernetes.io/aws-load-balancer-ssl-cert: <CERTIFICATE_ARN>

  env:
    ST2WEB_HTTPS: 1
  extra_volumes:
  - name: stackstorm-ssl-certs
    mount:
      mountPath: /etc/ssl/st2
    volume:
      secret:
        secretName: stackstorm-ssl-certs
  resources:
    limits:
      cpu: 100m
      memory: 200Mi
    requests:
      cpu: 50m
      memory: 25Mi
mongodb:
  image:
    repository: bitnamilegacy/mongodb
  architecture: replicaset
  enabled: true
  replicaCount: 2
  resources:
    limits:
      cpu: 500m
      memory: 1Gi
    requests:
      cpu: 10m
      memory: 80Mi
rabbitmq:
  image:
    repository: bitnamilegacy/rabbitmq
    tag: 3.8.9
  replicaCount: 1
  resources:
    limits:
      cpu: 500m
      memory: 1.5Gi
    requests:
      cpu: 10m
      memory: 80Mi
redis:
  image:
    repository: bitnamilegacy/redis
    tag: 6.0.9
  cluster:
    slaveCount: 1
  replica:
    resources:
      limits:
        cpu: 500m
        memory: 1Gi
      requests:
        cpu: 10m
        memory: 80Mi
  sentinel:
    resources:
      limits:
        cpu: 500m
        memory: 1Gi
      requests:
        cpu: 10m
        memory: 80Mi
st2actionrunner:
  replicas: 3
  resources:
    limits:
      cpu: 200m
      memory: 400Mi
    requests:
      cpu: 75m
      memory: 200Mi
st2api:
  replicas: 1
  resources:
    limits:
      cpu: 100m
      memory: 300Mi
    requests:
      cpu: 25m
      memory: 150Mi
st2auth:
  replicas: 1
  resources:
    limits:
      cpu: 100m
      memory: 170Mi
    requests:
      cpu: 50m
      memory: 85Mi
st2client:
  resources:
    limits:
      cpu: 200m
      memory: 500Mi
    requests:
      cpu: 5m
      memory: 5Mi
st2garbagecollector:
  replicas: 1
  resources:
    limits:
      cpu: 50m
      memory: 160Mi
    requests:
      cpu: 10m
      memory: 80Mi
st2notifier:
  replicas: 1
  resources:
    limits:
      cpu: 100m
      memory: 150Mi
    requests:
      cpu: 50m
      memory: 75Mi
st2rulesengine:
  replicas: 1
  resources:
    limits:
      cpu: 200m
      memory: 400Mi
    requests:
      cpu: 25m
      memory: 75Mi
st2scheduler:
  replicas: 1
  resources:
    limits:
      cpu: 100m
      memory: 150Mi
    requests:
      cpu: 50m
      memory: 75Mi
st2sensorcontainer:
  deployments: 1
  resources:
    limits:
      cpu: 500m
      memory: 1Gi
    requests:
      cpu: 50m
      memory: 100Mi
st2stream:
  replicas: 1
  resources:
    limits:
      cpu: 100m
      memory: 200Mi
    requests:
      cpu: 50m
      memory: 100Mi
st2timersengine:
  resources:
    limits:
      cpu: 50m
      memory: 150Mi
    requests:
      cpu: 10m
      memory: 75Mi
st2workflowengine:
  replicas: 1
  resources:
    limits:
      cpu: 200m
      memory: 400Mi
    requests:
      cpu: 100m
      memory: 200Mi

The following table describes each field in the core StackStorm settings file.

Table 104.

Field

Description

image.pullSecret

Name of the Docker registry secret created. See Create Docker credential secret.

global.storageClass

Default storage class for all dynamically provisioned volumes.

st2.username

Admin username for the StackStorm UI/API. Change from default.

st2.password

Admin password, change from default for security.

st2.packs.images

List of custom StackStorm pack images to install at startup.

st2.packs.images[].repository

Docker registry namespace.

st2.packs.images[].name

Pack image name.

st2.packs.images[].tag

Image tag.

st2.packs.images[].pullPolicy

When Kubernetes pulls the image.

st2.packs.images[].pullSecret

Registry credential secret to pull the image.

st2.packs.volumes.enabled

Enables persistent storage for packs.

st2.packs.volumes.packs.persistentVolumeClaim.claimName

PVC for installed packs.

st2.packs.volumes.virtualenvs.persistentVolumeClaim.claimName

PVC for pack-specific Python virtual environments.

st2.packs.volumes.configs.persistentVolumeClaim.claimName

PVC for pack configuration files.

st2.config

Inline st2.conf overrides are injected into the cluster. Used here to enable CORS, set the DB connection timeout, and configure proxy-based authentication (mode = proxy) so the StackStorm API trusts X-Auth-Token / X-Auth-User headers forwarded by an upstream proxy such as Nginx in front of Global View.



The following table describes each field of the StackStorm Web UI.

Table 105.

Field

Description

st2web.service.type

Kubernetes Service type to expose the UI.

st2web.replicas

Number of UI pod replicas.

st2web.service.annotations

Optional AWS LoadBalancer annotations used when terminating TLS at an AWS ELB with a certificate stored in AWS Certificate Manager.

st2web.env.ST2WEB_HTTPS

Set to 1 to enable HTTPS inside the st2web pod so it serves traffic over TLS using the mounted certificate.

st2web.extra_volumes

Mounts the stackstorm-ssl-certs Kubernetes secret into /etc/ssl/st2 inside the st2web pod so it can read st2.crt and st2.key.

st2web.resources.limits

Maximum CPU/memory allowed per pod.

st2web.resources.requests

Minimum CPU/memory reserved per pod



MongoDB is used as StackStorm's primary datastore. Holds rules, actions, executions, triggers, and history.

Table 106.

Field

Description

mongodb.image.repository

Container image repository for MongoDB.

mongodb.architecture

Deployment topology, replicaset provides HA across multiple Mongo pods.

mongodb.enabled

Enables the bundled MongoDB deployment.

mongodb.replicaCount

Number of MongoDB replica members to run.

mongodb.resources.limits/requests

CPU and memory limits/requests per MongoDB pod.



RabbitMQ is a message broker used by StackStorm to coordinate work between microservices.

Table 107.

Field

Description

rabbitmq.image.repository

Container image repository for RabbitMQ.

rabbitmq.image.tag

RabbitMQ version to deploy (3.8.9).

rabbitmq.replicaCount

Number of RabbitMQ pod replicas.

rabbitmq.resources.limits/requests

CPU and memory limits/requests per RabbitMQ pod.



Redis is used by StackStorm primarily for distributed coordination. Sentinel provides automatic failover.

Table 108.

Field

Description

redis.image.repository

Container image repository for Redis.

redis.image.tag

Redis version to deploy (6.0.9).

redis.cluster.slaveCount

Number of Redis replicas provisioned alongside the master.

redis.replica.resources.limits/requests

CPU and memory limits/requests per Redis replica pod.

redis.sentinel.resources.limits/requests

CPU and memory limits/requests per Redis Sentinel pod.



Each component below is a microservice that performs a specific role. They all support replicas and resources (limits/requests) configuration.

Table 109.

Component

Description

st2actionrunner

Executes actions triggered by workflows or rules.

st2api

Public REST API for the platform.

st2auth

Handles authentication and token issuance.

st2client

CLI client utilities.

st2garbagecollector

Cleans up old execution records to manage DB size.

st2notifier

Sends out notifications when actions complete.

st2rulesengine

Evaluates rules against incoming triggers.

st2scheduler

Schedules actions for execution.

st2sensorcontainer

Hosts sensors that watch for external events.

st2stream

Streams real-time events to clients via HTTP/SSE.

st2timersengine

Handles scheduled/timed triggers

st2workflowengine

Orchestrates multi-step workflows



For every microservice above:

Table 110.

Field

Description

<component>.replicas

Number of pod replicas to run for that microservice.

<component>.resources.limits.cpu

The maximum CPU the pod is allowed to consume.

<component>.resources.limits.memory

The maximum memory the pod is allowed to consume.

<component>.resources.requests.cpu

Minimum CPU reserved for the pod at scheduling time.

<component>.resources.requests.memory

Minimum memory reserved for the pod at scheduling time.



Deploy StackStorm with Helm

After finalizing stackstorm-values.yaml, add the Helm repo and install/upgrade StackStorm with your values file.

helm repo add stackstorm https://helm.stackstorm.com/

helm upgrade --install --namespace virtana-stackstorm --create-namespace \
  virtana-stackstorm stackstorm/stackstorm-ha -f stackstorm-values.yaml \
  --version "1.1.0"

On success, all StackStorm components will begin running. Use kubectl get pods -n virtana-stackstorm to watch readiness.

Configure DNS for StackStorm Web

Retrieve the external LoadBalancer address to create a DNS record. Run the following kubectl command, then add a DNS A/AAAA/CNAME record accordingly.

kubectl -n virtana-stackstorm get svc virtana-stackstorm-st2web \
  -o=jsonpath='{.status.loadBalancer.ingress[0].ip}'

#OR

kubectl -n virtana-stackstorm get svc virtana-stackstorm-st2web \
  -o=jsonpath='{.status.loadBalancer.ingress[0].hostname}'

Run the following command to create a DNS record:

Source: virtana-stackstorm.example.com
Target: <<LB Endpoint>>

Deploy Nginx proxy in the StackStorm namespace

Global View integrates with StackStorm through an Nginx proxy. You create a minimal values file, then install the virtana-co-controller chart with the required tag.

Create a new file stackstorm-nginx-values.yaml with the following content, change the values as per the comments.

stackstorm_nginx:
  cp_hostname: <<global-view-hostname.example.com>>
  st2_hostname: <<virtana-stackstorm.example.com>>
  st2_release_name: virtana-stackstorm
Table 111.

Field

Description

stackstorm_nginx.cp_hostname

Hostname of your Global View/Control Plane.

stackstorm_nginx.st2_hostname

Public hostname of the StackStorm Web endpoint you created in DNS.

stackstorm_nginx.st2_release_name

Name of the StackStorm Helm release.



helm repo add virtana-repo https://virtana.gitlab.io/helm-charts

helm upgrade --install virtana-stackstorm-nginx virtana-repo/virtana-co-controller \
  --namespace virtana-stackstorm --create-namespace \
  --set tags.stackstorm_nginx=true -f stackstorm-nginx-values.yaml \
  --version 2024.12.0

Caution

Ensure the Nginx proxy and StackStorm Web are in the same namespace or that cross-namespace references are properly configured.

Open StackStorm

Confirm the deployment is functional by logging in to the StackStorm web interface. Navigate to your configured FQDN and use the credentials you set.

  1. To open StackStorm, enter the URL https://virtana-stackstorm.example.com/ in your default browser.

  2. Login using st2.username and st2.password defined in stackstorm-values.yaml.

Use the Global View entrypoint to reach StackStorm through the Nginx proxy. Browse to the Global View URL and follow the link flow to authenticate.

  1. Enter the URL https://virtana-stackstorm.example.com/ui in your browser to open the Global View.

  2. Login with Control Plane credentials or SSO.

  3. Access StackStorm deep-link, for example: https://virtana-stackstorm.example.com/stackstorm/auth?username=john.wick@example.com.