Infrastructure Observability custom scripts

The scripts authenticate to the Virtana Platform API with an OAuth client ID and client secret. To generate these credentials, do the following:

You can't retrieve the client secret again after you leave the page, so store it before you continue.

Both scripts use jq, a command-line JSON processor, to read and modify policy data. Confirm that jq is installed on the system where you run the script. If jq isn't installed, install it with your operating system's package manager before you continue.

Each script requires three input values that identify your GV host and authenticate the API requests. You pass these values to the script when you run it. The following table describes each input parameter:

Before updating any policy, each script requests an OAuth access token from the /authorization/oauth/token endpoint, then sends that token with every subsequent request. The following table describes the authentication parameters in the scripts:

Before you run a script for the first time, give it execute permission. Run the following command, replacing the filename with the script you want to run:

chmod +x io-collection-policies-update.bsh

In some IO entity collections, the inventory topology doesn't form correctly. As a result, Global View displays only the entity itself and omits related entities, such as parent, host, and connected entities.

For collections created through the IO collection policy API (/api/ipm/io_collection_policy), the collectRelationship parameter isn't enabled by default. Without relationship collection, IO can't build the inventory topology, so related entities are missing.

To resolve the issue, run the io-collection-policies-update.bsh script. The script reads every IO collection policy, sets collectRelationship to true for each entity, and saves the updated policies through the API.

#!/bin/bash

set -e

if [ $# -ne 3 ]; then
    echo "Usage: $0 <BASE_URL> <CLIENT_ID> <CLIENT_SECRET>"
    echo "Example:"
    echo "$0 https://vp-qa2.k8s.vi.local 2aee7ae4e351465e86b8d5eac566ce7a aba2058fad7d4e7fabfaf20cb727097c"
    exit 1
fi

BASE_URL="$1"
CLIENT_ID="$2"
CLIENT_SECRET="$3"

echo "Getting OAuth token..."

TOKEN_RESPONSE=$(curl --silent --insecure \
    --location "${BASE_URL}/authorization/oauth/token?override_token=true" \
    --header "Accept: application/json" \
    --header "Content-Type: application/x-www-form-urlencoded" \
    --data-urlencode "client_id=${CLIENT_ID}" \
    --data-urlencode "client_secret=${CLIENT_SECRET}" \
    --data-urlencode "grant_type=client_credentials")

TOKEN=$(echo "${TOKEN_RESPONSE}" | jq -r '.access_token')

if [ -z "${TOKEN}" ] || [ "${TOKEN}" = "null" ]; then
    echo "Failed to obtain access token"
    echo "${TOKEN_RESPONSE}"
    exit 1
fi

echo "Successfully obtained access token" 

API_URL="${BASE_URL}/api/ipm/io_collection_policy"

echo "Fetching policies from ${BASE_URL} ..."

curl --silent --insecure \
     --location "${API_URL}" \
     --header "Authorization: Bearer ${TOKEN}" \
     > policies.json

COUNT=$(jq 'length' policies.json)

echo "Found ${COUNT} policies"
for ((i=0; i<COUNT; i++))
do
    POLICY=$(jq ".[$i]" policies.json)
    POLICY_ID=$(echo "$POLICY" | jq -r '.id')
    POLICY_NAME=$(echo "$POLICY" | jq -r '.collectionPolicyName')
    echo "Processing: ${POLICY_NAME}"

    UPDATED_POLICY=$(echo "$POLICY" | jq '
        .context |= map(
            if .collectionType == "ENTITY"
            then . + {"collectRelationship": true}
            else .
            end
        )
    ')

    TMP_FILE="/tmp/${POLICY_ID}.json"
    echo "$UPDATED_POLICY" > "$TMP_FILE"
    echo "Updating policy ${POLICY_ID}"
    curl --silent --show-error --insecure \
         --request PUT \
         --location "${API_URL}/${POLICY_ID}" \
         --header "Authorization: Bearer ${TOKEN}" \
         --header "Content-Type: application/json" \
         --data @"${TMP_FILE}"
    echo "Updated ${POLICY_NAME}"
done

echo "All policies updated successfully."

The script changes the following fields in each collection policy. The table describes what each field controls:

Run the script from the directory that contains it. Provide the base URL, client ID, and client secret as arguments:

./io-collection-policies-update.bsh <BASE_URL> <CLIENT_ID> <CLIENT_SECRET>

For example:

./io-collection-policies-update.bsh https://vp-qa2.k8s.vi.local 2aee7ae4e351465e86b8d5eac566ce7a aba2058fad7d4e7fabfaf20cb727097c

The script prints the name of each policy as it updates it, and prints a confirmation message when all policies are updated.

Updated collection settings take effect during the next scheduled collection cycle for each policy. By default, collection runs every 24 hours, unless you've configured a different schedule.

To apply the updated topology without waiting for the next collection cycle, do the following:

When IO Cloud Sync starts again, it runs all collection policies and synchronizes the updated topology.

After the updated policies run, IO collects entity relationships correctly. The inventory topology in GV then displays parent, host, and connected entities as expected.

Custom properties, both property names and their values, from IO entities aren't collected and synchronized to Global View. As a result, you can't use these custom properties to filter, search, or report in Global View.

To confirm whether this issue applies to your environment, check the IO collection policy configuration. Send the following request, replacing {GV_URL} with your Global View host URL:

GET {GV_URL}/api/ipm/io_collection_policy

Review the policy for the affected entity type. Custom properties appear in the entityProperties array in the format custom.<custom_property_name>. If the entityProperties array doesn't include any custom. entries, then IO isn't collecting those custom properties, and you can't filter on their values in Global View.

To resolve the issue, run the io-colpolicy-for-customProps.bsh script. The script reads every IO collection policy and adds the custom.* wildcard to each entity's entityProperties list, which makes all available custom properties for that entity type eligible for collection.

#!/bin/bash

set -e

if [ $# -ne 3 ]; then
    echo "Usage: $0 <BASE_URL> <CLIENT_ID> <CLIENT_SECRET>"
    exit 1
fi

BASE_URL="$1"
CLIENT_ID="$2"
CLIENT_SECRET="$3"

echo "Obtaining OAuth token..."

TOKEN_RESPONSE=$(curl --silent --insecure \
    --location "${BASE_URL}/authorization/oauth/token?override_token=true" \
    --header "Accept: application/json" \
    --header "Content-Type: application/x-www-form-urlencoded" \
    --data-urlencode "client_id=${CLIENT_ID}" \
    --data-urlencode "client_secret=${CLIENT_SECRET}" \
    --data-urlencode "grant_type=client_credentials")

TOKEN=$(echo "$TOKEN_RESPONSE" | jq -r '.access_token')

if [ -z "$TOKEN" ] || [ "$TOKEN" = "null" ]; then
    echo "Failed to obtain token"
    echo "$TOKEN_RESPONSE"
    exit 1
fi

echo "Token acquired"

API_URL="${BASE_URL}/api/ipm/io_collection_policy/aa1"

echo "Fetching policies..."

curl --silent --insecure \
     --location "${API_URL}" \
     --header "Authorization: Bearer ${TOKEN}" \
     > policies.json

COUNT=$(jq 'length' policies.json)

echo "Found ${COUNT} policies"

for ((i=0; i<COUNT; i++))
do
    POLICY=$(jq ".[$i]" policies.json)

    POLICY_ID=$(echo "$POLICY" | jq -r '.id')
    POLICY_NAME=$(echo "$POLICY" | jq -r '.collectionPolicyName')

    echo ""
    echo "Processing Policy: ${POLICY_NAME}"

    UPDATED_POLICY=$(echo "$POLICY" | jq '
      .context |= map(
        if .collectionType == "ENTITY" then
           .entityProperties |= (
        if index("custom.*")
        then .
        else . + ["custom.*"]
        end
      )
    else
      .
    end
      )
    ')

    TMP_FILE="/tmp/${POLICY_ID}.json"
    echo "$UPDATED_POLICY" > "$TMP_FILE"

    curl --silent --show-error --insecure \
         --request PUT \
         --location "${API_URL}/${POLICY_ID}" \
         --header "Authorization: Bearer ${TOKEN}" \
         --header "Content-Type: application/json" \
         --data @"${TMP_FILE}" >/dev/null

    echo "Updated ${POLICY_NAME}"
done

echo ""
echo "Completed updating all policies"

The script changes the following fields in each collection policy. The table describes what each field controls:

After you complete the steps in Prerequisities, run the script from the directory that contains it. Provide the base URL, client ID, and client secret as arguments:

./io-colpolicy-for-customProps.bsh <BASE_URL> <CLIENT_ID> <CLIENT_SECRET>

The script prints the name of each policy as it processes it, and prints a confirmation message when all policies are updated.

Updated collection policies take effect during the next scheduled collection cycle. By default, collection runs every 24 hours, unless you've configured a different schedule.

To collect the custom property data without waiting for the next collection cycle, restart the IO Cloud Sync service from IO Service Management. When IO Cloud Sync starts again, it runs all collection policies and synchronizes entity data, including the newly added custom properties. Later collections continue on the configured schedule.

After the updated policies run, IO collects and synchronizes all configured custom properties from IO entities to GV. You can then use these custom properties to filter, search, and report in GV.