Configure the CloudWatch Agent

The CloudWatch Agent enables Virtana Platform to collect additional EC2 metrics, such as memory utilization, from AWS. Reports display cost vs. CPU utilization by default. You must install the agent on each AWS instance on which you want to view cost vs. memory utilization.

Tip

The CloudWatch Agent configuration below adds a single memory metric to all instances on which it is installed. This incurs an additional charge to your CloudWatch bill. See AWS CloudWatch pricing for more information.

Options for Configuring the CloudWatch Agent

Installing the CloudWatch agent can be done in a variety of ways, but each method requires the use of Virtana Platform's unique agent configuration file.

AWS offers 3 ways to install the CloudWatch Agent:

  • via the Command Line

  • via AWS Systems Manager

  • via AWS CloudFormation

Complete the following tasks to install the agent on Linux and Windows instances using the CLI method and the AWS console.

Linux CLI Installation

  1. SSH into your Linux instance.

  2. Run one of the following to download the agent, depending on your Linux distribution:

    Red Hat: wget https://s3.amazonaws.com/amazoncloudwatch-agent/amazon_linux/amd64/latest/amazon-cloudwatch-agent.rpm

    Debian: wget https://s3.amazonaws.com/amazoncloudwatch-agent/ubuntu/amd64/latest/amazon-cloudwatch-agent.deb

    The file should immediately download.

  3. Run one of the following to install the agent:

    Red Hat: sudo rpm -i amazon-cloudwatch-agent.rpm

    Debian: sudo dpkg -i -E ./amazon-cloudwatch-agent.deb

  4. Navigate to the bin directory of the agent:

    cd /opt/aws/amazon-cloudwatch-agent/bin

  5. Create a file named config.json, place the Linux Agent Config File contents in it, and save the file.

    See below for JSON config file content.

  6. Create an IAM role and attach it to the instance.

    Follow the AWS links above for instructions.

    The IAM role provides permissions for reading information from the instance and writing it to CloudWatch.

  7. Run the following command to initialize the agent configuration:

    sudo /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -a fetch-config -m ec2 -c file:config.json -s

    • If you get a parsing error, do the following as a workaround:

      • sudo mkdir /usr/share/collectd

      • cd /usr/share/collectd

      • sudo touch types.db

  8. Start the agent:

    sudo /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -m ec2 -a start

  9. Verify the agent is running:

    sudo /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -m ec2 -a status

  10. A new metric cwagent.mem_used_percent is now available in AWS CloudWatch on the respective EC2 instance and can be accessed by Optimize. The Memory Utilization metric in Optimize will display an updated value the next time the cost reports run.

Example 1. Linux Agent Config File
      {
  "agent": {
    "metrics_collection_interval": 60,
    "run_as_user": "root"
  },
  "metrics": {
    "append_dimensions": {
      "InstanceId": "${aws:InstanceId}"
    },
    "metrics_collected": {
      "collectd": {
        "metrics_aggregation_interval": 60
      },
      "mem": {
        "measurement": [
          "mem_used_percent"
        ],
        "metrics_collection_interval": 60
      },
      "statsd": {
        "metrics_aggregation_interval": 60,
        "metrics_collection_interval": 60,
        "service_address": ":8125"
      }
    }
  }
}



Windows CLI Installation

  1. Log on to your Windows instance.

  2. Download the following file:

    https://s3.amazonaws.com/amazoncloudwatch-agent/windows/amd64/latest/amazon-cloudwatch-agent.msi

    The file should immediately download.

  3. Open the command prompt, navigate to the directory containing the downloaded file, and enter the following to install the agent:

    msiexec /i amazon-cloudwatch-agent.msi

  4. Open a text editor, create a new file, and place the Windows Agent Config File contents in it.

  5. Save the file with the name amazon-cloudwatch-agent.json to the following directory:

    C:\ProgramData\Amazon\AmazonCloudWatchAgent

    See below for JSON config file content.

  6. Create an IAM role and attach it to the instance.

    Follow the AWS links above for instructions.

  7. Open Windows PowerShell and run the following command to fetch the configuration and start the agent:

    & "C:\Program Files\Amazon\AmazonCloudWatchAgent\amazon-cloudwatch-agent-ctl.ps1" -a fetch-config -m ec2 -c file:"C:\ProgramData\Amazon\AmazonCloudWatchAgent\amazon-cloudwatch-agent.json" -s

  8. Verify the agent is running:

    & $Env:ProgramFiles\Amazon\AmazonCloudWatchAgent\amazon-cloudwatch-agent-ctl.ps1 -m ec2 -a status

  9. If necessary, start the agent using this command:

    & $Env:ProgramFiles\Amazon\AmazonCloudWatchAgent\amazon-cloudwatch-agent-ctl.ps1 -m ec2 -a start

    A new metric cwagent.memory % committed bytes in use is now available in AWS CloudWatch on the respective EC2 instance and can be accessed by Optimize. The Memory Utilization metric will display an updated value the next time the Optimize reports run.

Example 2. Windows Agent Config File
      {
  "agent": {
    "metrics_collection_interval": 60,
    "logfile": "c:\\ProgramData\\Amazon\\AmazonCloudWatchAgent\\Logs\\amazon-cloudwatch-agent.log"
  },
  "metrics": {
    "metrics_collected": {
      "Memory": {
        "measurement": [
          "% Committed Bytes In Use"
        ],
        "metrics_collection_interval": 60
      }
    },
    "append_dimensions": {
      "InstanceId": "${aws:InstanceId}"
    }
  }
}



Verify the Memory Metric is Collected in AWS

You can verify that the memory metric is being collected in AWS to ensure CloudWatch is properly configured.

  1. Navigate to CloudWatch and in the left pane select Metrics > All Metrics.

  2. On the Browse tab, under Metrics, locate Custom Namespaces and click the card labeled CWAgent.

    vp-settings-cloudwatch-verify-aws.png

    New cards display with information for the CWAgent.

  3. Click the card labeled Instanceid, objectname.

    vp-settings-cloudwatch-verify-aws-2.png

    You should see a list of instances producing Memory Usage Metrics.

    Cost vs Utilization (CvU) reports are updated daily in Optimize, so the day after your AWS configuration is completed you should see Memory Utilization Metrics on the CvU report

Verify the Memory Metric is Collected in Optimize

If you receive right-sizing or cost vs utilization reports, you can verify in those reports that Memory Utilization values are displayed. The Rigjht Sizing report updates every Saturday and the Cost vs Utilization (CvU) report updates daily.

If you do not receive reports, you can verify the metric in the Optimize UI.

  1. Navigate to the Cost Saving Opportunities page in Optimize and click the Right Sizing tab under Recommendation Details.

  2. Locate a compute instance in the table for which memory utilization is being collected and expand the row.

  3. View the Current and Proposed values on the Memory Utilization card in the details.

  4. Navigate to the Cost vs Utilization page and click the EC2 Cost tab.

  5. If the memory metric is not displayed, select a Settings file and click the Edit icon.

  6. Under Options, change the Metric Statistic to Memory Utilization %.

    The Summary chart and Cost Details table now display Memory Utilization %.

  7. Either Save the new settings under a new name or Revert the existing file.