Skip to main content

GCP Configuration Prerequisites

Note

Support for GCP is currently an Early Access feature. It is available to all Virtana Cloud Cost Management customers. Contact your Account Representative or cloudcostsavings@virtana.com to enable GCP for your account.

There are several configuration tasks that you must have completed in GCP prior to integrating with Virtana Platform. The process outlined below is just an overview of what needs to be done, with some specific configuration selections required by Virtana. If you need further details about how to perform the required tasks, see the GCP website:

You must enable BigQuery export of your Cloud Billing data and create a Virtana-specific role before you integrate with Virtana Platform. Following is a list of the configuration actions that must be completed in GCP to enable data export. The time required to complete these tasks depends on how familiar you are with the GCP console, but generally takes less than an hour.

Tip

You can use an existing project, dataset, and service account if they meet the configuration requirements. However, Virtana recommends creating these resources new, to ensure they are configured as needed.

Important

To integrate with Virtana Platform, you must be managing your Google Cloud resources using projects.

Required Permissions

To enable and configure the export of Cloud Billing data to a BigQuery dataset, you need the following permissions:

  • Billing Account Administrator role for the target Cloud Billing account.

  • BigQuery User role for the Cloud project that contains the BigQuery dataset that will be used to store the Cloud Billing data.

  • You might also need project permissions.

    If you are a member of a Google Cloud Organization, you need the Project Creator role on the Organization or Folder, to create a new project.

For more information about Google Cloud permissions, see Google Cloud documentation.

Navigating the Google Cloud Console

The following tasks describe navigation from the Google Cloud Console navigation (hamburger) menu in the upper left of the UI. You can also navigate to console pages by using the search tool.

Tip

If a sub-navigation panel doesn't display the UI as it should, you can bring it back by logging out of GCP and logging back in.

Create a Cloud Billing Account

If you already have a billing account, skip this task.

Google Cloud recommends that you create a single central cloud billing account, rather than multiple accounts. Multiple accounts are supported, but can cause issues. See the GCP Cloud Billing documentation for details.

  1. Navigate to Billing > Billing Management > Account management.

    Navigate using the left navigation pane or by using Search.

  2. Click Create Account and complete the form.

  3. Save a copy of the Billing Account ID.

    You will need to enter this ID into the Integration Setup form in Virtana Platform.

Create a Project as Billing Account

This project will contain the BigQuery dataset that will be used to store the Cloud Billing data.

Important: The project you select or create must be linked to the same cloud billing account as the GCP projects that generate the cost and pricing data that you want to export.

  1. Navigate to IAM & Admin > Create a Project.

    Navigate using the left navigation pane or by using Search.

  2. Complete the form to create the project.

    Give it a name that is easily identifiable as a project for Virtana integration.

    The Project ID is auto-generated, but can be modified at this point. It cannot be changed later.

Add a Dataset to Enable the Export of Billing Data

This dataset will be used to store the Cloud Billing data.

  1. Navigate to Billing > Billing Export.

  2. Under Detailed Usage Cost click Edit Settings.

    Be sure to edit Detailed usage cost settings so that all required data is exported.

    vp-integ-gcp-prereqs-billing-export-detailed-usage.png

  3. Select the project you just created and in the Dataset field click Create New Dataset, complete the form, and save.

    Recommendation: Name the dataset so it's clearly identifiable as being the detailed usage dataset to be used for the integration with Virtana. If you enter the wrong name in Virtana Platform, the integration will fail.

    Important:

    • Ensure Enable Table Expiration is NOT enabled.

    • Under Advanced Options, ensure "Google-managed encryption key" is selected. Do not use the "Customer-managed key" option.

Create a Role

You must create a role that allows Virtana Platform to access BigQuery data.

  1. Navigate to IAM & Admin > Roles and ensure the project you just created is still selected in the project selector.

  2. Filter the table for "bigquery" and select the required roles.

    Tip

    As you are selecting roles, be careful to click only on the checkboxes. If you click elsewhere in the table it will deselect any items you already selected.

    vp-integ-gcp-prereqs-role-create.png

  3. Click Create Role From Selection and complete the required fields.

    Give the role an easily identifiable title, such as Virtana Integration.

  4. Click Add Permissions and add any remaining permissions that are needed.

    vp-integ-gcp-prereqs-role-create-form.png

  5. In the table of assigned permissions, deselect any permissions you don't want included.

    Be careful to click only on the checkboxes.

  6. Click Create.

    The new role displays in the Roles table.

Create a Service Account

You must create a service account, assign the role you created, and give the Virtana Platform GCP service account access to your service account.

  1. Navigate to IAM & Admin > Service Accounts.

    Ensure the project you just created is still selected in the project selector.

  2. Click Create a Service Account and enter the service account details.

    Recommendation: Name the account so it's clearly identifiable as the account for the Virtana Platform integration. If you enter the wrong name in Virtana Platform, the integration will fail.

  3. Click Create and Continue.

    A section displays for granting access.

  4. Click Select a role, and select the role you previously created.

    Example: Virtana Integration

  5. Click Continue and Done.

    You don't need to grant access to users.

    The new account displays in the service accounts list.

  6. Click the name of the new service account, click the Permissions tab, and click Grant Access.

    vp-integ-gcp-prereqs-servacct-roles.png

  7. Enter the principal name.

    Principal name: customer-access@app-customer-access.iam.gserviceaccount.com

    You must enter this exact name. This is the Virtana service account that will access the GCP service account you created.

    The name is not case-sensitive, so capital and lowercase letters are read the same way.

  8. Assign two roles, replacing Service Account Admin role:

    Role 1: Service Account Token Creator

    Role 2: Service Account User

    screenshot of the principal and role fields filled in

  9. Click Save and verify that the new principal displays in the list of principals.

    vp-integ-gcp-prereqs-servacct-principal.png

Query your data in BigQuery to ensure everything is working

Your data won't be accessible for anywhere from a few hours to 48 hours. Google Cloud provides sample data you can use if you want to test the connection immediately. Information about the sample data is available at the end of the GCP tutorial entitled, Analyze your Cloud Billing data with BigQuery.

  1. Navigate to BigQuery > SQL Workspace.

  2. Expand the billing account project and click on the associated dataset.

    You might see a resource named "gcp_billing_export_resource..." This is a service account that is owned and managed by GCP. Do not delete it.

  3. Open the Editor, input your query, and run it.

    See the GCP BigQuery documentation or the blog BigQuery explained: How to query your data for details.

In this section: