Google Cloud Data Source
This data source establishes a connection to Google Cloud for extracting assets and properties.
Connection configuration
The configuration of this data source requires you to set up a service account with the appropriate permissions for the assets you want to import (e.g. Compute Viewer in order to import compute engine instances).
Export the service account key in JSON format and paste the content in the access data field of the data source configuration. Additionally, you need to provide the project ID to import assets from.
Service account and permission configuration
- Create a service account for the desired project
- Create a key in JSON format for this service account
- Create a role for this service account
- Link the role to the service account
- Add the desired permissions
It is recommended to reduce the permissions of the service account to a bare minimum - preferably read-only exclusively for the assets to be imported.
If no project is specified in the data source, all projects will be queried using the Resource Manager API, which requires the following additional permissions:
resourcemanager.projects.getresourcemanager.projects.list
Importable types and permissions
The Google Cloud importer has a fixed schema with the following asset types and required permissions:
| Asset-Type | Required permissions |
|---|---|
| GCE Instance | compute.instances.list, osconfig.inventories.get, compute.instanceGroups.list |
| GCE Image | compute.images.list |
| GCE Disk | compute.disks.list, compute.instances.list |
| Scalable Compute Group | compute.instanceGroups.list, compute.instanceGroupManagers.list, compute.autoscalers.list, compute.instanceTemplates.list |
| VPC | compute.networks.list |
| IP Address | compute.addresses.list |
| Load Balancer | compute.forwardingRules.list |
| Cloud DNS Records | dns.managedZones.list, dns.resourceRecordSets.list |
| Cloud SQL Instance | cloudsql.instances.list |
| Spanner Instance | spanner.instances.list |
| Cloud Storage Bucket | storage.buckets.list |
| Cloud Run Function | cloudfunctions.functions.list |
| PubSub Topic | pubsub.topics.list |
| Bigtable Instance | bigtable.instances.list |
| Project Costs | bigquery.jobs.create, bigquery.tables.getData |
| Project | resourcemanager.projects.get, resourcemanager.projects.list, resourcemanager.projects.getIamPolicy |
| Folder | resourcemanager.folders.get, resourcemanager.folders.list, resourcemanager.folders.getIamPolicy |
| Billing Project | billing.accounts.get, billing.accounts.list, billing.resourceAssociations.list |
| Network Traffic | monitoring.timeSeries.list |
Note: the link importer requires the permissions for both types that will be linked.
Fetching usage data
Fetching usage data with Cloud Monitoring can cause additional costs.
If you want to fetch usage data like uptime of a GCE Instance or average size of a Cloud Storage Bucket you can enable that functionality in the importer (default disabled).
The Cloud Monitoring API is required by asset-type Network Traffic and it is optional for GCE Disk, Cloud SQL Instance, Cloud Storage Bucket.
Be aware of additional costs too.
Required permission to fetch usage data: monitoring.timeSeries.list
Combined permissions
- billing.accounts.get
- billing.accounts.list
- billing.resourceAssociations.list
- bigquery.jobs.create
- bigquery.tables.getData
- bigtable.instances.list
- cloudfunctions.functions.list
- cloudsql.instances.list
- compute.addresses.list
- compute.autoscalers.list
- compute.disks.list
- compute.forwardingRules.list
- compute.images.list
- compute.instances.list
- compute.instanceGroups.list
- compute.instanceGroupManagers.list
- compute.instanceTemplates.list
- compute.machineTypes.list
- compute.networks.list
- dns.managedZones.list
- dns.resourceRecordSets.list
- monitoring.timeSeries.list
- osconfig.inventories.get
- pubsub.topics.list
- resourcemanager.folders.get
- resourcemanager.folders.getIamPolicy
- resourcemanager.folders.list
- resourcemanager.projects.get
- resourcemanager.projects.getIamPolicy
- resourcemanager.projects.list
- spanner.instances.list
- storage.buckets.list
Creating importers
After configuring the data source, you can create auto-configured importers to bring data from Google Cloud into Txture.