In modern technology, there is a clear need to reuse objects and let the user configure each object once and repeatedly use it over and over again.
This is the basic assumption of the “Connection” configuration in Totango Customer Data Hub.
Connection to a data source is usually done by Ops or Data teams, and the main idea is that it will be configured once and you can reuse the same connection for several integrations.
Important note, the Data Warehouse integration family is a closed system, therefore, the data source connectors which are created can be re-used for only Data Warehouse integrations and not for other integration families.
Topics covered in this article:
- Creating Google BigQuery Connection
- Configuring Google BigQuery Connection
- Customer Data Hub Whitelisting
- SSL Encryption
Creating Google BigQuery Connection
Go to your Totango instance and click on Global Settings > Data Management > Customer Data Hub > Click 'Data Warehouse' and select Google BigQuery
(note: you will need Totango admin privileges).
Create a new connection (or review your existing connections)
Configure Google BigQuery Connection
The Google BigQuery connector is a pure DB connection. It means that it is built as a connector and an SQL query which powers this connection.
Common information regarding Google BigQuery can be found here.
Creating a Google BigQuery connector includes 2 steps:
- Create a New GCP service account and download a Google BigQuery key file
- Create connector in Totango
Create a New GCP service account and download a Google BigQuery key file
In the Cloud Console, go to the Create service account key page.
Go to the Create Service Account Key page
- Select a project
- From the Service account list, select Create service account.
Service Account is a user with only access to applications and APIs. Read more.
- In the Service account name field, enter a name.
- From the Role list, select Project > Viewer (Read access to resources which enable Totango connector to test the connection).
Select additional Role, select BigQuery > BigQuery Data Viewer (Access to only view datasets).
- Click Done.
To create a Google Service Account JSON Key File file that contains your key, select the 'Manage Keys' option.
Then, click on 'Add Key' and choose 'Json'. Click on 'Create' and a file with the private key will be downloaded to your computer.
- To create a Google Service Account JSON Key File file that contains your key, select the 'Manage Keys' option.
Create Connector in Totango
- Name: a meaningful connection name to the connection, For example, “Company DWH - Customer and Contacts Data”
- Description: a meaningful description to help you and others understand the essence of this connection.
- Google Service Account Key File: upload the key file downloaded.
Google BigQuery Connection Q & A
Q: How does Totango connect to the Google BigQuery database?
A: Totango connector is using Google BigQuery API.
- Q: Does the Google BigQuery connector supports a read-only connection mode?
A: Yes. Totango Customer Data Hub is using a read-only mode on the connector level (we do not allow any queries with delete, insert, update, ...).
This is part of the service account user roles described above.
Q: Can I change the Google BigQuery database port?
A: No, there is no database port.
Q: Does Totango's Google BigQuery connector supports shared tables?
A: Yes. you have access according to configured permissions.
Q: Does GoogleBigQuery connector supports Cloud SQL federated tables or logical views?
A: Yes, it supports federated queries. Read more here.
Q: Does GoogleBigQuery connector supports querying Nested Objects or Array Objects?
A: Yes, by using SQL JOIN as part of the query.
Q: Does GoogleBigQuery connector supports querying partitioned tables?
A: Yes, it supports partitioned tables. Read more here.
Customer Data Hub Whitelisting
In the event that your network is behind a firewall, you will need to whitelist our servers so that we may retrieve information from your data warehouse.
Please follow the instructions in this Customer Data Hub IP Whitelisting article to configure your IP whitelist.
All data warehouse connectors support SSL encryption out-of-the-box (no need to configure anything to enable SSL encryption).
Totango connectors will use an SSL encrypted connection in case it is activated and use a non-SSL encrypted connection otherwise.
The implementation logic is as follows:
by default, SSL encryption is used during connecting, if the data warehouse server does not support SSL encryption, then the connection will using an unencrypted connection to stream data from the data warehouse.