Topics covered in this article: 

• Overview
• How To Create a Connection
• How to Configure SFTP Connection
• How to Configure SFTP (Key File) Connection
  
• How to Configure Amazon S3 Connection
• How to Configure Amazon S3 Connection (Role based authentication)
• How to Configure Google Cloud Storage (S3 Compatible) Connection
• How to Configure Minio Connection
• How to Configure Dropbox Connection
• How To Configure Google Drive Connector
• Customer Data Hub SFTP Whitelisting

Overview

 In modern technology, there is a clear need to reuse objects and let the user configure each object once and repeatedly use it over and over again.
This is the basic assumption of the “Connection” configuration in Totango Customer Data Hub.
Connection to a data source is usually done by Ops or Data teams, and the main idea is that it will be configured once and you can reuse the same connection for several file integrations.

Important note, the data file integration family is a closed system, therefore, the data source connectors which are created can be re-used for only file-based integrations and not for other integration families. 

How To Create a Connection

1. Go to your Totango instance and click on Global Settings > Data Management > Customer Data Hub > find the data file source you wish to upload data from by searching or filtering connectors.
   
   (note: you will need Totango admin privileges). 
   
   
   
2. Create a new connection (or review your existing connections)
   

How to Configure SFTP Connection

Insert the mandatory information for connecting to SFTP

• Username - SFTP login username
• Password -  SFTP login password 
• Host Name - SFTP host domain e.g. www.totango.com
• Folder path - the path to the file in one of the following forms
• Relative path - a relative path from your user home folder to the CSV file. for e.g "documents\"
• Full path - the full path to the CSV file. for e.g "\usr\joe\documents\"
• To indicate a “root” folder, just use a slash sign (“/”).

How to Configure SFTP (Key File) Connection

Insert the mandatory information for connecting to SFTP (Key File)

• Name: a meaningful connection name, For example, “Company Customer Data Files”
• Description: a meaningful description to help you and others understand the essence of this connection.
• Username - SFTP login username
• SSH Private Key -  SSH Private key file (for example *.pkk files). 
• Host Name - SFTP host domain e.g. www.totango.com
• Folder path - the path to the file in one of the following forms
• Relative path - a relative path from your user home folder to the CSV file. for e.g "documents\"
• Full path - the full path to the CSV file. for e.g "\usr\joe\documents\"
• To indicate a “root” folder, just use a slash sign (“/”).

How to Configure Amazon S3 Connection

Insert the required information for connecting to AWS S3

• Access Key ID - AWS API key ID
• Secret Key -  AWS API key secret
• Bucket Name -  AWS bucket name
• Folder path - The folder path is optional and should include the path of the file folder in the bucket without the "/" characters at the beginning or the end

See this article to understand how to view, modify, and create your AWS Access Key and Secret Access Key.

Access and Permissions

Setting up permissions

In order to access S3 buckets, you must provide roles to the service account that allow access to specific buckets.

The Customer Data Hub needs the below permissions in order to access S3 buckets, you must provide these permissions to the user that is used by Customer Data Hub:

• S3:GetObject - Required to retrieve an object from Amazon S3.
• S3:ListBucket - Required to list objects within a given bucket in Amazon S3.

How to Configure Amazon S3 Connection (Role based authentication)

For a more secure way to connect AWS S3, we also support role based authentication to AWS S3.

See this article to learn how to use external ID when granting access to your AWS resources to a third party.

See this article on how to find the ARN of an S3 bucket.

Insert the required information to connect to AWS S3:

In your AWS S3 instance, you need to define Totango ARN:

see also here

Totango ARN is: “arn:aws:iam::671587110562:root”

The external id and the Totango ARN should go to the Trust relationship:

Set the permissions configuration (example : “Action”: “s3:GetObject” and “Action”: “s3:List*”) in the permissions area:

How to Configure Google Cloud Storage (S3 Compatible) Connection

What is Google Cloud Storage (S3 Compatible)?

Cloud storage services are a clear choice for ease of management and to incrementally scale costs with growth. Google Cloud Storage offers a reasonably priced service, with clear reliability guarantees and uniquely cheap and fast lowest-tier storage. Amazon S3 was the first wildly popular cloud storage service, its API is now the de-facto standard for object storage.

Totango Customer Data Hub is enabling access to Google Cloud Storage via an S3-compatible API. 

How to pull data from Google Cloud Storage using S3 Compatible APIs?

Insert the mandatory information for connecting to GCS bucket

• Application Key -GCS application key.
  Follow the steps in this article to create or get your application and secret keys.
• Secret Key - GCS key secret.
  
• Follow the steps in this article to create or get your application and secret keys.
  
• Bucket Name - GCS bucket name.
• Folder path - folder path is optional and should include the path of the file folder in the bucket without the "/" characters at the beginning or the end.
  In case the files you want to upload are at the folder root, keep the folder path empty.

Access and Permissions

GCP Service Account

You can create a new service account to provide Toatngo Customer Data Hub with access to your GCP resources. The instructions on creating a new service account are covered in detail in GCP documentation.

Once you create the service account, create a new key, and ensure sure that you download the JSON associated with that key. This key is required in order to create the GCS integration to Toatngo Customer Data Hub

Setting up permissions

In order to access Google Cloud Storage buckets, you must provide roles to the service account that allow access to specific buckets. 

The Customer Data Hub needs the below permissions in order to access Google Cloud Storage buckets, you must provide these roles to the service account that is used by Customer Data Hub:

• storage.objects.get - Required to retrieve an object from Google Cloud Storage.
• storage.objects.list - Required to list objects within a given bucket in Google Cloud Storage.

Read more about GCS permissions.

How to Configure Minio Connection

What is Minio?

Minio is a self-hosted object storage server fully compatible with Amazon S3.

It's an S3-compatible storage server that you can run on your own on-premise servers or even local computer. Read more.

How to pull data from Minio?

Insert the mandatory information for connecting to Minio

• Endpoint URL - your Minio URL address 
• Access Key ID - Minio key ID
• Secret Key -  Minio key secret
• Bucket Name -  Minio bucket name
• Folder path - folder path is optional and should include the path of the file folder in the bucket without the "/" characters at the beginning or the end.
  In case the files you want to upload are at the folder root, keep the folder path empty.

How to Configure Dropbox Connection

Dropbox configuration is based on OAuth credentials. Enter the connection name and description, and click the “Test and Save connection” to connect using your OAuth credentials.
You will be asked to enter your Dropbox credentials unless you are already connected in a different tab of the browser. In the later, the oAuth configuration will take the credentials stored in your active session.

How To Configure Google Drive Connector

Google Drive configuration is based on OAuth credentials. Enter the connection name and description, and click the “Test and Save connection” to connect using your OAuth credentials.

You will be asked to enter your Google Drive credentials. 

For security reasons, only the user that created the Google Drive connection can create, edit or duplicate integration jobs. Other users will need to create their own connection with their own Google drive authentication in order to create integration jobs.

Once connected you can sync to Totango these file types:

CSV files with .csv extension

CSV files with .txt extension

CSV files with no extension

Google spreadsheet files: with .gheet extension

Note, you can use encrypted or unencrypted CSV files in the Google Drive connector

Google Spreadsheets Files

You can sync to Totango Google spreadsheets files (with .gsheet extension) within the Google Drive connector. 

Please note, Spreadsheet file will be downloaded as a regular CSV file, the Google connector will trim empty lines and empty columns to avoid empty results.

Known Limitations

• Currently, there is no option to work with multi-tabs spreadsheet (a spreadsheet with more than several tabs). In these cases, Totango uses automatically the first tab.
• Google spreadsheet cannot serve as an encrypted file
  

Customer Data Hub SFTP Whitelisting

In the event that your network is behind a firewall, you will need to whitelist our servers so that we may retrieve files from your SFTP server.

Please follow the instructions in this Customer Data Hub IP Whitelisting article to configure your IP whitelist.