Create a BigQuery metadata collector

Create a collector to import metadata from BigQuery.

Before you begin

Before you begin, verify the following:

• A MID Server is setup for the collectors. For more information, see MID Server for metadata collectors.
• All per-requisite tasks are completed. For more information, see Prepare to run the BigQuery collector.
• Role required: connection-admin

Procedure

1. Navigate to All > Workflow Data Fabric > Workflow Data Fabric Home.

2. Select the Connect Hub

1. Select Create > Metadata collector.

2. From the System list, select BigQuery.

3. From the Connection type list, select one of the following:

   1. Select New connection to configure a new connection.

   2. Select Existing connection to reuse an existing connection and select an existing connection from the Connections list.

      The configuration form is filled with details from the existing connection. The name is appended with the word Copy and sensitive details like password aren't copied.

4. Complete the form.

   Field	Description
   Connection name	Unique identifier for the connection. This field can't be modified once the connection is established.
   Short description	Purpose and details of the connection.

5. Enter the BigQuery authentication details.

   Field	Description
   BigQuery Credential File Path	Upload the credentials file you generated for authentication.

6. Enter the BigQuery configuration details.

   Field	Description
   BigQuery Project	BigQuery project that contains the datasets to catalog.
   BigQuery Datasets to Catalog	BigQuery datasets to catalog in the project. By default, all datasets in a project are cataloged. When multiple projects are specified, the dataset filter applies to all projects.

7. Select Save.

Result

The metadata collector is created and appears on the Connectors page with a Configured status. It is now ready to connect to the source system and harvest metadata.

What to do next

After creating the collector, you can perform any of the following tasks:

• Run the collector manually to harvest metadata immediately. See Run metadata collectors manually.
• Automate metadata collection by scheduling regular collector runs. See Schedule metadata collector runs.
• Monitor execution status and troubleshoot issues by viewing the runtime logs. See View runtime logs for collector runs.
• Discover and evaluate the harvested data assets in the Data Catalog. See Governing the Data Catalog.

Parent Topic:BigQuery metadata collector