Redshift Data Source API Reference Guide
The redshift endpoint allows you to connect and manage Redshift data sources in Immuta.
Note
Additional fields may be included in some responses you receive; however, these attributes are for internal purposes and are therefore undocumented.
Redshift workflow
Create a data source
Data source duplicates
In order to avoid two data sources referencing the same table, users can not create duplicate data sources. If you attempt to create a duplicate data source using the API, you will encounter a warning stating "duplicate tables are specified in the payload."
Endpoint
| Method | Path | Purpose |
|---|---|---|
| POST | /redshift/handler |
Save the provided connection information as a data source. |
Query Parameters
None.
Payload Parameters
| Attribute | Description | Required |
|---|---|---|
| private | boolean When false, the data source will be publicly available in the Immuta UI. |
Yes |
| blobHandler | array[object] A list of full URLs providing the locations of all blob store handlers to use with this data source. |
Yes |
| blobHandlerType | string Describes the type of underlying blob handler that will be used with this data source (e.g., MS SQL). |
Yes |
| recordFormat | string The data format of blobs in the data source, such as json, xml, html, or jpeg. |
Yes |
| type | string The type of data source: ingested (metadata will exist in Immuta) or queryable (metadata is dynamically queried). |
Yes |
| name | string The name of the data source. It must be unique within the Immuta instance. |
Yes |
| sqlTableName | string A string that represents this data source's table in the Query Engine. |
Yes |
| organization | string The organization that owns the data source. |
Yes |
| category | string The category of the data source. |
No |
| description | string The description of the data source. |
No |
| hasExamples | boolean When true, the data source contains examples. |
No |
Response Parameters
| Attribute | Description |
|---|---|
| id | integer The handler ID. |
| dataSourceId | integer The ID of the data source. |
| warnings | string This message describes issues with the created data source, such as the data source being unhealthy. |
| connectionString | string The connection string used to connect the data source to Immuta. |
Request example
This request creates two Redshift data sources, which are specified in example-payload.json.
curl \
--request POST \
--header "Content-Type: application/json" \
--header "Authorization: Bearer dea464c07bd07300095caa8" \
--data @example-payload.json \
https://demo.immuta.com/redshift/handler
Payload example
{
"handler": [{
"metadata": {
"ssl": true,
"userFiles": [],
"port": 5439,
"hostname": "redshift.database.example.io",
"database": "test",
"username": "user",
"password": "yourpassword",
"schemaProjectName": "Army",
"staleDataTolerance": 86400,
"bodataSchemaName": "army",
"bodataTableName": "army_records",
"dataSourceName": "Army Army Records",
"table": "army_records",
"schema": "army",
"nativeViewName": "army_records_immuta",
"nativeSchemaName": "army_immuta",
"nativeWorkspaceName": "immuta_test"
}
}, {
"metadata": {
"ssl": true,
"userFiles": [],
"port": 5439,
"hostname": "redshift.database.example.io",
"database": "test",
"username": "user",
"password": "your-password",
"schemaProjectName": "Army",
"staleDataTolerance": 86400,
"bodataSchemaName": "army",
"bodataTableName": "data",
"dataSourceName": "Army Data",
"table": "data",
"schema": "army",
"nativeViewName": "data_immuta",
"nativeSchemaName": "army_immuta",
"nativeWorkspaceName": "immuta_test"
}
}],
"dataSource": {
"blobHandler": {
"scheme": "https",
"url": ""
},
"blobHandlerType": "Redshift",
"recordFormat": "",
"type": "queryable",
"schemaEvolutionId": null,
"columnEvolutionEnabled": true
},
"schemaEvolution": {
"ownerProfileId": 2,
"config": {
"nameTemplate": {
"nameFormat": "<Schema> <Tablename>",
"tableFormat": "<tablename>",
"sqlSchemaNameFormat": "<schema>",
"schemaProjectNameFormat": "<Schema>"
}
},
"schemas": []
}
}
Response example
{
"connectionString":"user@redshift.database.example.io:5439/test"
}
Get information about a data source
Endpoint
| Method | Path | Purpose |
|---|---|---|
| GET | /redshift/handler/{handlerId} |
Get the handler metadata associated with the provided handler ID. |
Query Parameters
| Attribute | Description | Required |
|---|---|---|
| handlerId | integer The ID of the handler. |
Yes |
| skipCache | boolean When true, will skip the handler cache when retrieving metadata. |
No |
Response Parameters
| Attribute | Description |
|---|---|
| body | array[object] Metadata about the data source, including the data source ID, schema, database, and connection string. |
Request example
This request returns metadata for the handler with the ID 41.
curl \
--request GET \
--header "Content-Type: application/json" \
--header "Authorization: Bearer dea464c07bd07300095caa8" \
https://demo.immuta.com/redshift/handler/41
Response Example
{
"dataSourceId": 41,
"metadata": {
"ssl": true,
"port": 5439,
"query": null,
"table": "data",
"schema": "army",
"database": "test",
"hostname": "redshift.database.example.io",
"username": "user",
"eventTime": "fileinfo.creationtime",
"userFiles": [],
"dataSourceName": "Army Data",
"bodataTableName": "data",
"bodataSchemaName": "army",
"columnsNormalized": false,
"schemaProjectName": "Army",
"staleDataTolerance": 86400
},
"type": "odbcHandler",
"connectionString": "user@redshift.database.example.io:5439/test",
"remoteTableDescription": "[{\"name\":\"text\",\"datatype\":\"string\",\"nullable\":null,\"isprimarykey\":null},{\"name\":\"date\",\"datatype\":\"string\",\"nullable\":null,\"isprimarykey\":null},{\"name\":\"hashed_id\",\"datatype\":\"string\",\"nullable\":null,\"isprimarykey\":null},{\"name\":\"filename\",\"datatype\":\"string\",\"nullable\":null,\"isprimarykey\":null},{\"name\":\"storage_location\",\"datatype\":\"string\",\"nullable\":null,\"isprimarykey\":null},{\"name\":\"size\",\"datatype\":\"integer\",\"nullable\":null,\"isprimarykey\":null},{\"name\":\"owner\",\"datatype\":\"string\",\"nullable\":null,\"isprimarykey\":null},{\"name\":\"dc:publisher\",\"datatype\":\"string\",\"nullable\":null,\"isprimarykey\":null},{\"name\":\"meta:page-count\",\"datatype\":\"string\",\"nullable\":null,\"isprimarykey\":null},{\"name\":\"publisher\",\"datatype\":\"string\",\"nullable\":null,\"isprimarykey\":null},{\"name\":\"author\",\"datatype\":\"string\",\"nullable\":null,\"isprimarykey\":null},{\"name\":\"extended-properties:template\",\"datatype\":\"string\",\"nullable\":null,\"isprimarykey\":null},{\"name\":\"application-version\",\"datatype\":\"string\",\"nullable\":null,\"isprimarykey\":null},{\"name\":\"revision-number\",\"datatype\":\"string\",\"nullable\":null,\"isprimarykey\":null},{\"name\":\"page-count\",\"datatype\":\"string\",\"nullable\":null,\"isprimarykey\":null},{\"name\":\"last-author\",\"datatype\":\"string\",\"nullable\":null,\"isprimarykey\":null},{\"name\":\"character-count-with-spaces\",\"datatype\":\"string\",\"nullable\":null,\"isprimarykey\":null},{\"name\":\"xmptpg:npages\",\"datatype\":\"string\",\"nullable\":null,\"isprimarykey\":null},{\"name\":\"creation-date\",\"datatype\":\"string\",\"nullable\":null,\"isprimarykey\":null},{\"name\":\"meta:last-author\",\"datatype\":\"string\",\"nullable\":null,\"isprimarykey\":null},{\"name\":\"extended-properties:application\",\"datatype\":\"string\",\"nullable\":null,\"isprimarykey\":null},{\"name\":\"meta:creation-date\",\"datatype\":\"string\",\"nullable\":null,\"isprimarykey\":null},{\"name\":\"meta:author\",\"datatype\":\"string\",\"nullable\":null,\"isprimarykey\":null},{\"name\":\"custom:contenttypeid\",\"datatype\":\"string\",\"nullable\":null,\"isprimarykey\":null},{\"name\":\"creator\",\"datatype\":\"string\",\"nullable\":null,\"isprimarykey\":null},{\"name\":\"x-parsed-by\",\"datatype\":\"string\",\"nullable\":null,\"isprimarykey\":null},{\"name\":\"content-type\",\"datatype\":\"string\",\"nullable\":null,\"isprimarykey\":null},{\"name\":\"modified\",\"datatype\":\"string\",\"nullable\":null,\"isprimarykey\":null},{\"name\":\"extended-properties:totaltime\",\"datatype\":\"string\",\"nullable\":null,\"isprimarykey\":null},{\"name\":\"application-name\",\"datatype\":\"string\",\"nullable\":null,\"isprimarykey\":null},{\"name\":\"meta:character-count-with-spaces\",\"datatype\":\"string\",\"nullable\":null,\"isprimarykey\":null},{\"name\":\"meta:save-date\",\"datatype\":\"string\",\"nullable\":null,\"isprimarykey\":null},{\"name\":\"paragraph-count\",\"datatype\":\"string\",\"nullable\":null,\"isprimarykey\":null},{\"name\":\"line-count\",\"datatype\":\"string\",\"nullable\":null,\"isprimarykey\":null},{\"name\":\"template\",\"datatype\":\"string\",\"nullable\":null,\"isprimarykey\":null},{\"name\":\"meta:character-count\",\"datatype\":\"string\",\"nullable\":null,\"isprimarykey\":null},{\"name\":\"last-save-date\",\"datatype\":\"string\",\"nullable\":null,\"isprimarykey\":null},{\"name\":\"last-modified\",\"datatype\":\"string\",\"nullable\":null,\"isprimarykey\":null},{\"name\":\"dcterms:modified\",\"datatype\":\"string\",\"nullable\":null,\"isprimarykey\":null},{\"name\":\"meta:line-count\",\"datatype\":\"string\",\"nullable\":null,\"isprimarykey\":null},{\"name\":\"dcterms:created\",\"datatype\":\"string\",\"nullable\":null,\"isprimarykey\":null},{\"name\":\"word-count\",\"datatype\":\"string\",\"nullable\":null,\"isprimarykey\":null},{\"name\":\"extended-properties:company\",\"datatype\":\"string\",\"nullable\":null,\"isprimarykey\":null},{\"name\":\"dc:creator\",\"datatype\":\"string\",\"nullable\":null,\"isprimarykey\":null},{\"name\":\"meta:word-count\",\"datatype\":\"string\",\"nullable\":null,\"isprimarykey\":null},{\"name\":\"meta:paragraph-count\",\"datatype\":\"string\",\"nullable\":null,\"isprimarykey\":null},{\"name\":\"extended-properties:appversion\",\"datatype\":\"string\",\"nullable\":null,\"isprimarykey\":null},{\"name\":\"total-time\",\"datatype\":\"string\",\"nullable\":null,\"isprimarykey\":null},{\"name\":\"cp:revision\",\"datatype\":\"string\",\"nullable\":null,\"isprimarykey\":null},{\"name\":\"fileinfo.group\",\"datatype\":\"string\",\"nullable\":null,\"isprimarykey\":null},{\"name\":\"fileinfo.isdirectory\",\"datatype\":\"boolean\",\"nullable\":null,\"isprimarykey\":null},{\"name\":\"fileinfo.isother\",\"datatype\":\"boolean\",\"nullable\":null,\"isprimarykey\":null},{\"name\":\"fileinfo.permissions\",\"datatype\":\"string\",\"nullable\":null,\"isprimarykey\":null},{\"name\":\"fileinfo.issymboliclink\",\"datatype\":\"boolean\",\"nullable\":null,\"isprimarykey\":null},{\"name\":\"fileinfo.size\",\"datatype\":\"integer\",\"nullable\":null,\"isprimarykey\":null},{\"name\":\"fileinfo.file\",\"datatype\":\"string\",\"nullable\":null,\"isprimarykey\":null},{\"name\":\"fileinfo.filename\",\"datatype\":\"string\",\"nullable\":null,\"isprimarykey\":null},{\"name\":\"fileinfo.filekey\",\"datatype\":\"string\",\"nullable\":null,\"isprimarykey\":null},{\"name\":\"fileinfo.isregularfile\",\"datatype\":\"boolean\",\"nullable\":null,\"isprimarykey\":null},{\"name\":\"fileinfo.creationtime\",\"datatype\":\"timestamp\",\"nullable\":null,\"isprimarykey\":null},{\"name\":\"fileinfo.lastmodifiedtime\",\"datatype\":\"timestamp\",\"nullable\":null,\"isprimarykey\":null},{\"name\":\"fileinfo.lastaccesstime\",\"datatype\":\"timestamp\",\"nullable\":null,\"isprimarykey\":null},{\"name\":\"fileinfo.owner\",\"datatype\":\"string\",\"nullable\":null,\"isprimarykey\":null},{\"name\":\"topsecretboolean\",\"datatype\":\"double precision\",\"nullable\":null,\"isprimarykey\":null},{\"name\":\"secretboolean\",\"datatype\":\"string\",\"nullable\":null,\"isprimarykey\":null},{\"name\":\"confidentialsecretboolean\",\"datatype\":\"double precision\",\"nullable\":null,\"isprimarykey\":null},{\"name\":\"classificationlevel\",\"datatype\":\"string\",\"nullable\":null,\"isprimarykey\":null},{\"name\":\"releasability\",\"datatype\":\"string\",\"nullable\":null,\"isprimarykey\":null},{\"name\":\"compartment\",\"datatype\":\"string\",\"nullable\":null,\"isprimarykey\":null}]",
"id": 41,
"createdAt": "2021-10-01T18:15:09.139Z",
"updatedAt": "2021-10-01T18:15:09.274Z",
"dbms": {
"name": "redshift"
}
}
Manage data sources
| Method | Path | Purpose |
|---|---|---|
| PUT | /redshift/handler/{handlerId} |
Update the data source metadata associated with the provided handler ID. This endpoint does not perform partial updates, but will allow the dictionary to be omitted. In this case, it uses the current dictionary. |
| PUT | /redshift/bulk |
Update the data source metadata associated with the provided connection string. |
| PUT | /redshift/handler/{handlerId}/triggerHighCardinalityJob |
Recalculate the high cardinality column for the specified data source. |
| PUT | /redshift/handler/{handlerId}/refreshNativeViewJob |
Refresh the native view of a data source. |
Update a specific data source
Endpoint
| Method | Path | Purpose |
|---|---|---|
| PUT | /redshift/handler/{handlerId} |
Update the data source metadata associated with the provided handler ID. This endpoint does not perform partial updates, but will allow the dictionary to be omitted. In this case, it uses the current dictionary. |
Query Parameters
| Attribute | Description | Required |
|---|---|---|
| handlerId | integer The ID of the handler. |
Yes |
| skipCache | boolean When true, will skip the handler cache when retrieving metadata. |
No |
Payload Parameters
| Attribute | Description | Required |
|---|---|---|
| handler | metadata Includes metadata about the handler, such as ssl, port, database, hostname, username, and password. |
Yes |
| connectionString | string The connection string used to connect to the data source. |
Yes |
Response Parameters
| Attribute | Description |
|---|---|
| id | integer The ID of the handler. |
| ca | string The certificate authority. |
| columns | array[object] This is a Data Dictionary object, which provides metadata about the columns in the data source, including the name and data type of the column. |
Request example
This request updates the metadata for the data source with the handler ID 41.
curl \
--request PUT \
--header "Content-Type: application/json" \
--header "Authorization: Bearer dea464c07bd07300095caa8" \
--data @example-payload.json \
https://demo.immuta.com/redshift/handler/41
Payload example
The payload below removes the paragraph_count column from the data source.
{
"handler": {
"policyHandler": null,
"dataSourceId": 41,
"metadata": {
"ssl": true,
"port": 5439,
"table": "data",
"schema": "army",
"database": "test",
"hostname": "redshift.database.example.io",
"username": "user",
"eventTime": "fileinfo.creationtime",
"userFiles": [],
"dataSourceName": "Army Data",
"bodataTableName": "data",
"bodataSchemaName": "army",
"columnsNormalized": false,
"schemaProjectName": "Army",
"staleDataTolerance": 86400,
"columns": [{
"name": "text",
"dataType": "text",
"remoteType": "string",
"nullable": true
}, {
"name": "date",
"dataType": "text",
"remoteType": "string",
"nullable": true
}, {
"name": "hashed_id",
"dataType": "text",
"remoteType": "string",
"nullable": true
}, {
"name": "filename",
"dataType": "text",
"remoteType": "string",
"nullable": true
}, {
"name": "storage_location",
"dataType": "text",
"remoteType": "string",
"nullable": true
}, {
"name": "size",
"dataType": "integer",
"remoteType": "integer",
"nullable": true
}, {
"name": "owner",
"dataType": "text",
"remoteType": "string",
"nullable": true
}, {
"name": "dc:publisher",
"dataType": "text",
"remoteType": "string",
"nullable": true
}, {
"name": "meta:page-count",
"dataType": "text",
"remoteType": "string",
"nullable": true
}, {
"name": "publisher",
"dataType": "text",
"remoteType": "string",
"nullable": true
}, {
"name": "author",
"dataType": "text",
"remoteType": "string",
"nullable": true
}, {
"name": "extended-properties:template",
"dataType": "text",
"remoteType": "string",
"nullable": true
}, {
"name": "application-version",
"dataType": "text",
"remoteType": "string",
"nullable": true
}, {
"name": "revision-number",
"dataType": "text",
"remoteType": "string",
"nullable": true
}, {
"name": "page-count",
"dataType": "text",
"remoteType": "string",
"nullable": true
}, {
"name": "last-author",
"dataType": "text",
"remoteType": "string",
"nullable": true
}, {
"name": "character-count-with-spaces",
"dataType": "text",
"remoteType": "string",
"nullable": true
}, {
"name": "xmptpg:npages",
"dataType": "text",
"remoteType": "string",
"nullable": true
}, {
"name": "creation-date",
"dataType": "text",
"remoteType": "string",
"nullable": true
}, {
"name": "meta:last-author",
"dataType": "text",
"remoteType": "string",
"nullable": true
}, {
"name": "extended-properties:application",
"dataType": "text",
"remoteType": "string",
"nullable": true
}, {
"name": "meta:creation-date",
"dataType": "text",
"remoteType": "string",
"nullable": true
}, {
"name": "meta:author",
"dataType": "text",
"remoteType": "string",
"nullable": true
}, {
"name": "custom:contenttypeid",
"dataType": "text",
"remoteType": "string",
"nullable": true
}, {
"name": "creator",
"dataType": "text",
"remoteType": "string",
"nullable": true
}, {
"name": "x-parsed-by",
"dataType": "text",
"remoteType": "string",
"nullable": true
}, {
"name": "content-type",
"dataType": "text",
"remoteType": "string",
"nullable": true
}, {
"name": "modified",
"dataType": "text",
"remoteType": "string",
"nullable": true
}, {
"name": "extended-properties:totaltime",
"dataType": "text",
"remoteType": "string",
"nullable": true
}, {
"name": "application-name",
"dataType": "text",
"remoteType": "string",
"nullable": true
}, {
"name": "meta:character-count-with-spaces",
"dataType": "text",
"remoteType": "string",
"nullable": true
}, {
"name": "meta:save-date",
"dataType": "text",
"remoteType": "string",
"nullable": true
}, {
"name": "line-count",
"dataType": "text",
"remoteType": "string",
"nullable": true
}, {
"name": "template",
"dataType": "text",
"remoteType": "string",
"nullable": true
}, {
"name": "meta:character-count",
"dataType": "text",
"remoteType": "string",
"nullable": true
}, {
"name": "last-save-date",
"dataType": "text",
"remoteType": "string",
"nullable": true
}, {
"name": "last-modified",
"dataType": "text",
"remoteType": "string",
"nullable": true
}, {
"name": "dcterms:modified",
"dataType": "text",
"remoteType": "string",
"nullable": true
}, {
"name": "meta:line-count",
"dataType": "text",
"remoteType": "string",
"nullable": true
}, {
"name": "dcterms:created",
"dataType": "text",
"remoteType": "string",
"nullable": true
}, {
"name": "word-count",
"dataType": "text",
"remoteType": "string",
"nullable": true
}, {
"name": "extended-properties:company",
"dataType": "text",
"remoteType": "string",
"nullable": true
}, {
"name": "dc:creator",
"dataType": "text",
"remoteType": "string",
"nullable": true
}, {
"name": "meta:word-count",
"dataType": "text",
"remoteType": "string",
"nullable": true
}, {
"name": "meta:paragraph-count",
"dataType": "text",
"remoteType": "string",
"nullable": true
}, {
"name": "extended-properties:appversion",
"dataType": "text",
"remoteType": "string",
"nullable": true
}, {
"name": "total-time",
"dataType": "text",
"remoteType": "string",
"nullable": true
}, {
"name": "cp:revision",
"dataType": "text",
"remoteType": "string",
"nullable": true
}, {
"name": "fileinfo.group",
"dataType": "text",
"remoteType": "string",
"nullable": true
}, {
"name": "fileinfo.isdirectory",
"dataType": "boolean",
"remoteType": "boolean",
"nullable": true
}, {
"name": "fileinfo.isother",
"dataType": "boolean",
"remoteType": "boolean",
"nullable": true
}, {
"name": "fileinfo.permissions",
"dataType": "text",
"remoteType": "string",
"nullable": true
}, {
"name": "fileinfo.issymboliclink",
"dataType": "boolean",
"remoteType": "boolean",
"nullable": true
}, {
"name": "fileinfo.size",
"dataType": "integer",
"remoteType": "integer",
"nullable": true
}, {
"name": "fileinfo.file",
"dataType": "text",
"remoteType": "string",
"nullable": true
}, {
"name": "fileinfo.filename",
"dataType": "text",
"remoteType": "string",
"nullable": true
}, {
"name": "fileinfo.filekey",
"dataType": "text",
"remoteType": "string",
"nullable": true
}, {
"name": "fileinfo.isregularfile",
"dataType": "boolean",
"remoteType": "boolean",
"nullable": true
}, {
"name": "fileinfo.creationtime",
"dataType": "timestamp",
"remoteType": "timestamp",
"nullable": true
}, {
"name": "fileinfo.lastmodifiedtime",
"dataType": "timestamp",
"remoteType": "timestamp",
"nullable": true
}, {
"name": "fileinfo.lastaccesstime",
"dataType": "timestamp",
"remoteType": "timestamp",
"nullable": true
}, {
"name": "fileinfo.owner",
"dataType": "text",
"remoteType": "string",
"nullable": true
}, {
"name": "topsecretboolean",
"dataType": "double precision",
"remoteType": "double precision",
"nullable": true
}, {
"name": "secretboolean",
"dataType": "text",
"remoteType": "string",
"nullable": true
}, {
"name": "confidentialsecretboolean",
"dataType": "double precision",
"remoteType": "double precision",
"nullable": true
}, {
"name": "classificationlevel",
"dataType": "text",
"remoteType": "string",
"nullable": true
}, {
"name": "releasability",
"dataType": "text",
"remoteType": "string",
"nullable": true
}, {
"name": "compartment",
"dataType": "text",
"remoteType": "string",
"nullable": true
}],
"password": "yourpassword"
},
"type": "odbcHandler",
"connectionString": "user@redshift.database.example.io:5439/test",
"remoteTableDescription": "[{\"name\":\"text\",\"datatype\":\"string\",\"nullable\":null,\"isprimarykey\":null},{\"name\":\"date\",\"datatype\":\"string\",\"nullable\":null,\"isprimarykey\":null},{\"name\":\"hashed_id\",\"datatype\":\"string\",\"nullable\":null,\"isprimarykey\":null},{\"name\":\"filename\",\"datatype\":\"string\",\"nullable\":null,\"isprimarykey\":null},{\"name\":\"storage_location\",\"datatype\":\"string\",\"nullable\":null,\"isprimarykey\":null},{\"name\":\"size\",\"datatype\":\"integer\",\"nullable\":null,\"isprimarykey\":null},{\"name\":\"owner\",\"datatype\":\"string\",\"nullable\":null,\"isprimarykey\":null},{\"name\":\"dc:publisher\",\"datatype\":\"string\",\"nullable\":null,\"isprimarykey\":null},{\"name\":\"meta:page-count\",\"datatype\":\"string\",\"nullable\":null,\"isprimarykey\":null},{\"name\":\"publisher\",\"datatype\":\"string\",\"nullable\":null,\"isprimarykey\":null},{\"name\":\"author\",\"datatype\":\"string\",\"nullable\":null,\"isprimarykey\":null},{\"name\":\"extended-properties:template\",\"datatype\":\"string\",\"nullable\":null,\"isprimarykey\":null},{\"name\":\"application-version\",\"datatype\":\"string\",\"nullable\":null,\"isprimarykey\":null},{\"name\":\"revision-number\",\"datatype\":\"string\",\"nullable\":null,\"isprimarykey\":null},{\"name\":\"page-count\",\"datatype\":\"string\",\"nullable\":null,\"isprimarykey\":null},{\"name\":\"last-author\",\"datatype\":\"string\",\"nullable\":null,\"isprimarykey\":null},{\"name\":\"character-count-with-spaces\",\"datatype\":\"string\",\"nullable\":null,\"isprimarykey\":null},{\"name\":\"xmptpg:npages\",\"datatype\":\"string\",\"nullable\":null,\"isprimarykey\":null},{\"name\":\"creation-date\",\"datatype\":\"string\",\"nullable\":null,\"isprimarykey\":null},{\"name\":\"meta:last-author\",\"datatype\":\"string\",\"nullable\":null,\"isprimarykey\":null},{\"name\":\"extended-properties:application\",\"datatype\":\"string\",\"nullable\":null,\"isprimarykey\":null},{\"name\":\"meta:creation-date\",\"datatype\":\"string\",\"nullable\":null,\"isprimarykey\":null},{\"name\":\"meta:author\",\"datatype\":\"string\",\"nullable\":null,\"isprimarykey\":null},{\"name\":\"custom:contenttypeid\",\"datatype\":\"string\",\"nullable\":null,\"isprimarykey\":null},{\"name\":\"creator\",\"datatype\":\"string\",\"nullable\":null,\"isprimarykey\":null},{\"name\":\"x-parsed-by\",\"datatype\":\"string\",\"nullable\":null,\"isprimarykey\":null},{\"name\":\"content-type\",\"datatype\":\"string\",\"nullable\":null,\"isprimarykey\":null},{\"name\":\"modified\",\"datatype\":\"string\",\"nullable\":null,\"isprimarykey\":null},{\"name\":\"extended-properties:totaltime\",\"datatype\":\"string\",\"nullable\":null,\"isprimarykey\":null},{\"name\":\"application-name\",\"datatype\":\"string\",\"nullable\":null,\"isprimarykey\":null},{\"name\":\"meta:character-count-with-spaces\",\"datatype\":\"string\",\"nullable\":null,\"isprimarykey\":null},{\"name\":\"meta:save-date\",\"datatype\":\"string\",\"nullable\":null,\"isprimarykey\":null},{\"name\":\"paragraph-count\",\"datatype\":\"string\",\"nullable\":null,\"isprimarykey\":null},{\"name\":\"line-count\",\"datatype\":\"string\",\"nullable\":null,\"isprimarykey\":null},{\"name\":\"template\",\"datatype\":\"string\",\"nullable\":null,\"isprimarykey\":null},{\"name\":\"meta:character-count\",\"datatype\":\"string\",\"nullable\":null,\"isprimarykey\":null},{\"name\":\"last-save-date\",\"datatype\":\"string\",\"nullable\":null,\"isprimarykey\":null},{\"name\":\"last-modified\",\"datatype\":\"string\",\"nullable\":null,\"isprimarykey\":null},{\"name\":\"dcterms:modified\",\"datatype\":\"string\",\"nullable\":null,\"isprimarykey\":null},{\"name\":\"meta:line-count\",\"datatype\":\"string\",\"nullable\":null,\"isprimarykey\":null},{\"name\":\"dcterms:created\",\"datatype\":\"string\",\"nullable\":null,\"isprimarykey\":null},{\"name\":\"word-count\",\"datatype\":\"string\",\"nullable\":null,\"isprimarykey\":null},{\"name\":\"extended-properties:company\",\"datatype\":\"string\",\"nullable\":null,\"isprimarykey\":null},{\"name\":\"dc:creator\",\"datatype\":\"string\",\"nullable\":null,\"isprimarykey\":null},{\"name\":\"meta:word-count\",\"datatype\":\"string\",\"nullable\":null,\"isprimarykey\":null},{\"name\":\"meta:paragraph-count\",\"datatype\":\"string\",\"nullable\":null,\"isprimarykey\":null},{\"name\":\"extended-properties:appversion\",\"datatype\":\"string\",\"nullable\":null,\"isprimarykey\":null},{\"name\":\"total-time\",\"datatype\":\"string\",\"nullable\":null,\"isprimarykey\":null},{\"name\":\"cp:revision\",\"datatype\":\"string\",\"nullable\":null,\"isprimarykey\":null},{\"name\":\"fileinfo.group\",\"datatype\":\"string\",\"nullable\":null,\"isprimarykey\":null},{\"name\":\"fileinfo.isdirectory\",\"datatype\":\"boolean\",\"nullable\":null,\"isprimarykey\":null},{\"name\":\"fileinfo.isother\",\"datatype\":\"boolean\",\"nullable\":null,\"isprimarykey\":null},{\"name\":\"fileinfo.permissions\",\"datatype\":\"string\",\"nullable\":null,\"isprimarykey\":null},{\"name\":\"fileinfo.issymboliclink\",\"datatype\":\"boolean\",\"nullable\":null,\"isprimarykey\":null},{\"name\":\"fileinfo.size\",\"datatype\":\"integer\",\"nullable\":null,\"isprimarykey\":null},{\"name\":\"fileinfo.file\",\"datatype\":\"string\",\"nullable\":null,\"isprimarykey\":null},{\"name\":\"fileinfo.filename\",\"datatype\":\"string\",\"nullable\":null,\"isprimarykey\":null},{\"name\":\"fileinfo.filekey\",\"datatype\":\"string\",\"nullable\":null,\"isprimarykey\":null},{\"name\":\"fileinfo.isregularfile\",\"datatype\":\"boolean\",\"nullable\":null,\"isprimarykey\":null},{\"name\":\"fileinfo.creationtime\",\"datatype\":\"timestamp\",\"nullable\":null,\"isprimarykey\":null},{\"name\":\"fileinfo.lastmodifiedtime\",\"datatype\":\"timestamp\",\"nullable\":null,\"isprimarykey\":null},{\"name\":\"fileinfo.lastaccesstime\",\"datatype\":\"timestamp\",\"nullable\":null,\"isprimarykey\":null},{\"name\":\"fileinfo.owner\",\"datatype\":\"string\",\"nullable\":null,\"isprimarykey\":null},{\"name\":\"topsecretboolean\",\"datatype\":\"double precision\",\"nullable\":null,\"isprimarykey\":null},{\"name\":\"secretboolean\",\"datatype\":\"string\",\"nullable\":null,\"isprimarykey\":null},{\"name\":\"confidentialsecretboolean\",\"datatype\":\"double precision\",\"nullable\":null,\"isprimarykey\":null},{\"name\":\"classificationlevel\",\"datatype\":\"string\",\"nullable\":null,\"isprimarykey\":null},{\"name\":\"releasability\",\"datatype\":\"string\",\"nullable\":null,\"isprimarykey\":null},{\"name\":\"compartment\",\"datatype\":\"string\",\"nullable\":null,\"isprimarykey\":null}]",
"id": 41,
"createdAt": "2021-10-01T18:15:09.139Z",
"updatedAt": "2021-10-01T18:15:09.274Z",
"dbms": {
"name": "redshift"
}
}
}
Response example
{
"id": 41,
"ca": ["-----BEGIN CERTIFICATE-----\ncertificatedata\n-----END CERTIFICATE-----"],
"metadata": {
"columns": [{
"name": "text",
"dataType": "text",
"remoteType": "string",
"nullable": true
}, {
"name": "date",
"dataType": "text",
"remoteType": "string",
"nullable": true
}, {
"name": "hashed_id",
"dataType": "text",
"remoteType": "string",
"nullable": true
}, {
"name": "filename",
"dataType": "text",
"remoteType": "string",
"nullable": true
}, {
"name": "storage_location",
"dataType": "text",
"remoteType": "string",
"nullable": true
}, {
"name": "size",
"dataType": "integer",
"remoteType": "integer",
"nullable": true
}, {
"name": "owner",
"dataType": "text",
"remoteType": "string",
"nullable": true
}, {
"name": "dc:publisher",
"dataType": "text",
"remoteType": "string",
"nullable": true
}, {
"name": "meta:page-count",
"dataType": "text",
"remoteType": "string",
"nullable": true
}, {
"name": "publisher",
"dataType": "text",
"remoteType": "string",
"nullable": true
}, {
"name": "author",
"dataType": "text",
"remoteType": "string",
"nullable": true
}, {
"name": "extended-properties:template",
"dataType": "text",
"remoteType": "string",
"nullable": true
}, {
"name": "application-version",
"dataType": "text",
"remoteType": "string",
"nullable": true
}, {
"name": "revision-number",
"dataType": "text",
"remoteType": "string",
"nullable": true
}, {
"name": "page-count",
"dataType": "text",
"remoteType": "string",
"nullable": true
}, {
"name": "last-author",
"dataType": "text",
"remoteType": "string",
"nullable": true
}, {
"name": "character-count-with-spaces",
"dataType": "text",
"remoteType": "string",
"nullable": true
}, {
"name": "xmptpg:npages",
"dataType": "text",
"remoteType": "string",
"nullable": true
}, {
"name": "creation-date",
"dataType": "text",
"remoteType": "string",
"nullable": true
}, {
"name": "meta:last-author",
"dataType": "text",
"remoteType": "string",
"nullable": true
}, {
"name": "extended-properties:application",
"dataType": "text",
"remoteType": "string",
"nullable": true
}, {
"name": "meta:creation-date",
"dataType": "text",
"remoteType": "string",
"nullable": true
}, {
"name": "meta:author",
"dataType": "text",
"remoteType": "string",
"nullable": true
}, {
"name": "custom:contenttypeid",
"dataType": "text",
"remoteType": "string",
"nullable": true
}, {
"name": "creator",
"dataType": "text",
"remoteType": "string",
"nullable": true
}, {
"name": "x-parsed-by",
"dataType": "text",
"remoteType": "string",
"nullable": true
}, {
"name": "content-type",
"dataType": "text",
"remoteType": "string",
"nullable": true
}, {
"name": "modified",
"dataType": "text",
"remoteType": "string",
"nullable": true
}, {
"name": "extended-properties:totaltime",
"dataType": "text",
"remoteType": "string",
"nullable": true
}, {
"name": "application-name",
"dataType": "text",
"remoteType": "string",
"nullable": true
}, {
"name": "meta:character-count-with-spaces",
"dataType": "text",
"remoteType": "string",
"nullable": true
}, {
"name": "meta:save-date",
"dataType": "text",
"remoteType": "string",
"nullable": true
}, {
"name": "line-count",
"dataType": "text",
"remoteType": "string",
"nullable": true
}, {
"name": "template",
"dataType": "text",
"remoteType": "string",
"nullable": true
}, {
"name": "meta:character-count",
"dataType": "text",
"remoteType": "string",
"nullable": true
}, {
"name": "last-save-date",
"dataType": "text",
"remoteType": "string",
"nullable": true
}, {
"name": "last-modified",
"dataType": "text",
"remoteType": "string",
"nullable": true
}, {
"name": "dcterms:modified",
"dataType": "text",
"remoteType": "string",
"nullable": true
}, {
"name": "meta:line-count",
"dataType": "text",
"remoteType": "string",
"nullable": true
}, {
"name": "dcterms:created",
"dataType": "text",
"remoteType": "string",
"nullable": true
}, {
"name": "word-count",
"dataType": "text",
"remoteType": "string",
"nullable": true
}, {
"name": "extended-properties:company",
"dataType": "text",
"remoteType": "string",
"nullable": true
}, {
"name": "dc:creator",
"dataType": "text",
"remoteType": "string",
"nullable": true
}, {
"name": "meta:word-count",
"dataType": "text",
"remoteType": "string",
"nullable": true
}, {
"name": "meta:paragraph-count",
"dataType": "text",
"remoteType": "string",
"nullable": true
}, {
"name": "extended-properties:appversion",
"dataType": "text",
"remoteType": "string",
"nullable": true
}, {
"name": "total-time",
"dataType": "text",
"remoteType": "string",
"nullable": true
}, {
"name": "cp:revision",
"dataType": "text",
"remoteType": "string",
"nullable": true
}, {
"name": "fileinfo.group",
"dataType": "text",
"remoteType": "string",
"nullable": true
}, {
"name": "fileinfo.isdirectory",
"dataType": "boolean",
"remoteType": "boolean",
"nullable": true
}, {
"name": "fileinfo.isother",
"dataType": "boolean",
"remoteType": "boolean",
"nullable": true
}, {
"name": "fileinfo.permissions",
"dataType": "text",
"remoteType": "string",
"nullable": true
}, {
"name": "fileinfo.issymboliclink",
"dataType": "boolean",
"remoteType": "boolean",
"nullable": true
}, {
"name": "fileinfo.size",
"dataType": "integer",
"remoteType": "integer",
"nullable": true
}, {
"name": "fileinfo.file",
"dataType": "text",
"remoteType": "string",
"nullable": true
}, {
"name": "fileinfo.filename",
"dataType": "text",
"remoteType": "string",
"nullable": true
}, {
"name": "fileinfo.filekey",
"dataType": "text",
"remoteType": "string",
"nullable": true
}, {
"name": "fileinfo.isregularfile",
"dataType": "boolean",
"remoteType": "boolean",
"nullable": true
}, {
"name": "fileinfo.creationtime",
"dataType": "timestamp",
"remoteType": "timestamp",
"nullable": true
}, {
"name": "fileinfo.lastmodifiedtime",
"dataType": "timestamp",
"remoteType": "timestamp",
"nullable": true
}, {
"name": "fileinfo.lastaccesstime",
"dataType": "timestamp",
"remoteType": "timestamp",
"nullable": true
}, {
"name": "fileinfo.owner",
"dataType": "text",
"remoteType": "string",
"nullable": true
}, {
"name": "topsecretboolean",
"dataType": "double precision",
"remoteType": "double precision",
"nullable": true
}, {
"name": "secretboolean",
"dataType": "text",
"remoteType": "string",
"nullable": true
}, {
"name": "confidentialsecretboolean",
"dataType": "double precision",
"remoteType": "double precision",
"nullable": true
}, {
"name": "classificationlevel",
"dataType": "text",
"remoteType": "string",
"nullable": true
}, {
"name": "releasability",
"dataType": "text",
"remoteType": "string",
"nullable": true
}, {
"name": "compartment",
"dataType": "text",
"remoteType": "string",
"nullable": true
}]
}
}
Update multiple data sources
Endpoint
| Method | Path | Purpose |
|---|---|---|
| PUT | /redshift/bulk |
Update the data source metadata associated with the provided connection string. |
Query Parameters
None.
Payload Parameters
| Attribute | Description | Required |
|---|---|---|
| handler | metadata Includes metadata about the handler, such as ssl, port, database, hostname, username, and password. |
Yes |
| connectionString | string The connection string used to connect to the data sources. |
Yes |
Response Parameters
| Attribute | Description |
|---|---|
| bulkId | string The ID of the bulk data source update. |
| connectionString | string The connection string shared by the data sources bulk updated. |
| jobsCreated | integer The number of jobs that ran to update the data sources; this number corresponds to the number of data sources updated. |
Request example
This request updates the metadata for all data sources with the connection string specified in example-payload.json.
curl \
--request PUT \
--header "Content-Type: application/json" \
--header "Authorization: Bearer dea464c07bd07300095caa8" \
--data @example-payload.json \
https://demo.immuta.com/redshift/bulk
Payload example
The payload below adds a certificate (certificate.json) to connect to the data sources with the provided connection
string.
{
"handler": {
"metadata": {
"ssl": true,
"port": 5439,
"database": "test",
"hostname": "redshift.database.example.io",
"username": "user",
"userFiles": [{
"keyName": "testcert",
"filename": "bf349cbbfd3bc4eaf4ace7824a599fbc7cb5f7d5.json",
"userFilename": "certificate.json"
}],
"password": "yourpassword"
}
},
"connectionString": "user@sredshift.database.example.io:5000/test"
}
Response example
{
"bulkId": "bulk_ds_update_f50a8010dca03c07bcd0dc55eaf13c07",
"connectionString": "yourusername@redshift.database.example.io:5000/tpc",
"jobsCreated": 2
}
Recalculate the high cardinality column for a data source
Endpoint
| Method | Path | Purpose |
|---|---|---|
| PUT | /redshift/handler/{handlerId}/triggerHighCardinalityJob |
Recalculate the high cardinality column for the specified data source. |
Query Parameters
| Attribute | Description | Required |
|---|---|---|
| handlerId | integer The ID of the handler. |
Yes |
Response Parameters
The response returns a string of characters that identify the high cardinality job run.
Request example
This request re-runs the job that calculates the high cardinality column for the data source with the handler ID 41.
curl \
--request PUT \
--header "Content-Type: application/json" \
--header "Authorization: Bearer dea464c07bd07300095caa8" \
https://demo.immuta.com/redshift/handler/41/triggerHighCardinalityJob
Response example
0f5f08d0-22ee-11ec-b2b8-874838eeef05
Refresh a native view
Endpoint
| Method | Path | Purpose |
|---|---|---|
| PUT | /redshift/handler/{handlerId}/refreshNativeViewJob |
Refresh the native view of a data source. |
Query Parameters
| Attribute | Description | Required |
|---|---|---|
| handlerId | integer The ID of the handler. |
Yes |
Response Parameters
The response returns a string of characters that identifies the refresh view job run.
Request example
This request refreshes the view for the data source with the handler ID 7.
curl \
--request PUT \
--header "Content-Type: application/json" \
--header "Authorization: Bearer dea464c07bd07300095caa8" \
https://demo.immuta.com/redshift/handler/7/refreshNativeViewJob
Response example
53c256d0-eb57-11ec-b275-d95a8e998142