Skip to content

You are viewing documentation for Immuta version 2024.1.

For the latest version, view our documentation for Immuta SaaS or the latest self-hosted version.

cancel

S3 Integration API

These endpoints allow you to communicate with Immuta the same way you would with S3, making Immuta easy to integrate with tools you may already be using to work with S3. In this integration, Immuta implements a single bucket (with data sources broken up as sub-directories under that bucket), since some S3 tools only support the new virtual-hosted style requests.

The endpoints outlined below support basic AWS functionality; the requests and responses for each are identical to those in S3.

Note

Additional fields may be included in some responses you receive; however, these attributes are for internal purposes and are therefore undocumented.

S3 integration workflow

  1. List the bucket all data sources are grouped under.
  2. List the contents of a specific bucket.
  3. List the contents of a specific blob.

List the bucket all data sources are grouped under

Endpoint

Method Path Purpose
GET /s3p Return constant bucket that all data sources are grouped under.

Query Parameters

None.

Response Parameters

Attribute Description
body string Lists the bucket, organization, and owner of the bucket.

Request example

The following request returns the bucket all data sources are grouped under for the organization.

curl \
    --request GET \
    --header "Authorization: AWS <API KEY>:immuta" \
    https://demo.immuta.com/s3p

Response example

The following response lists the bucket immuta.

<?xml version="1.0" encoding="UTF-8"?>
<ListAllMyBucketsResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
    <Owner>
        <ID>Immuta</ID>
        <DisplayName>Immuta Inc</DisplayName>
    </Owner>
    <Buckets>
        <Bucket>
            <Name>immuta</Name>
            <CreationDate>2006-03-19T00:00:00.000Z</CreationDate>
        </Bucket>
    </Buckets>
</ListAllMyBucketsResult>

List the contents of a specific bucket

Endpoint

Method Path Purpose
GET /s3p/{bucket} Return the contents of a given bucket (data sources or data source blobs).

Query Parameters

Attribute Description Required
bucket string Name of the bucket (this should match the bucket name in configuration). Yes
prefix string The prefix to start a search under. No
max-keys number Max number of content objects to return. No
delimiter string The terminating character for a search. No
marker string When provided, only prefixes after this marker will be returned. No
location string If provided, will instead return the bucket location. No
encoding-type string Encoding type for content keys. No
list-type integer If set to 2, will use v2 of the API response. No
fetch-owner boolean When true, will return owner field in v2 API response. No
start-after string Only prefixes after this marker will be returned. (This is the v2 API version of marker). No

Response Parameters

Attribute Description
body string Lists the data sources and/or data source blobs in the bucket.

Request example

The following request returns a list of data sources in the bucket immuta.

curl \
    --request GET \
    --header "Authorization: AWS <API KEY>:immuta" \
    https://demo.immuta.com/s3p/immuta

Response example

The following response lists the data source blob immuta.

<?xml version="1.0" encoding="UTF-8"?>
<ListBucketResult xmlns="http://doc.s3.amazonaws.com/2006-03-01/">
    <IsTruncated>false</IsTruncated>
    <Marker></Marker>
    <Name>immuta</Name>
    <Prefix>/</Prefix>
    <MaxKeys>1000</MaxKeys>
    <Delimiter></Delimiter>
</ListBucketResult>

List the contents of a specific blob

Endpoint

Method Path Purpose
GET /s3p/{bucket}/{dataSource}/{key} Return a specific blob.

Query Parameters

Attribute Description Required
bucket string Name of the bucket (this should match the bucket name in configuration). Yes
dataSource string Immuta S3 Folder to search in. Yes
key string Key path to search for the specified file. Yes

Response Parameters

Attribute Description
body string Lists the data sources and/or data source blobs in the bucket.

Request example

The following request returns the contents of the single file that exists in the requested directory in immuta.

curl \
    --request GET \
    --header "Authorization: AWS <API KEY>:immuta" \
    https://demo.immuta.com/s3p/immuta/my_data_source/path/to/file/myfile.json

Response example

The following response lists the size, storage class, owner, and the last date of modification of the single file.

<?xml version="1.0" encoding="UTF-8"?>
<ListBucketResult xmlns="http://doc.s3.amazonaws.com/2006-03-01/">
    <IsTruncated>false</IsTruncated>
    <Marker></Marker>
    <Name>immuta</Name>
    <Prefix>my_data_source/path/to/file</Prefix>
    <MaxKeys>1000</MaxKeys>
    <Delimiter>/</Delimiter>
    <Contents>
        <Key>my_data_source/path/to/file/myfile.json</Key>
        <LastModified>2018-11-05T21:25:04.000Z</LastModified>
        <ETag>5b0810c82a69a70e552cece19b20585fc94b67fe4eaa8b</ETag>
        <Size>389</Size>
        <StorageClass>STANDARD</StorageClass>
        <Owner>
            <ID>Immuta</ID>
            <DisplayName>Immuta</DisplayName>
        </Owner>
    </Contents>
</ListBucketResult>