S3 API Support

The SwiftStack S3 API support provides Amazon S3 API compatibility. This functionality is enabled by default but can be disabled. This will allow end users the ability to access objects in SwiftStack using software designed to interact with S3-compatible endpoints.

To get to the configuration page, sign into your SwiftStack Controller account, click on the Clusters tab and then click Manage for the cluster. Click the Middleware tab. Under the Available Swift Middleware section, locate and click on the S3 API Support row.

Don't forget to Deploy Middleware Changes to apply any changes.

S3 Account Secret Key

In order to use the S3 middleware, the end user must also get an S3 key. This key will be used for signing authentication requests, in the same way that Amazon uses the secret access key for signing requests.

The S3 key can be found in the Accounts page under the Cluster List page.

S3 Account Secret Key

Cross-Account Access

Unlike S3, bucket names in SwiftStack are not globally unique. For example, users alice and bob can each have their own bucket named test-bucket, which may map to the containers /v1/AUTH_alice/test-bucket and /v1/AUTH_bob/test-bucket in SwiftStack. If Alice sets appropriate ACLs on her buckets, however, Bob may still access them by using the access key bob:AUTH_alice instead of his normal bob.

Configuring S3 Clients

To find the S3 API URL, navigate to the Cluster detail page on the SwiftStack Controller. On this page is a list of URLs used when configuring storage.

For S3 API access, copy the S3 API URL as the SwiftStack cluster endpoint and the S3 API Region for region/location configuration.

s3cmd

http://s3tools.org/s3cmd

S3cmd is a command line tool and client for uploading, retrieving and managing data in Amazon S3 and other cloud storage service providers that use the S3 protocol. This tool looks for config parameters in ~/.s3cfg. You can create it with s3cmd --configure. A sample config is provided below. There are many options that may be configured; the ones shown below are required for basic functionality.

[default]
access_key = <your username here>
host_base = <your swiftstack cluster endpoint hostname>
host_bucket = <your swiftstack cluster endpoint hostname>
secret_key = <your S3 API key here>

By default, s3cmd connects using HTTPS. To override this, additionally set:

use_https = False

With these config values, you can start using s3cmd to talk to the SwiftStack cluster using the S3 API:

$ s3cmd ls
2009-02-03 16:45  s3://somebucket
$ s3cmd ls s3://somebucket

awscli

awscli is Amazon's official command line interface for their web services. With awscli-endpoint-plugin, it's easy to configure alternate endpoints for different profiles. This tool looks for config parameters in ~/.aws/config. A sample config is provided below. There are many options that are settable; the ones shown below are required for basic functionality.

[plugins]
endpoint = awscli_plugin_endpoint

[profile default]
aws_access_key_id = <your username here>
aws_secret_access_key = <your S3 API key here>
region = <the "location" configured for your cluster>
s3 =
    endpoint_url = https://<your swiftstack cluster endpoint hostname>

With this config file, you can interact with the cluster normally:

$ aws s3 mb s3://test-bucket
make_bucket: test-bucket
$ aws s3 ls
2009-02-03 08:45:09 test-bucket
$ aws s3 cp file.txt s3://test-bucket/obj.txt
upload: ./file.txt to s3://test-bucket/obj.txt
$ aws s3 ls s3://test-bucket
2017-07-27 11:29:29         28 obj.txt

Boto3

Boto3 is a newer rewrite of Boto. This tool looks for config parameters in ~/.aws/config. A sample config is provided below.

[default]
aws_access_key_id = <your username here>
aws_secret_access_key = <your S3 API key here>
region = <the "location" configured for your cluster>

With this config file, you only need to modify your application to use a different endpoint_url (which includes specifying whether to use HTTP or HTTPS):

#!/usr/bin/env python
import io, boto3

session = boto3.Session()
s3 = session.resource('s3', endpoint_url='http://<your swiftstack cluster endpoint hostname>')

bucket = s3.create_bucket(Bucket='test-bucket')
print(list(s3.buckets.all()))

bucket.upload_fileobj(Key='test-object',
                      Fileobj=io.BytesIO(b'Hello, world!'))
print(list(bucket.objects.all()))

key = bucket.Object('test-object')
print(key.get()['Body'].read())

key.delete()
bucket.delete()
print(list(s3.buckets.all()))