Developer
Search…
Welcome
The StoreWise Decentralized S3 Object Storage Developer Documentation.
The latest gateway release can be found here: https://dl.bintray.com/storewise/s3-gateway/​

Introduction

StoreWise is a secure & distributed object storage layer for a broad range of applications and use cases.
StoreWise is designed for individuals and organizations that require scale-out data storage infrastructure with a high level of security, privacy, control, sovereignty and availability.
StoreWise's object storage service is built to be 100% bit-compatible with Amazon's Simple Storage Service (AWS S3) including Identity and Access Management (IAM) APIs. With this any existing AWS S3-compatible application or gateway device will work seamlessly with StoreWise.
Although StoreWise is S3 compatible, we also work with our technology partners to design and implement custom storage interfaces for specific applications.
The primary purpose of this guide is to provide a in-depth understanding of the StoreWise technology stack and provide a developer quick-start for using the S3 API.

πŸ‘‰Get Started

Can't wait to try it out? πŸ”₯Jump straight to our Get Started tutorial!

Throughout this guide, for the sake of brevity, the StoreWise API functionality is designated as β€œStoreWise” and the AWS S3 API functionality is designed as β€œS3.”

StoreWise - Infrastructure

Due to the distributed nature of StoreWise's architecture, the deployed infrastructure is quite different from that of AWS S3.
The storage infrastructure consists of four main components:
    Metadata server
    Client Gateway
    Host Selection Engine (HSE)
    Public blockchain
By default the metadata server and HSE is hosted and maintained by StoreWise while the gateway runs on-prem client-side. Technology partners have the ability to run the entire stack on-prem. Please contact [email protected].
πŸ“‘ Metadata Server
☁️ Gateway
πŸ”— Public Blockchain
⛏️ Host Selection Engine (HSE)
The metadata-server is responsible for forming, maintaining and renewing storage contracts on the blockchain. Multiple metadata servers are available across geographic regions.
Features:
    Handles all complex tasks associated with storage nodes and smart contracts.
    Can be called for more granular functionality.
    Access across different regions to decrease latency.
The gateway is a small client-side application that can either be integrated at the application layer with the AWS-S3 SDK, or, accessed through a self-contained local S3 Server. Features:
    Client Side encryption.
    Fully self-contained, continues to operate if metadata-server is down.
    Storage contracts are stored locally.
    Data is sharded, erasure coded and transferred directly to the storage nodes.
The public blockchain is designed for creating and storing (smart) data storage contracts with storage nodes around the globe and ensure no tampering occurs to both the storage nodes and data itself.
Features:
    Immutable.
    Highly secure & reliable.
    Creation of smart storage contracts.
    No downtime.
The host (storage node) selection engine is responsible for finding the best storage nodes for the customer's application.
Features:
    Select only the best nodes for your business application.
    Filter nodes across a variety of parameters (for example. Latency, throughput, geographic location etc.) and expose these as granular control leavers to the user.
The gateway is designed to operate at 100% availability even if the StoreWise infrastructure suffers an outage. In case of a unforeseen outages, the gateway can continue to Push/Pull data.
Infrastructure schematic.
Workflow;
    1.
    Developer set's up an account and creates a bucket with policies (e.g. Geo-fence, Storage class etc.)
    2.
    A request is sent to the Metadata server for storage contract formation.
    3.
    The Metadata Server sends a request to the host selection engine (HSE) to parse a list with selected storage nodes.
    4.
    With the whitelist, the blockchain is asked to form smart contracts with the selected storage nodes.
    5.
    Contracts are encrypted, downloaded and stored by the gateway locally.
    6.
    The gateway signs the contracts and runs a local S3 Server.
    7.
    Gateway encrypts the data and uploads it directly to the storage nodes. To ensure resilience data gets erasure coded and mirrored across different nodes (default 3x).

Operation workflow

Operation workflow during all systems OK & during outage.
    1.
    Nominal operation. Gateway synchronizes telemetry to the metadata-server akin to performance and the state of the contracts. The metadata-server ensures contracts stay healthy, performant and enough funds are present.
    2.
    During outage. Although telemetry gets sent to the metadata server, the gateway is a completely self-sustaining, self-contained system. In case the metadata server goes down, no critical failures happens and the gateway can continue transferring data. This is possible because the gateway stores all storage contracts locally in addition to a local DB storing all S3 metadata.
    3.
    Once the metadata server is back up, metadata gets synchronized again and everything is back to normal operation.
TL;DR Web-console (SETUP)
➑
Metadata server
➑
host selection engine
➑
blockchain
➑
Metadata Server
➑
gateway
➑
Store Data!

Implemented Methods List

Common Operations

Feature
Status
Remark
GET Service
βœ”οΈ
Returns a list of all of the buckets

Bucket Operations

Feature
Status
Remark
GET Bucket
βœ”οΈ
Lists objects of the bucket
GET Bucket accelerate
✘
Returns the Transfer Acceleration state of a bucket
GET Bucket ACL
βœ”οΈ
Returns the bucket ACL
GET Bucket analytics
✘
Returns an analytics configuration for the bucket
GET Bucket CORS
✘
Returns the cross-region resource sharing configuration set for a bucket
GET Bucket encryption
✘
Returns the server-side encryption configuration of a bucket
GET Bucket inventory
✘
Returns an inventory configuration from the bucket
GET Bucket lifecycle
✘
Returns the lifecycle configuration information set on a specified bucket
GET Bucket location
βœ”οΈ
Returns the region where the bucket is
GET Bucket logging
✘
Returns the logging status of a bucket and the permissions users have to view and modify that status
GET Bucket metrics
✘
Returns a metrics configuration from the bucket
GET Bucket notification
✘
Returns the notification configuration of a bucket
GET Bucket Object versions
✘
Lists the metadata about all of the versions of objects in a bucket
GET Bucket policy
✘
Returns the policy of a specified bucket
GET Bucket replication
✘
Returns the replication configuration of a bucket
GET Bucket requestPayment
✘
Returns the request payment configuration of a bucket
GET Bucket tagging
✘
Returns the tag set associated with the bucket
GET Bucket versioning
βœ”οΈ
Returns the versioning state of a bucket
GET Bucket website
✘
Returns the website configuration associated with a bucket
DELETE Bucket
βœ”οΈ
Deletes bucket
DELETE Bucket analytics
✘
Deletes an analytics configuration
DELETE Bucket CORS
✘
Deletes the CORS configuration of a bucket
DELETE Bucket encryption
✘
Deletes the server-side encryption configuration from the bucket
DELETE Bucket inventory
✘
Deletes an inventory configuration from the bucket
DELETE Bucket lifecycle
✘
Deletes the lifecycle configuration from the bucket
DELETE Bucket metrics
✘
Deletes a metrics configuration from the bucket
DELETE Bucket policy
✘
Deletes the policy from the bucket
DELETE Bucket replication
✘
Deletes the replication configuration from the bucket
DELETE Bucket tagging
✘
Deletes a tag set from the specified bucket
DELETE Bucket website
✘
Removes the website configuration for a bucket
HEAD Bucket
✘
Determines if a bucket exists and you have permission to access it
List Bucket Analytics Configurations
✘
Returns a list of analytics configurations for the bucket
List Bucket Inventory Configurations
✘
Returns a list of inventory configurations for the bucket
List Bucket Metrics Configurations
✘
Returns a list of metrics configurations for the bucket
PUT Bucket
βœ”οΈ
Creates bucket
PUT Bucket accelerate
✘
Uses the accelerate subresource to set the Transfer Acceleration state of an existing bucket
PUT Bucket ACL
βœ”οΈ
Configures the Access Control List of a bucket
PUT Bucket analytics
✘
Adds an analytics configuration to the bucket
PUT Bucket CORS
✘
Sets the cors configuration for your bucket
PUT Bucket encryption
✘
Uses the encryption subresource to set the default encryption state of an existing bucket
PUT Bucket inventory
✘
Adds an inventory configuration to the bucket
PUT Bucket lifecycle
✘
Creates a new lifecycle configuration for the bucket or replaces an existing lifecycle configurations
PUT Bucket logging
✘
Set the logging parameters for a bucket and to specify permissions for who can view and modify the logging parameters
PUT Bucket notification
✘
Enables notifications of specified events for a bucket
PUT Bucket policy
✘
Adds or replaces a policy on a bucket
PUT Bucket replication
✘
Creates a replication configuration or replaces an existing one
PUT Bucket requestPayment
✘
Sets the request payment configuration for a bucket
PUT Bucket tagging
✘
Adds a set of tags to a bucket
PUT Bucket versioning
✘
Sets the versioning state of an existing bucket
PUT Bucket website
✘
Set the website configuration for a bucket

Object Operations

Feature
Status
Remark
GET Object
βœ”οΈ
Lists objects
GET Object ACL
βœ”οΈ
Returns the access control list of an object
GET Object tagging
✘
Returns the tag-set of an object
GET Object torrent
✘
Returns torrent files from a bucket
DELETE Multiple Objects
✘
Delete multiple objects from a bucket
DELETE Object
βœ”οΈ
Removes the null version (if there is one) of an object and inserts a delete marker, which becomes the latest version of the object
DELETE Object tagging
✘
Deletes a tag set from the specified object
HEAD Object
✘
Retrieves metadata from an object without returning the object itself
OPTIONS Object
✘
Allows to send a preflight request to trigger an evaluation of the rules that are defined in the CORS configuration
POST Object
✘
Adds an object to a bucket by using HTML forms
POST Object restore
✘
Restore a temporary copy of an archived object
PUT Object
βœ”οΈ
Adds an object to a bucket
PUT Object copy
βœ”οΈ
Copies an object
PUT Object ACL
βœ”οΈ
Adds an ACL to an object
PUT Object tagging
✘
Adds one or several tags to an object
SELECT Object Content
✘
Filters the contents of an object based on a simple Structured Query Language (SQL) statement
Multipart
✘
Initiates, aborts, completes, lists, uploads, uploads copy
The following AWS documents apply to the StoreWise API design:
AWS S3
AWS IAM
AWS S3 capabilities referenced in this Guide are based on API version 2006-03-01, described in the Amazon Simple Storage Service API Reference: http://docs.aws.amazon.com/AmazonS3/latest/API/s3-api.pdf
AWS IAM capabilities referenced in this Guide are based on API version 2010-05-08, described in the AWS Identity and Access Management API Reference: http://docs.aws.amazon.com/IAM/latest/APIReference/iam-api.pdf​
Last modified 9mo ago