Go library providing a common interface for working across multiple storage backends.
Supported storage backends:
- Alibaba Cloud OSS Storage (alibaba.go)
- Amazon S3 (amazon.go)
- Baidu Cloud BOS Storage (baidu.go)
- DigitalOcean Spaces (amazon.go, using custom endpoint and us-east-1)
- etcd (etcd.go)
- Google Cloud Storage (google.go)
- Local filesystem (local.go)
- Microsoft Azure Blob Storage (microsoft.go)
- Minio (amazon.go, using custom endpoint and us-east-1)
- Netease Cloud NOS Storage (netease.go)
- Openstack Object Storage (openstack.go)
- Oracle Cloud Infrastructure Object Storage (oracle.go)
- Tencent Cloud Object Storage (tencent.go)
This code was originally part of the Helm project: ChartMuseum, but has since been released as a standalone package for others to use in their own projects.
Backend is a common interface that is implemented by all the supported storage backends and their associated types:
type Backend interface { ListObjects(prefix string) ([]Object, error) GetObject(path string) (Object, error) PutObject(path string, content []byte) error DeleteObject(path string) error }
Object is a struct that represents a single storage object:
type Object struct { Path string Content []byte LastModified time.Time }
ObjectSliceDiff is a struct that represents overall changes between two Object slices:
type ObjectSliceDiff struct { Change bool Removed []Object Added []Object Updated []Object }
GetObjectSliceDiff is a function that takes two Object slices, compares them, and returns an ObjectSliceDiff:
func GetObjectSliceDiff(prev []Object, curr []Object, timestampTolerance time.Duration) ObjectSliceDiff
The following is a simple program that will upload a file either to an Azure Blob Storage bucket (container) or a Google Cloud Storage bucket based on the command line options provided:
// Usage: go run example.go <cloud> <bucket> <file> package main import ( "fmt" "io/ioutil" "os" "path/filepath" "github.com/chartmuseum/storage" ) type ( Uploader struct { Backend storage.Backend } ) func NewUploader(cloud string, bucket string) *Uploader { var backend storage.Backend switch cloud { case "azure": backend = storage.NewMicrosoftBlobBackend(bucket, "") case "google": backend = storage.NewGoogleCSBackend(bucket, "") default: panic("cloud provider " + cloud + " not supported") } uploader := Uploader{Backend: backend} fmt.Printf("uploader created (cloud: %s, bucket: %s)\n", cloud, bucket) return &uploader } func (uploader *Uploader) Upload(filename string) { basename := filepath.Base(filename) content, err := ioutil.ReadFile(filename) if err != nil { panic(err) } err = uploader.Backend.PutObject(basename, content) if err != nil { panic(err) } fmt.Printf("%s successfully uploaded\n", basename) } func main() { args := os.Args[1:] uploader := NewUploader(args[0], args[1]) uploader.Upload(args[2]) }
Example of using to upload the file index.html to an Azure bucket:
go run example.go azure mycontainer index.html
Example of using to upload the file index.html to a Google Cloud bucket:
go run example.go google mybucket index.html
Each supported storage backend has its own type that implements the Backend interface.
All available types are described in detail on GoDoc.
In addition, authentication methods are based on the runtime environment and vary from cloud to cloud.