objst is an emmedable object storage written in golang. It can also be used to serve public http traffic. It is based upon BadgerDB, a highly performant key-value store.
A bucket is representing an object storage. It allows you to interact with the underlying object storage in an easy manner. To create a simple bucket you can use the following code snippet:
func main() {
opts := objst.NewDefaultBucketOptions()
bucket, err := objst.NewBucket(opts)
if err != nil {
panic(err)
}
// use the bucket for different operations
}An object is the main abstraction in objst to represent different payload with some metadata.
An object can be created using the NewObject function:
func main() {
obj, err := objst.NewObject("name", "owner")
if err != nil {
panic(err)
}
}NOTE: The owner of an object has to be a valid uuid-v4. The uuid can be created using the google/uuid package. It is used internally for testing and promises the most resilient results in production use. If you don't differentiate between different owners you can use
objst.SystemOwnerwhich will assign a one time calculated uuid to the object. This should be only used if the different objects are all managed by the same owner and the object storage will only be consumed by the same owner over http. Otherwise you will risk some security issues on your end (no authorization).
The object struct has implemented many useful interfaces which allow you to use it as a
usual file. For example an object can be passed to any function which accepts an io.Reader,
io.Writer, io.WriterTo or io.ReaderFrom.
func main() {
obj, err := objst.NewObject("name", "owner")
if err != nil {
panic(err)
}
buf := new(bytes.Buffer)
if _, err := buf.ReadFrom(obj); err != nil {
panic(err)
}
if _, err := buf.WriteTo(obj); err != nil {
return err
}
}The most powerful feature of objst is the use of meta data. Meta data are custom key-value
pairs which will be associated with object and used for querying purposes. Setting a key-value
on an object can be done using the obj.SetMetaKey function. Some meta data is managed directly
by objst and cannot be set by you. For example objst.MetaKeyID or objst.MetaKeyCreatedAt
can not be set using obj.SetMetaKey. The key of the meta data is of type objst.MetaKey and the value
is a string.
func main() {
obj, err := objst.NewObject("name", "owner")
if err != nil {
panic(err)
}
// set the meta data contentType
obj.SetMetaKey(objst.MetaKeyContentType, "text/plain")
// get the meta data id
id := obj.GetMetaKey(objst.MetaKeyID)
}There are some helper function implemented for the object struct e.g ID() or Owner() which will return the
meta data in a convenient way. Calling ID() is the same as obj.GetMetaKey(objst.MetaKeyID).
One of the most important meta data is the content-type of the object. This will be used as the content-type to
serve the object over http and is required for every object. objst is making best effort assumptions to get the official
mime-type of the uploaded file using the specified file extension. If the file extension cannot be found it will fallback
to the user defined multipart form key objst.MetaKeyContentType. If none is provided an error will be returned. If one is
found it will automatically be registered to the runtime and the content-type meta data will be specified. Every new object
created after the extension is registered to the runtime using objst.NewObject will automatically have the MetaKeyContentType
set so you don't have to worry about it. To make your life as easy as possible you can implement any kind of init function which will
register your unofficial mime types using mime.AddExtensionType:
func main() {
if err := run(); err != nil {
log.Fatal(err)
}
}
// run can also be named `init` so the go
// runtime will be calling it on your behalf.
// But be aware that using init is sometimes
// not considered best practice (uber-styleguide/google-styleguid)
func run() error {
// every extension has to begin with a leading `.`
unofficialTypes := map[string]string{
".test": "text/plain",
}
for ext, mimeType := range unofficialTypes {
objst.AddExtensionType(ext, mimeType)
}
}An example is provided at examples.
objst.NewQuery allows you to get multiple or one object at once in a convenient way. For example
you can get all the objects which have the meta data foo=bar in the following way:
func main() {
opts := objst.NewDefaultBucketOptions()
bucket, err := objst.NewBucket(opts)
if err != nil {
panic(err)
}
// Create a query with the parameter foo=bar and the `Get` operation
// which is the default.
q := objst.NewQuery().Param("foo", "bar")
objs, err := bucket.Execute(q)
if err != nil {
panic(err)
}
}The query is smart engough to figure out if only one record will be fetched or multiple. This allows you to use queries to fetch one record in an efficient manner:
func main() {
opts := objst.NewDefaultBucketOptions()
bucket, err := objst.NewBucket(opts)
if err != nil {
panic(err)
}
// Get one object for the owner `owner` and name `name`.
q := objst.NewQuery().Owner("owner").Name("name")
objs, err := bucket.Execute(q)
if err != nil {
panic(err)
}
// or fetch by id
q = objst.NewQuery().ID("id")
objs, err := bucket.Execute(q)
if err != nil {
panic(err)
}
}objst delivers a default HTTPHandler to serve objects over http.
func main() {
opts := objst.NewDefaultBucketOptions()
bucket, err := objst.NewBucket(opts)
if err != nil {
panic(err)
}
// the handler options allow you to set different
// parameters to modify the behavior of the handler.
// For the different options available see:
// https://pkg.go.dev/github.com/naivary/objst#HTTPHandlerOptions
handlerOpts := objst.DefaultHTTPHandlerOptions()
hl := objst.NewHTTPHandler(handlerOpts)
}All endpoints require authentication some require authorization. You can specify
how authorization or authentication is implemented by setting the IsAuthorized
and IsAuthenticated middleware in the handler's options. By default IsAuthenticated
and IsAuthorized will allow all incoming request.
The endpoints are as follow:
GET /objst/{id}: Get the object as a model without the payload. The model includes the name, owner, id and the user defined meta data.GET /objst/read/{id}: Read the payload of the objectDELETE /objst/{id}: Delete the objectPOST /objst/upload: Upload a file to the object storage. The file will be retrived using opts.FormKey. The Content-Type of the object can be specified using thecontentTypekey in the multipart form.
The first three endpoints require authentication and authorization the last one only requires authentication and the objst.CtxKeyOwner set in the request context.
Some examples are being provided in the examples directory. Use these as a starting point and tailor the behavior to your liking and needed requirements.