DOSS 2

DOSS 2 will be an object storage abstraction layer with transactional write-once semantics. Objects are uniquely and persistently identified and can never be overwritten only created and (soft) deleted.

(Huboard: Tasks, Backlog) (Javadoc) (Test coverage)

Status

Java API: stable
Local backend: complete
Network backend: inprogress
CLI frontend: incomplete
FUSE frontend: complete
Container support: complete

Feature Goals

write once blob storage semantics
concurrent distributed random read access
multiple storage pools and devices (with different policies)
two-phase commit write transactions
soft deletes ("rubbish bin")
consistency and integrity checks
packs small files into containers for efficient bulk processing

Java API

See java-api.md for more examples.

Ingesting files

try (BlobStore bs = LocalBlobStore.open("/doss-devel");
     BlobTx tx = bs.begin()) {    
    Blob blob1 = tx.put(Paths.get("/tmp/myimage.jpg"));
    Blob blob2 = tx.put(Paths.get("/tmp/mytext.txt"));
    blob2.verifyDigest("SHA1", "2aae6c35c94fcfb415dbe95f408b9ce91ee846ed");
    tx.commit();
    return blob1.id();
}

Random access

try (BlobStore bs = LocalBlobStore.open("/doss-devel")) {
    Blob blob = bs.get(273);
    try (Channel channel = blob.openChannel()) {
        // do something with the channel
    }
}

Non-Java read access

See doc/native-interface.md.

dossfs /doss-devel /mnt
grep -i endeavour /mnt/962b6910
convert /mnt/014908b0 /tmp/kangaroo.jpg

Building

Requirements:

Java 7
Maven 3

Compile by typing:

mvn package

License

DOSS 2 is free software; you can redistribute it and/or modify it under the terms of the Apache License, Version 2.0:

http://www.apache.org/licenses/LICENSE-2.0

See the file LICENSE for details.

azeemaf/doss