xtreemfs

Status Info

Master (stable) branch

Development branch

Overview - What is the XtreemFS module?
Module Description - What does the module do?
Setup - The basics of getting started with XtreemFS module
- Configuring the installation
Usage - How to use the module for various tasks
Tests - how to perform unit and acceptance tests
Contributing - how to send your work?

Overview

The XtreemFS module allows you to easily manage XtreemFS installation, volumes and mounts with Puppet.

Module Description

XtreemFS is a fault-tolerant distributed file system for all storage needs. The xtreemfs module allows you to manage XtreemFS packages and services on several operating systems, while also supporting basic management of XtreemFS volumes and mounts. The module offers support for basic management of replication settings.

Setup

What wavesoftware/xtreemfs affects:

package/service/configuration files for XtreemFS
listened-to ports

###Configuring the installation

The main configuration you'll need to do will be around the xtreemfs::role::directory, xtreemfs::role::metadata and xtreemfs::role::storage classes. The default parameters are reasonable.

####To manage a XtreemFS with sane defaults on one server:

include xtreemfs::role::directory
include xtreemfs::role::metadata
include xtreemfs::role::storage

####Hiera configuration

xtreemfs::settings::dir_host
- Provide an host to where metadata and storage nodes will be connecting, defaults: $::fqdn
xtreemfs::settings::dir_port
- A port for directory service connection
xtreemfs::settings::dir_protocol
- A protocol for directory service connection
xtreemfs::settings::object_dir
- A direcory where storage nodes will hold their replicated data. Good idea is to provide a directory on secure RAID drive, defaults: /var/lib/xtreemfs
xtreemfs::settings::install_packages
- If set to true will install packages of XtreemFS, defaults: truedi
xtreemfs::settings::add_repo
- If set to true will add to system repository for XtreemFS, defaults: true
xtreemfs::settings::properties
- A properties hash to provide configuration options in form exactly like: http://www.xtreemfs.org/xtfs-guide-1.5/index.html#tth_sEc3.2.6

####For distributed without hiera

Directory service

# In this example fqdn is dir.vagrant.dev
include xtreemfs::role::directory

Metadata server

class { 'xtreemfs::role::metadata':
  dir_host => 'dir.vagrant.dev',
}

Storage node(s)

class { 'xtreemfs::role::storage':
  dir_host   => 'dir.vagrant.dev',
  object_dir => '/mnt/sdb1/xtreem', # actual object will be in: /mnt/sdb1/xtreem/objs 
}

Client(s)

Described in usage section.

Usage

###Creating a volume

There are many ways to set up a XtreemFS volume using the xtreemfs::volume definition. For instance, to set up simple volume on default parameters:

xtreemfs::volume { 'myVolume':
  ensure   => 'present',
  dir_host => 'dir.vagrant.dev',
}

In this example, you would create volume with default parameters.

###Managing mounts

To manage mount point:

xtreemfs::mount { '/mnt/xtreemfs':
  ensure   => 'mounted',
  volume   => 'myVolume',
  dir_host => 'dir.vagrant.dev',
}

In this example, you would mount an existing volume into /mnt/xtreemfs directory that already exists.

###Replication

To replicate an existing file for ex.: /mnt/xtreemfs/centos7.iso inside the XtreemFS mount point, use:

xtreemfs::replicate { '/mnt/xtreemfs/centos7.iso':
  policy  => 'WqRq',
  factor  => 2,      # Required storage nodes
}

###Automatic replication

To automatically replicate new files in XtreemFS mount point you can set a policy for directory. This will not affect existing files. Take a look at example usage:

xtreemfs::policy { '/mnt/xtreemfs':
  policy          => 'WqRq',  # Replicate policy
  factor          => 2,       # Required storage nodes
  striping_policy => 'RAID0', # Actually only one supported by XtreemFS
  stripe_count    => 1,       # Number of storage servers used for striping
  stripe_size     => 128,     # The size of an individual stripe in KiB
}

Tests

There are two types of tests distributed with the module. Unit tests with rspec-puppet and system tests using rspec-system.

For unit testing, make sure you have:

rake
bundler

Install the necessary gems (gems will be downloaded to private .vendor directory):

bundle install --path .vendor

And then run the unit tests:

bundle exec rake spec

The unit tests are ran in Travis-CI as well, if you want to see the results of your own tests register the service hook through Travis-CI via the accounts section for your Github clone of this project.

If you want to run the system acceptance tests, make sure you also have:

vagrant > 1.2.x
Virtualbox > 4.2.10

Then run the tests using:

bundle exec rake acceptance

To run the tests on different operating system, see the sets available in spec/acceptance/nodesets/ and run the specific set with the following syntax:

bundle exec rake acceptance RS_SET=debian-76-x64

You can also run system acceptance tests against Docker containers. If you want to do this, make sure you also have:

docker > 1.0.0

To run the the test on docker container use:

bundle exec rake acceptance RS_SET=centos-65-x64-docker

###Contributing

Contributions are welcome!

To contribute, follow the standard git flow of:

Fork it
Create your feature branch (git checkout -b feature/my-new-feature)
Commit your changes (git commit -am 'Add some feature')
Push to the branch (git push origin feature/my-new-feature)
Create new Pull Request

Even if you can't contribute code, if you have an idea for an improvement please open an issue.

wavesoftware/puppet-xtreemfs