/Journey_to_the_Compute

Two day hands-on workshop for Docker, Singularity, and leveraging HPC and cloud resources.

MIT LicenseMIT

Journey to the Compute

Two day hands-on workshop for Docker, Singularity, and leveraging HPC and cloud resources.

Schedule

Journey to the Compute, Episode 1: Containers to Quiet the Dependency Hells

Docker

  1. Pre-built containers
  1. Building your own containers
  2. Limitations
  • Docker for Windows but not Singularity
  • Virtual environment

Bingo list of resources for industry and academic folks Build a container of visualization tools Plotly Bokeh Pandas

Journey to the Compute, Episode 2: XSEDE Bound and Down, Dockered Up and Truckin'

HPC National (XSEDE) OSG Comet (GPU!) Giant List () UA HPC (academics, potential for industry to buy time)

Cloud services Atmo/Jetstream AWS/Google Docker Swarm??? Pegasus?

References

Kurtzer GM, Sochat V, Bauer MW (2017) Singularity: Scientific containers for mobility of compute. PLOS ONE 12(5): e0177459. https://doi.org/10.1371/journal.pone.0177459 Upendra Kumar Devisetty, Kathleen Kennedy, Paul Sarando, Nirav Merchant, Eric Lyons. Bringing your tools to CyVerse Discovery Environment using Docker. DOI: http://dx.doi.org/10.12688/f1000research.8935.3

Acknowledgements

This work was adapted from previous work done by Upendra Devisetty (@upendrak), and John Fonner (@johnfonner).

Useful Links

Workshop Events

  • Meetup Day 1 June 2017 In case you're not familiar, shipping containers are responsible for moving most of the goods that you consume in your everyday. Without them, we'd be in trouble. It's also interesting to note they they transit from giant ships, to railways, to semi-trucks, and fit into the back loading dock of your local grocery because they are a uniform size. This same concept has been applied to computation. We'll start out by discussing the container technologies Docker and Singularity. We'll go over some use cases and highlight what containers won't do. Docker is NOT perfect. Then we'll break into hands-on setting up containers, finding pre-built containers, and running containers.

    Bring a laptop! Or if you're really really strong you can bring a desktop. Ultimately, we'll move a Docker container into a Singularity container to set up for the July meet up.

  • Meetup Day 2 July 2017 We're gonna do what they say can't be done! 'Scaling up' compute is a tricky thing. Not only are there hardware issues to consider, but there are many secret sauce code streamlining fixes to deploy it on bigger compute. We're going to focus on how to move out to bigger compute, because that's generally the first step. Come join us!