Skip to end of metadata
Go to start of metadata

At the 6/9 Tech Stack WG Meeting (Joint with ESIP Interoperability and Technology / Tech), Axiom Data science developers Kyle Wilcox, Shane StClair, and Dave Foster gave a brief overview and then dove into Docker demos. The demos were “jaw dropping” (actual quote, I do not make this stuff up.) Also, there were photos of James Van Der Beek in the slide defining Docker:

The slides and recording for the last IT&I "Dive into Docker" webinar are now linked on the ESIP IT&I Page. (YouTube link)

Highlights from a novice's perspective:

What is Docker?

Docker containers wrap a piece of software in a complete filesystem that contains everything needed to run: code, runtime, system tools, system libraries – anything that can be installed on a server. This guarantees that the software will always run the same, regardless of its environment. -

Difference between Docker and Virtual Machines

  • Virtual machines have their own operating system with memory and CPU management - that takes measurable overhead = a performance hit. Takes 10s of minutes to set up.
  • Docker containers are executed in a Docker engine - no emulator resources taken, better performance, better compatibility with host resource. Set up fairly instantly.


Demos showed running figlet and spinning up postgres from a docker container, and Docker Swarms (orchestration tool for many containers).

Required screenshot of a terminal demo.

Docker Swarm is native clustering for Docker. It turns a pool of Docker hosts into a single, virtual Docker host. Because Docker Swarm serves the standard Docker API, any tool that already communicates with a Docker daemon can use Swarm to transparently scale to multiple hosts. -

Orchestration/Ecosystem tools are exploding and there's a lot of competition. Here's a post to help you navigate that world: "The container ecosystem map from an engineer's perspective"  ( post)

Some additional resources

Next month: NOAA OneStop

The next Tech Dive will be July 13 at 3pm ET, "The NOAA OneStop Data Discovery and Access Framework Project": Ken Casey, NOAA/NCEI.

Summary: The OneStop Project is designed to improve NOAA's data discovery and access framework. Focusing on all layers of the framework and not just the user interface, OneStop is addressing data format and metadata best practices, ensuring more data are available through modern web services, working to improve the relevance of dataset searches, and improving both collection-level metadata management and granule level metadata systems to accommodate the wide variety and vast scale of NOAA's data.

More CDI Blog posts