CDI Monthly Meeting - 20171108

The Community for Data Integration (CDI) meetings are held the 2nd Wednesday of each month from 11:00 a.m. to 12:30 p.m. Eastern Time.

WebEx:
https://usgs.webex.com/ -
Under the Meeting Center tabs, search for meeting name: "Community for Data Integration." or 
https://usgs.webex.com/usgs/j.php?MTID=m3608e9d64fa49e9ec1e7b64e803b3a4e

Audio:
USGS/DOI Dial In Number: (703) 648-4848 (for USGS and DOI offices)
Toll Free Dial In Number: (855) 547-8255 (for other offices and telecommute locations)
Conference Code: 47919# (same for both numbers)

Webex Recording

Webex recordings are available to CDI Members approximately 24 hours after the completion of the meeting. Please login to view the recording. If you would like to become a member of CDI, please email cdi@usgs.gov.


Webex Recording

Download the Webex recording [.mp4]


Ask and upvote questions at slido.com #A383. 

Agenda (in Eastern time)

11:00a Leveraging Crowdsourcing in FEMA-led Response Efforts - Sophia B Liu, USGS, Mission Assignment to FEMA [PDF]

11:10a Welcome [PDF] 

11:15a Working Group Announcements [PDF]

11:25a Reproducible Notebook Series: Notebooks as a Data Management Superpower - Colin Talbert, USGS

Batch metadata Propagation and Data Upload

Notebook for querying and interacting with IPDS Content (must log in with USGS credentials)

11:40a National Earthquake Information Center: Overview real-time data acquisition, processing, and archive - Michelle Guy, USGS

Abstract: We will provide an overview of real-time data acquisition and distribution for continuous time series and derived event-centric solutions.  And along the way highlight the software tools that are support this effort for earthquakes and how they are also being used in other hazards disciplines.

Bio: Michelle Guy - Lead computer scientist for software systems architectures for the USGS Geological Hazards Science Center.

12:00p National Earthquake Information Center: Data-first concept for presentation and delivery - Lynda Lastowka, USGS

Abstract: We use a data first concept which supports us in delivering data and building interfaces. By starting with the data we invest up front in building usable data structures.   The data itself can be considered a "Minimum Viable Product" so that the earliest adopters have access to the data soonest. We then have the options to build interfaces on top of the data if needed.  The public also has access to the same data as us which is a win for transparency and allows others to build custom interfaces if desired. 

Bio: Lynda Lastowka - USGS Geological Hazards Science Center Software Development and Web (HazDev) Project Chief

12:30p  Adjourn

Presentations

Liu: Leveraging Crowdsourcing in FEMA-led Response Efforts (pdf)

Guy: Real-Time Data Acquisition, Distribution & Archive (pdf)

Lastowka: Geologic Hazards Science Center Data Delivery and Presentation (pdf)


Presentations: Slides are available to CDI Members. Please login to download the slides. If you would like to become a member of CDI, please email cdi@usgs.gov.


Presentation Q/A

Q: Is any of the earthquake data considered "released"?
A: Keith Kirk: These data qualify under FSP (Fundamental Science Practices) as web data service. Designed for ‘data in motion’. Individual approved data releases are not required. 502.8 under web data services.


Q: Do you have any advice for other groups that are trying to improve data flow for rapid response?
A: Michelle Guy: Something that helps us to be rapid is to decouple process and presentation. Everything goes back to the common Derived Product Data Distribution Layer, other processes don't have to be able to communicate with each other, there is only one communication mechanism.


Q: What is CI/CD?
A: Lynda Lastowka: Continuous Integration / Continuous Deployment. Integration part is testing. Automated suite of tests, deployment, if test passes it is automatically pushed to production environment. We are using Jenkins to help support these pipelines. Good way to get products out there faster. More reliable than relying on human QC.

Attendees


WebEx Participant Report [.txt]


A WebEx Participant Report is available to CDI Members. Please login to download the report. If you would like to become a member of CDI, please email cdi@usgs.gov.




DateNovember 8, 2017
Presentation Title(s)

Reproducible Notebook Series: Notebooks as a Data Management Superpower

National Earthquake Information Center

Speaker(s)
Colin Talbert, USGS
Michelle Guy and Lynda Lastowka, USGS