Confluence Retirement

Due to the feedback from stakeholders and our commitment to not adversely impact USGS science activities that Confluence supports, we are extending the migration deadline to January 2023.

In an effort to consolidate USGS hosted Wikis, myUSGS’ Confluence service is targeted for retirement. The official USGS Wiki and collaboration space is now SharePoint. Please migrate existing spaces and content to the SharePoint platform and remove it from Confluence at your earliest convenience. If you need any additional information or have any concerns about this change, please contact myusgs@usgs.gov. Thank you for your prompt attention to this matter.
Skip to end of metadata
Go to start of metadata


September 9, 2020: Structured data on the web for earth science and CDI Pop-Up Lab

The Community for Data Integration (CDI) meetings are held the 2nd Wednesday of each month from 11:00 a.m. to 12:30 p.m. Eastern Time.

Connection information

The Microsoft Teams call information is emailed to the CDI mailing list. If you would like to become a member of CDI, join at https://listserv.usgs.gov/mailman/listinfo/cdi-all.


Meeting Recording and Slides

These are the publicly available materials.

Log in to view all the meeting resources. If you would like to become a member of CDI, join at https://listserv.usgs.gov/mailman/listinfo/cdi-all.

Agenda (in Eastern time)

11:00 am Welcome and Opening Announcements, CDI Request for Proposals - Leslie Hsu and Tim Quinn

11:10 am Collaboration Area Announcements

11:20 am Structured data on the web for earth science: emerging best practices and remaining challenges - Dave Blodgett, USGS

11:50 am CDI Pop-Up Lab

Meeting the CDI Community - Mining Expertise Keywords - Sky Bristol, USGS

Highlights from the South East Region Science Meeting  - Greg Steyer, USGS

Machine learning, satellite imagery, and tile drains - Tanja Williamson, USGS

12:30 pm  Adjourn

Abstracts

Structured data on the web for earth science: emerging best practices and remaining challenges

TL;DR; This month, Dave Blodgett will provide an update on progress from two OGC Interoperability Experiments, topics of interest from the CDI/ESIP IT&I webinar series, and the Water Mission Area National Hydrologic Geospatial Fabric project. 

The landscape of spatiotemporal information we observe and predict exists as a spectrum between continuous fields to discrete objects. With this as context, the evolving technology upon which we base our information systems presents tremendous opportunities to understand and provide insights about our data. Simultaneously, the evolving technical basis presents challenges to the long term stability of our IT systems and the reproducibility of our science. This presentation will explore recent developments in concepts, architectures, and technologies that help characterize the real world, define a stable and cohesive architecture, and seek to increase the impact of earth science data in society. This presentation will present some specific activities on these theme with a focus on hydrologic science and concepts as an integrator of environmental systems and data with an eye toward EarthMap and an even broader Integrated Hydro-terrestrial Modeling community. 

Dave Blodgett is an EDGE Civil Engineer in the Water Resources Mission Area Geospatial Intelligence Branch. He takes part in numerous community groups including the OGC Architecture Board, the ESIP Program Committee, and the Unidata Strategic Advisory Committee. His active research and development includes interoperable modular workflow tools development for hydrology, application of consistent conceptual models to diverse technical challenges, and advancing adoption of structured data for environmental feature and observations data.

Highlights

  1. CDI's Request for Proposals is now open.

Notes

  1. Announcements
    1. CDI's Request for Proposals is now open. Visit the FY21 RFP page for more.
      1. Projects build USGS capabilities in data integration and management. CDI projects are short term projects that leverage existing resources, demonstrate scalable solutions, improve access to data and tools, develop best practices, share lessons, and more.
      2. Encourages interdisciplinary collaboration and wide communication.
      3. Fire science and coastal resilience are the themes for the FY21 RFPs. Proposals from all areas are accepted.
  2. Collaboration Areas
    1. Data Management working group
      1. Next event: Monday, September 14
    2. Tech Stack
      1. Next event: Thursday, September 10
        1. https://www.gotomeeting.com/join/533510693
    3. Software Development Cluster
      1. Next event: Thursday, September 24
    4. Usability
      1. Next event: Wednesday, September 16 (resource review)
    5. Risk
      1. Next event: Thursday, September 17
      2. October 15 - FY21 RFP Information Session
    6. Open Innovation
      1. Next event: September webinar TBD
    7. AI/ML
      1. Next event: Tuesday, October 13
  3. Structured data on the web for earth science: emerging best practices and remaining challenges - Dave Blodgett, USGS
    1. Observation and feature
      1. Observation - act of measuring or otherwise determining the value of a property
      2. Feature - an abstraction of a real-world phenomenon
    2. Second Environment Linked Features Interoperability Experiment
      1. Use case: as a web users, I want to find all the information available for an environmental feature, so I can find what I'm looking for and retrieve it
      2. Four functions to satisfy linked environmental feature data
        1. Identification
        2. Landing page/content
        3. Structured data
        4. Data
      3. Resource model vs content model
        1. Resource is identified by URL
        2. Content model is more like a data model or information schema. Information about the kind of information you have in a document.
        3. See graph for more.
      4. Tech Stack themes
        1. Science Data gateways
          1. Pangeo, HydroShare, and more
          2. Formalization of bringing processing and content management together. Provides pathway for scientists to work in these environments as a community.
        2. Linked data and knowledge networks
          1. Messy data found from a myriad of sources, relies on data authors/content creators for curation
          2. Knowledge networks - more centralized, can query them rapidly, curated or less messy
        3. Discrete global grids & dataset indexing
          1. ex: integrating datasets, allowing better probability calculations, etc
          2. Google dataset search
      5. National Hydrologic Geospatial Fabric Project
        1. Elements include hydroinformatics, reference fabric, enterprise systems, tools development, river corridors, landscape characteristics, and hydrogeology
        2. Research to operations and data to knowledge
      6. Future work
        1. Linked data, knowledge networks, and the google index "space"
          1. Leveraging the google index using linked data and knowledge networks
          2. Persistent identification of places with multiple representations
          3. Negotiation of multiple content profiles from the same resource
          4. A minimal set of representative domain ontologies for knowledge management
        2. Please review EDR!
  4. CDI Pop-Up Lab
    1. Meeting the CDI Community - Mining Expertise Keywords - Sky Bristol, USGS
      1. Created a python package pylincmd to make corporate master data to be linked

      2. Wrote some code to scrape staff profile inventories into a data structure to look at individual pages

      3. Wrote as web scraper

      4. Work examining what is out there

      5. Also looking at additional vocabulary sources
      6. Notebook records comments and future directions
      7. https://github.com/skybristol/pylinkedcmd

    2. Highlights from the South East Region Science Meeting  - Greg Steyer, USGS
      1. Questionnaire on capacities that exist within USGS
      2. Lightning talks pointed to various examples of how we can integrate across system components for large datasets.
        1. Ex of technology examined: cloud-based frameworks for camera-based streamflow measurements
      3. Additional slides available on this page soon.
      4. SER Science Workshop: https://doimspp.sharepoint.com/sites/gs-ser-fy20scienceworkshop
    3. Machine learning, satellite imagery, and tile drains - Tanja Williamson, USGS
      1. Tile drains are present in every state in the U.S. - moves water out of fields so that people can farm larger areas. This water then moves into the stream network. Some of these tile drains carry sediments and pesticides into the stream system.
      2. Data gap inferred in flooding and water quality models. 
      3. Need is to identify and catalog tile drains visible to satellite images. Can do this now, but the goal is to scale up.
      4. Focusing on linear features. Drilling down to be more specific. Can see the sub-service features in the satellite images.
      5. Still getting a little bit of background noise (farming structures usually). Want to focus on the tile drains instead of the background

Questions & Answers

  1. Dave Blodgett
    1. What is a good next step for the CDI community to look at future work problems or learn about these things in more detail?
      1. A group engaging as a group in review and evaluation activities. To rally around these concepts and represent them with more force than we could individually.
    2. for Dave - one thing we've discussed is data being findable.  is there an effort to go thru sciencebase to address this?

      1. I've not engaged with the ScienceBase team on the structured data front specifically. Just glancing at the Google "rich results" for a sciencebase item -- they are pretty good.

  2. Sky Bristol
    1. how did you search expertise terms? In the entire page? or a specific part of page?

      1. under the section specifically for keywords, on the left of the page

      2. I'd recommend joining the Configuration Management Committee for activities related to our business information systems.

  3. Greg Steyer
    1. What is the follow-up for this workshop?
      1. Outcomes from presentations and discussions are synthesized on the page, post-workshop summary folder. Trying to gather the needs from the community for applying solutions.