Confluence Retirement

In an effort to consolidate USGS hosted Wikis, myUSGS’ Confluence service is scheduled for retirement on January 27th, 2023. The official USGS Wiki and collaboration space is now SharePoint. Please migrate existing spaces and content to the SharePoint platform and remove it from Confluence at your earliest convenience. If you need any additional information or have any concerns about this change, please contact myusgs@usgs.gov. Thank you for your prompt attention to this matter.
Skip to end of metadata
Go to start of metadata

April 14, 2021: Linking water use, runoff, and recharge models, and a "fire-aware" stream gage network

The Community for Data Integration (CDI) meetings are held the 2nd Wednesday of each month from 11:00 a.m. to 12:30 p.m. Eastern Time.



Meeting Recording and Slides

Recordings and slides are available to CDI Members approximately 24 hours after the completion of the meeting.

Log in to view the meeting resources. If you would like to become a member of CDI, join at https://listserv.usgs.gov/mailman/listinfo/cdi-all.

 

During the call, you can ask and up-vote questions at slido.com, event code #CDIAPR.

Agenda (in Eastern time)

11:00 am Welcome and Opening Announcements

11:15 am Collaboration Area Announcements

11:25 am Moving towards EarthMAP: Establishing linkages among USGS land use, water use, runoff, and recharge models - Terry Sohl, USGS

11:45 am Developing a "fire-aware" stream gage network by integrating USGS enterprise databases - Kitty Kolb, USGS

12:05 pm Ask an Expert/Pop-up Lab - Questions and Answers from the CDI Community

12:30 pm  Adjourn

Abstracts

The presentations this month are all from FY2020 CDI-supported projects. 

Moving towards EarthMAP: Establishing linkages among USGS land use, water use, runoff, and recharge models - Terry Sohl, USGS

Understanding and anticipating change in dynamic Earth systems is vital for societal adaptation and welfare. USGS possesses the multidisciplinary capabilities to anticipate Earth systems change, yet our work is often bound within a single discipline and/or Mission Area. The proposed work breaks new ground in moving USGS towards an interdisciplinary predictive modeling framework. We are initially leveraging three research elements that cross the Land Resources and Water Mission Areas in an attempt to “close the loop” in modeling interactions among water, land use, and climate. Using the Delaware River Basin as a proof-of-concept, we are modeling 1) historical and future landscapes (~1850 to 2100), 2) evapotranspiration and water use by vegetation, and 3) groundwater and surface water flows and interactions. We will develop a conceptual framework for characterizing feedbacks among these processes and define a roadmap for a potential scaling up to a national/enterprise-scale capability.


Developing a "fire-aware" stream gage network by integrating USGS enterprise databases - Kitty Kolb, USGS

Wildfires affect streams and rivers when they burn vegetation and scorch the ground. This makes floods more likely to happen and reduces water quality. Public managers, first responders, fire scientists, and hydrologists need timely information before and after a fire to plan for floods and water treatment. This project will create a method to combine national fire databases with the StreamStats water web mapping application to help stakeholders make informed decisions. When the project is finished, people will be able to use StreamStats to estimate post-wildfire peak flows in streams and rivers for most of the United States (where data is available). There will also be tools that allow users to trace upstream and downstream on a stream or river to find nearby burned areas and stream gages.

Highlights

  1. April 30th is the abstract deadline for the 2021 CDI Workshop: Designing Data-Intensive Science, held virtually from May 25-28.
  2. Congratulations to the 2021 CDI Project Teams!

Notes

Welcome and Opening Announcements

  1. Check out www.twitter.com/USGS_DataSci for the #30DayChartChallenge 
  2. More from the Water Data Visualizations Team at https://www.usgs.gov/mission-areas/water-resources/science/water-data-visualizations
  3. April 30th is the abstract deadline for the 2021 CDI Workshop: Designing Data-Intensive Science, held virtually from May 25-28.
  4. Congratulations to the 2021 CDI Project Teams!

Collaboration Area Announcements

  1. See more on all collaboration areas here.
  2. Geomorphology
    1. Next event: April 27
  3. Software Development
    1. Next event: April 22, Working remotely: collaboration tools, fostering informal discussions, and more
  4. DevOps
    1. Next event: June 1, GitLab Pipelines to Deploy on OpenShift Kubernetes
  5. Tech Stack
    1. Next event: May 13, Landsat Cloud-Based Imagery Application (tentative)
  6. Risk
    1. Next event: April 15, Earthquake tsunami multi-hazards: engineering challenges
  7. eDNA
    1. New publication from eDNA members
    2. Still designing a logo for eDNA newsletters
  8. Data Management
    1. Next event: No May DMWG monthly meeting. Instead, attend CDI breakout sessions:
      1. Tuesday, May 25: How to talk to your data manager/scientists: breaking the ice
      2. Friday, May 28: Assessing the value and usage of USGS Data Management Plans
  9. Usability
    1. Next event: April 21, user research demo: contextual inquiry
  10. Semantic Web
    1. Next event: May 13, final arrangements for a Semantic Web 101 session at the CDI workshop; discussion of sources of semantic terms for use in USGS webpages
  11. Metadata Reviewers
    1. Next event: May 3

Moving towards EarthMAP: Establishing linkages among USGS land use, water use, runoff, and recharge models - Terry Sohl, USGS

  1. Landscape modeling - elements of a USGS national-scale modeling framework
  2. Delaware River Basin application
    1. adding more puzzle pieces to integrated modeling, with linkages among land use, ET and water use, and water availability
    2. new urban modeling framework to represent urban class densities
    3. combination of evapotranspiration models with landscape change models
    4. long-term historical landscape reconstruction to facilitate assessments
  3. Can look at actual and theoretical evapotranspiration values to see which areas are getting more water/irrigation.
  4. What CDI could take away/use
    1. Interdisciplinary modeling of Earth system function is complex! Need to learn the language, terms, and communicate what we need
    2. Technical challenges need to be thought through in concert with the scientific challenges

Developing a "fire-aware" stream gage network by integrating USGS enterprise databases - Kitty Kolb, USGS

  1. Wildfires are a national problem, not only in the west. Example: the Great Smoky Mountains fire in 2016.
  2. Problem
    1. Timely information about wildfires and stream flow
    2. How to monitor before, during, and after events
    3. The USGS has two tools to handle these problems. We will focus on the USGS Stream Stats program
  3. StreamStats
    1. StreamStats program reduces a process that used to take several days with a planimeter down to minutes.
    2. StreamStats Fire-Hydro collates:
      1. watersheds
      2. basin characteristcs
      3. NIFC & MTBS
      4. Peak Flow equations
    3. Prototype: Upper Colorado Guinnison Basin
  4. Fire Hydro application is built in StreamStats, which is built on Leaflet
    1. https://test.streamstats.usgs.gov/fire-hydro-demo/ (Note: sometimes turned off after business hours)
  5. See recording for demo.
  6. Challenges
    1. Finding programmers
    2. Keeping the application light and open source - no loading rage!
    3. NHDPlus Medium Res
      1. There are bugs in this data sometimes, requiring manual fixes
    4. MTBS queryable layers
  7. Use to the community
    1. GIS layers on ScienceBase
    2. Code will be on code.usgs.gov
    3. Integrates with other national StreamStats projects

Ask an Expert/Pop-up Lab - Questions and Answers from the CDI Community

  1. Data Help Desk
    1. Virtual Data Help Desk EGU 2021: 19-23 April 2021
      1. Megan Carter from ESIP: the help desk will provide resources for making software more open and FAIR; engage with data and software experts
      2. The data help desk is for earth science specifically, partnering with AGU, EarthCube, and others. We cast a wide net as far as volunteer experts.
      3. Questions have included: what to do with large volumes of data, like model data (where to put it, what to save)
      4. The data help desk is very collaborative; like that we get to speak to early career researchers
      5. Ask questions through Twitter via #DataHelpDesk
  2. Ask an Expert
    1. Could be more relevant for CDI/USGS specific questions
    2. Goal: use the CDI community to lessen "feeling lost"
    3. When to ask: now until the workshop
    4. How to ask: Today, on sli.do
      1. Questions!
        1. My team hasn't started using version control for their code yet. What options are available at the USGS and how do I get started? Is there training available?

    5. How to see answers: join workshop
  3. Sli.do poll on online professional profiles
  4. Intro Python resources
    1. Teams thread: Intro Python Resources

Questions

  1. Can you explain when you say the model is "heavily parameterized," what is the alternative? Could you explain a little more?
    1. Terry Sohl: We can look at the past to see characteristics of change, can add this as a parameter to the model; information on forest cutting can be used as a parameter, etc.
  2. What are you using as a source for parcel/ field data?
    1. Terry Sohl: That is one of our biggest challenges. USDA has common land unit data; privacy restrictions on these - we have an older version for these. Homeland security information network and remote sensing data also provide information.
  3. I joined very late so I apologize if you spoke to this but have you spoken to geophysics group in Vermont about this work?
    1. No, will look into this
  4. Can you expand more on the challenges with domain centric languages and approaches to bridge them in a scalable way?

    1. We will need more Rich Signells in the world. It's one thing to know it will work scientifically and another to work it out technically. 
  5. How did the inputs or outputs from one model feed the others in your project?

    1. *UPDATED* The primarily linkages for this initial step were between evapotranspiration monitoring/modeling, and the FORE-SCE land-use model. Land use feeds into calculation of ET, with vegetative water use quantified at the pixel level. ET feeds into land use from the perspective of anticipated water use for a given crop/climate helping to drive likelihood of a given crop being able to be grown. We initiated work on also linking with the ground/surface water model, but unexpected resource reductions (outside of CDI funding) limited progress and final model linkages have not yet been made.
  6. Did you model any policy interventions (or do you see any ones of potential interest to stakeholders)?

    1. Yes - GCAM feeds into FORE-SCE.
  7. Regarding modeling scale at parcel, what's the size of one parcel in general?

    1. *UPDATED* It varies greatly. We have urban parcels that were down to the lot level in most cases, so often as small as a fraction of an acre. Parcels were also used for agricultural change, with individual parcels again ranging broadly in size, from a few acres, to several hundred. In the Delaware River Basin, most agricultural parcels were relatively small, compared to other parts of the country.
  8. Does any of this modeling require the use of supercomputing clusters?

    1. *UPDATED* FORE-SCE does run on the DENALI HPC system at EROS. The evapotranspiration work was run in USGS Cloud Hosting Solutions architectures.
  9. Can you expand on what you needed to add with your "own code", what tools/languages met your needs? Is it reusable/accessible?

    1. Kolb: used Python, JavaScript, and this code will be published on code.usgs.gov. Incorporating and querying the different layers, parameter traces, etc., are things we wrote code for.
  10. When you said "get creative" with staffing, did that include training of new/more programmers? // You mentioned that you needed to get creative with staffing for finding programmers. Can you expand on what your solutions were?

    1. Kolb: Trained new programmers. As part of the CDI funding, some people on the team were sent to do technical training. Also got some staff loaned to them.
  11. Some fires burn hotter than others. Does the model take into account estimated temperatures and their effect on soil composition and subsequent runoff?
    1. Kolb: No. Temperature is not taken into effect. It can calculate burn severity, which can be taken into account. This is taking a lot of programming so is not done yet. 
  12. If the error is in NHD+, shouldn't that be corrected instead of fixing your results manually?

    1. Kolb: We have talked with the NHD+ team, but these processes take a lot of time to get through (years). Our goal is to get this out now, ASAP, so we couldn't wait.
    2. Peter McCarthy: NHD+ is not currently maintained, moving to another version.