Skip to end of metadata
Go to start of metadata

February 10, 2021: Bathymetry VR and CDI 2021 Workshop: Designing Data Intensive Science


Meeting Recording and Slides

Recordings and slides are available to CDI Members approximately 24 hours after the completion of the meeting.

Log in to view the meeting resources. If you would like to become a member of CDI, join at https://listserv.usgs.gov/mailman/listinfo/cdi-all.

During the call, you can ask and up-vote questions at slido.com, event code #CDIFEB.

Agenda (in Eastern time)

11:00 am Welcome and Opening Announcements

11:15 am Working Group Announcements

11:25 am  Bathymetry-VR (Virtual Reality) - Dan Beckman, Martyn Smith, Greg Petrochenkov, Hydrologic Applied Innovations Lab (HAIL), USGS

11:55 am  Designing Data-Intensive Science, Discussion of topics of interest for the 2021 CDI Workshop - Leslie Hsu, USGS

12:30 pm  Adjourn

Abstracts

Bathymetry-VR (Virtual Reality)

Dan Beckman, Martyn Smith, Greg Petrochenkov, Hydrologic Applied Innovations Lab (HAIL), USGS

What can a virtual reality headset and 3D gaming development platforms do for you and your 3D elevation data? Learn about the tech, and algorithms behind it, that allows you and your colleagues to experience and engage with topographic data. Bring your questions for the presenters from the USGS Hydrologic Applied Innovations Laboratory.

Designing Data-Intensive Science, Discussion of topics of interest for the 2021 CDI Workshop 

Leslie Hsu, USGS

Designing Data-Intensive Science is the theme of this year's all-virtual CDI Workshop, to be held May 25-28. What is data-intensive science? What are the challenges you have in designing data-intensive science? And how can we tap into the expertise in the Community for Data Integration to help address those challenges? We'll review take-aways, resources, and follow-ups from our last workshop and gather data to help us design this year's event. 


Highlights

  1. Submit a session proposal for the CDI FY21 Workshop, due March 1: 2021 CDI Workshop 
  2. The CDI workshop will be held online from May 25-28 
  3. Demo and workflow examples for VR available through recording
  4. Browse the 2021 CDI Workshop wiki page: 2021 CDI Workshop

Notes

Welcome and Opening Announcements

  1. ESIP winter meeting was a useful event. See resources:
    1. https://2021esipwintermeeting.sched.com/
    2. https://esip.figshare.com/
  2. Kevin Gallagher comments
    1. Exciting new topics and communities emerging in CDI
    2. Call for sessions ends March 1st for the 2021 CDI Workshop, May 25-28 (virtual)
      1. Submit a session proposal here: 2021 CDI Workshop
    3. The theme for the conference is "Designing Data Intensive Science"
  3. Tim Quinn comments
    1. The 2021 CDI Leadership and Innovation Award nominations are due February 15, 2021. Submit here: https://my.usgs.gov/confluence/x/DRzbJQ 

Collaboration Area Updates

  1. For more info on any of the collaboration areas: https://my.usgs.gov/confluence/x/yhv1I  
  2. Geomorphology
    1. Next event: 2/23
  3. Usability
    1. Kicked off office hour format this month; went over three discussion topics
      1. Short presentations/discussions
    2. Next event: 2/17, user-centered design process overview
  4. eDNA
    1. Job opportunities popping up in eDNA. Check the eDNA wiki page.
    2. Highlighting publications in eDNA on the wiki page.
  5. Risk
    1. Next event: 2/18, FEMA National Risk Index
  6. Data Management
    1. Next event: 3/8, USGS legal agreements and their relation to data management plans
    2. Ask questions for the OPA: https://forms.office.com/Pages/ResponsePage.aspx?id=urWTBhhLe02TQfMvQApUlMz6Y43cn25MmjKaQXJVO0ZUQkhGUUhLVThCUFAxN0w4RlNKNEdSRzc0SCQlQCN0PWcu 
  7. Data Viz
    1. Next event: 3/18, webinar with Alvin Chang, data reporter
  8. Tech Stack
    1. Next event: 2/11, Elevation data processing at scale - deploying open source geotools using docker, kubernetes
  9. DevOps
    1. Next event: 3/2, USGS EROS, AWS imagery processing workflows
    2. Next event: 5/4, USGS Dynamic Mapping, USGS NGTOC Dynamic mapping - a custom mapping platform for user defined topo maps

Bathymetry-VR (Virtual Reality)

Dan Beckman, Martyn Smith, Greg Petrochenkov, Hydrologic Applied Innovations Lab (HAIL), USGS

  1. Hydrologic Applied Innovations Lab (HAIL)
    1. Look to innovate around a problem to find a solution, with many focus topics
  2. Disclaimer that VR is emerging technology!
  3. XR is a blanket term that encompasses Virtual Reality, Augmented Reality, and Mixed Reality.
    1. Virtual reality has gone from novelty to utility.
    2. To get impact in virtual reality, you need a wealth of data, and have to be mature in the 3D data space. USGS has a large collection of 3D data.
  4. VR
    1. Uses stereo vision by double projecting a scene (creating an individual scene per eye)
    2. Two classes of headset
      1. Purpose built tethered devices (90+Hz)
      2. Cell phone base 6 degrees of freedom devices (~70Hz)
  5. Physics Engines (Unity, Unreal, CryEngine, etc.)
    1. Allow one person to do the work of 100s of people. Efficiency system.
  6. See recording for VR demo.
    1. Video of demohttps://doimspp.sharepoint.com/:v:/s/GS-NYWSC-HAIL/EZ9ljpn6oE5GqqEFaGhks50BC-EJLGryqPppOA1pfE6S2w?e=w5GPbG (USGS only)
  7. Hardware/software for simulation:
    1. Hardware: Used Oculus V1
      1. Pros:
        1. Low-cost headset
        2. Excellent workflow
        3. Solid platform, bug-free
      2. Cons
        1. Service constraints
          1. Facebook owns Oculus
        2. Underpowered for some USGS use cases
        3. Screen resolution on the V1 is less than desirable
    2. Software: Unity
      1. Pros
        1. Free
        2. Simple to learn
        3. Excellent workflow
      2. Cons
        1. Can be buggy
  8. Preparation for 3D terrain object in simulation
    1. Started with Mohawk River bathymetry collected via Vessel Based MBES
    2. Converted to 1 meter DEM using LAStools
    3. 10 meter NHD Plus HR DEM data to fill in gaps
    4. Merge and spatial gaussian filter 20 meter radius with a std dev of 2
    5. Least cost depression breached - following flood inundation modeling, want to make sure you don't have pits in the DEM
    6. DEMTo3D QGIS plugin to create STl
    7. See slides for more detailed explanation.
  9. Simulation workflow
    1. Rendered on an i5 processor (grossly underpowered for this kind of work; bottom rung for CP level)
    2. Terrain .obj into Unity
    3. Add a rigid body (no gravity, yes kinematic,)
    4. Add mesh collider
    5. Add teleportational area component
    6. Child probe objects
    7. Map Xmax, Zmax
    8. Deploy in approximately one hour
  10. Flood Stage API Workflow
    1. As used in rapid FIM techniques, create watersheds and normalize to DEM.
    2. Create cross sections along stream and either aggregate properties or establish max likelihood contributing points to flood on non-stream pixels
    3. See slides for full instructions.
  11. Python Flask API
    1. Leveraging cloud infrastructure
    2. Deploying is very simple. Dependencies on slide, suite of simple Python tools.
    3. Workflow
      1. Call pint via terrain ratios and translate to DEM to generate cross-section with desired length and angle
      2. Call images in to SIM dashboard
      3. Call API to get stage:
        1. Create synthetic rating curve using input Mannings n and computed slope
        2. Assess stage based on input discharge to retrieve stage and assess flood height
  12. Limitations and enhancements
    1. Terrain Object files beyond 250MB slow down the simulation frame rate (non-starter).
    2. Oculus can easily create boundary of 15' x 15', but have to fight device to get to 20' by 20'
    3. Currently our rig (mechanism to act upon VR environments) has four methods of locomotion, want to add two more
    4. Would like to start resting gesture mapping into the integration
    5. Would like to test fully with multi-users
    6. Particle based physics models for floods
  13. How can this technology be used in a variety of domains?
    1. Modeling
      1. Use of established physics engines or development of new physics
      2. Could be useful for validating models
      3. Modular components for more loosely coupled physics and modeling
    2. AI
      1. Chip classification in three dimensions (banks of streams, features in point clouds)
      2. Previously unidentified features to be added for supervised learning
    3. Insights/Analysis
      1. Remote virtual survey of data due to hazards or inability to visit
      2. In simulation analysis workflow and capabilities
      3. Collaboration with other technicians and scientists
  14. Contact email for questions: GS-W-NY_HAIL@usgs.gov

Designing Data-Intensive Science, Discussion of topics of interest for the 2021 CDI Workshop

Leslie Hsu, USGS

  1. Poll: How do you feel about week-long virtual workshops? (sli.do)
  2. What's the workshop about?
    1. Host conversations for sharing and solving data challenges.
    2. Inform community about USGS resources for data-intensive computing and analysis.
    3. Highlight examples of collaborative projects.
    4. Meeting and interacting with people!
    5. Link to previous workshop pages: 2019 CDI Workshop
    6. Link to 2021 workshop page: 2021 CDI Workshop
  3. Ways to participate in the virtual workshop
    1. Plenary talks
    2. Breakout sessions
    3. DataBlast posters and demos
    4. Training
    5. Birds-of-a-feather
  4. Highlights from past workshops
    1. Lightning talks
    2. Key-takeaways from sessions
    3. Formation of new collaboration areas
    4. #1 benefit reported is the networking opportunity
  5. Polls on sli.do
    1. What phrases come to mind when you hear "Designing Data-Intensive Science"?
    2. Challenges in data-intensive science
    3. What DataBlast formats would you prefer to browse as an attendee?
    4. How would you prefer with DataBlast presenters?
    5. You just joined a session at the online workshop. What is your reaction to being told that there will be an interactive position?

Questions & Answers

  1. Was the bridge part of the survey?
    1. No, it was mistake in the presentation. It's actually the Amtrak bridge.
  2. The Army is doing quite a bit of work with synthetic environments. Have you talked with that team at all?
    1. We haven't. My experience is that they sometimes collaborate, but mostly they keep the findings of what you discover to themselves
  3. Can you stay fixed and visualize repeated scans of the same area?
    1. Yes, there is a system for this. You would stack up your models in different scenes; have a lobby space; select which pre-loaded model you want to look at.
  4. Could this be used for groundtruthing?
    1. Yes. Being immersed can be helpful for groundtruthing and gathering insights in to the data.
  5. How would these VR environments be published (IPDS review etc) or distributed to the public and how might the public interact with USGS VR data sets/environments? IS USGS PUBS and IT infrastructure ready for this?
    1. Yes, if you were creating a model. If you're just interacting with data or looking at a model, wouldn't need review. Data interaction device rather than a product.
    2. Very new technology. Difficult to say what the standards will be. Investigation mechanism, could be part of the IPDS review. If using the physics in an actual workflow, part of software release would include dependencies for the software so that it is reproducible. 
  6. Can you talk more about your vision for getting more USGS people up and running with this technology?
    1. Everything was free and open source but need the headset. Tech is getting better and cheaper.
    2. Starting with research and development and demos of capabilities to show utility. If people want to engage in VR for their projects, consult academic resources and presentations to show capabilities. 
  7. What would it take to port this CHS or EROS?
    1. There is a physics engine available for AWS, but not really for this kind of technology. 
  8. What Cloud are you using? 
    1. WIM, web informatics and mapping - Geospatial platform cloud
  9. How can USGS NGTOC provision elevation data services to allow broader adoption of VR/AR visualizations?
    1. Haven't established such a workflow. We pre-processed this terrain object. Looking into streamlining that process. 
  10. Are we expected to join all portions of the workshop?
    1. No, we will have many sessions, and you're free to attend the ones that interest you.
  11. To present at the data-intensive workshop, do we have to be working with the CDI to accomplish the research?

    1. You can present any of your work that you would like to share with the CDI community, you do not need to be working with the CDI on that project.