Agenda (in Eastern time)

11:00a Welcome and Opening Announcements

11:10a Working Group Announcements

11:20a   Application Evaluation: How to get to a Portfolio of Mission Effective Applications, Nicole Herman-Mercer, USGS 

11:50a   USGS Open Innovation Strategy for Crowdsourcing, Citizen Science, and Competitions - Sophia B Liu, USGS Science & Decisions Center

12:30p  Adjourn

Abstracts

Application Evaluation: How to get to a Portfolio of Mission Effective Applications

Web-based Applications and tools are an important mechanism for delivering data, science, and information.  Existing Water Resources Mission Area (WMA) web applications provide several types of science information: data for download to scientists, policy makers, and the public; important tools for decision-makers in water management and emergency situations; great educational materials for students of all ages.  While WMA web-based applications are important to many different users across the country, a comprehensive evaluation of how well we are meeting our users’ needs has never been completed.  Systematic evaluation can help us understand how well we are meeting our science communication and delivery goals and identify how well our applications are meeting the needs of stakeholders and other users.  This presentation will give an overview of the Portfolio Evaluation in support of Mission Effective Applications (MEA) project.  The MEA project is using a variety of methods and tools to measure the use, value, impact, and reach of WMA web-based applications that will be described.   We welcome questions and feedback on this project as part of this presentation.

Nicole Herman-Mercer is a social scientist in the Decision Support Branch of the Water Resources Mission Area's Integrated Information Dissemination Division.  Nicole has been working for the USGS since 2008.  In addition to evaluating web applications she manages a community-based water-quality monitoring program in the Yukon River Basin of Alaska and Canada with partners from the Yukon River Inter-Tribal Watershed Council called the Indigenous Observation Network.  She also conducts research on the impacts of climate change on Indigenous communities in Alaska.

USGS Open Innovation Strategy for Crowdsourcing, Citizen Science, and Competitions

The USGS has a long history of well-established projects using citizen science and other community-based research methods to enhance USGS science, but these projects are typically ad hoc and developed in response to specific science needs. “Open innovation” has become an emerging umbrella term to refer to the broad spectrum of public engagement methods like citizen science, crowdsourcing, and prize competitions. While there is growing interest within USGS to use these methods, there is currently no bureau-wide guidance, resources, or policy for how to use open innovation methods at USGS. Also, there are often concerns and misconceptions about data quality and validity of open innovation efforts that need to be addressed and dispelled. A comprehensive strategy is needed to provide uniform guidance and direction to USGS scientists, managers, and leadership on how to use open innovation methods most effectively and evaluate their effectiveness while ensuring compliance with Bureau and federal policy. A USGS Open Innovation Strategy is currently being developed that will include USGS-specific Guidance, Catalog, Toolkit, and Policy. These products will be collaboratively developed with input from USGS representatives across all mission areas, regions, and science support offices to ensure this strategy is relevant, reputable, and relatable to USGS programs and priorities. To participate and provide feedback on this strategy, join and contribute your input at the USGS Open Innovation Strategy Teams Site.

Sophia B Liu is an Innovation Specialist at the USGS Science and Decisions Center in Reston, VA as well as the USGS Crowdsourcing and Citizen Science Coordinator and Co-Chair of the Federal Community of Practice for Crowdsourcing and Citizen Science. She began working at USGS in 2011 as a Mendenhall Postdoctoral Fellow working on the Tweet Earthquake Dispatch project at the National Earthquake Information Center and the iCoast - Did the Coast Change? Project at the St. Petersburg Coastal Marine Science Center. Starting in 2017, she helped initiate a Crowdsourcing Unit at FEMA’s National Response Coordination Center. Sophia has an interdisciplinary Ph.D. researching the use of social media in disasters, and also received a B.A. in Social Science with minors in Computer Science and Digital Arts.

Highlights

1. Opening
   1. Use the CDI to reach your goals in 2020 - share your topics and questions on the two following platforms
      1. CDI Forum: CDI Community Forum - good for more persistent and discoverable, longer content
         
      2. MS Teams: https://my.usgs.gov/confluence/x/AI2pJg - good for transient, Q&A, reminders, more frequent content. 
   2. CDI Activity Poll - what will help you reach your goals?
      1. cassandra ladino: I would like to see a USGS offering for DMBOK training
         
         
2. Group Announcements
   1. Usability collaboration area is looking for people who are interested in being usability test users. Sign up at: https://my.usgs.gov/confluence/x/ZMmpJg December Usability event: Inaugural meeting was on Wednesday, 12/18/2019. Meeting information is available: https://my.usgs.gov/confluence/x/17mpJg, Link to the Usability Collaboration Area’s wiki: Usability Collaboration Area
   2. Paper to be discussed in the Semantic Web Working Group on Thursday: Semantic Web WG Paper: See https://pubs.er.usgs.gov/publication/70073700
      
   3. Data Visualization Past PNAMP visualization tool materials: https://my.usgs.gov/confluence/x/oYIAJw
   4. USGS Open Innovation Strategy Team: http://bit.ly/USGSOpenInnovationTeam  (full link - for USGS)
      
      
3. Application Evaluation: How to get to a Portfolio of Mission Effective Applications
   1. This was a study conducted in the USGS Water Resources Mission Area.
   2. To get citations of the apps (acronyms, application name, URL), the Publish or Perish software was used (https://harzing.com/resources/publish-or-perish).
   3. Study report (for USGS)
   4. Nicole would be interested in more discussion about how to mine policy docs for information on use of tools
   5. The study was based on the Landsat Imagery Use case studies, Uses and Benefits of Landsat Imagery: https://www.usgs.gov/centers/fort/science/case-studies-landsat-imagery-use
      
      
4. USGS Open Innovation Strategy for Crowdsourcing, Citizen Science, and Competitions
   1. An overview of well-established USGS crowdsourcing, citizen science, and prize competition projects were listed in the presentation.
      1. 44 USGS crowdsourcing and citizen science projects, 13 projects mentioned in the presentation
      2. 19 USGS prize competitions
   2. The following various Federal open innovation, crowdsourcing, citizen science, and competition resources were presented.
      1. CitizenScience.gov
      2. Challenge.gov
      3. EPA Citizen Science / EPA Challenges & Prizes / EPA Generic ICR for Citizen Science and Crowdsourcing Projects
      4. US Forest Service Citizen Science
      5. Bureau of Reclamation Water Prize Competition Center
      6. US Fish & Wildlife Service Citizen Science
      7. NSF Public Participation in STEM Research Letter
      8. NASA Open Innovation / NASA Citizen Science / NASA Goddard Space Flight Center Citizen Science / NASA SOLVE / NASA International Space Apps Challenge / NASA Centennial Challenge
      9. FEMA Crowdsourcing Toolkit for Emergency Management
   3. Participants during the presentation were asked to use the following Mentimeter poll to answer short questions and provide feedback on the talk. People can still participate in this poll and view the voting results with the following links.
      1. Mentimeter Voting Link (no code required)
      2. Mentimeter Voting Results
   4. Sophia is looking for representatives from across all USGS mission areas, regions, and science support offices interested in giving feedback on the guidance, catalog, toolkit, and policies she is developing for the USGS Open Innovation Strategy. Feedback can be provided by joining the USGS Open Innovation Strategy Teams Site or emailing her at sophialiu@usgs.gov.

Q&A

Group Announcements

Q: Jason Ferrante : Sophie, can you please give a laymans definition of "usability" - I think I know what it is, but a clarification would give me better context if you don't mind.

A: From Sophie Hou : Hi Jason, sure; in terms of the collaboration area, “usability” is defined as a user interface’s user friendliness/ease of use. On the collaboration area’s wiki, I included an image showing how “usability” is related to “utility” and “usefulness”. I hope this is helpful, but please feel welcome to get in touch if you have additional questions or would like me to provide additional information. Thanks!

Application Evaluation: How to get to a Portfolio of Mission Effective Applications

Q: Nicole - how did you get citations for your applications?
A: To get citations of the apps (acronyms, application name, URL), the Publish or Perish software was used (https://harzing.com/resources/publish-or-perish).

Q: Hi Nicole, did you weigh the survey responses equally? Or do you have prioritized user types? If you have prioritized user types, how did you determine these?
A: There was not weighting for the internal responses. Later there will be external responses and there may be prioritization.

Q: Will you also perform user studies/usability testing with your tools?
A: Yes, that is in the next steps.

Q: How did you handle PII? I understand that we shouldn't be capturing IP addresses...only down to the domain level?
A: Dept of Interior has some sort of clearance for Google Analytics data.

USGS Open Innovation Strategy for Crowdsourcing, Citizen Science, and Competitions

Participants during the presentation were asked to use the following Mentimeter poll to answer short questions and provide feedback on the talk. People can still participate in this poll and view the voting results with the following links.

Mentimeter Voting Link (no code required)

Mentimeter Voting Results

(Screenshot of the word cloud for one of the Voting Questions)