Confluence Retirement

In an effort to consolidate USGS hosted Wikis, myUSGS’ Confluence service is scheduled for retirement on January 27th, 2023. The official USGS Wiki and collaboration space is now SharePoint. Please migrate existing spaces and content to the SharePoint platform and remove it from Confluence at your earliest convenience. If you need any additional information or have any concerns about this change, please contact myusgs@usgs.gov. Thank you for your prompt attention to this matter.
Skip to end of metadata
Go to start of metadata

What: Planned evaluation method

Metrics that are collected about the CDI, and their potential impact, are described below.

Funded Project tracking

The CDI has been funding projects annually since 2010. Each project has a page with a description and list of outputs on ScienceBase, at Community for Data Integration (CDI).

On the ScienceBase pages, under the external resources sub-section, all known projects are listed and categorized by output type. Output types are 

Data Release: A formal USGS data release that will go through FSP review and approval

Mobile Application: Interactive application built specifically for a mobile device

Presentation:   Slides, video, or other presentation media

Publication:   Peer-reviewed publication (USGS or external journal publication)

Software:    Executable or compiled code that can be downloaded

Source Code:    A code repository for the project's source code

Web Application: Interactive application that runs on a web browser

Web Link:    Project webpage, wiki page, white paper, or online resources that do not fit other categories

Web Service:    A service endpoint URL where your service can be accessed by a client application

A publication about CDI funded projects, outputs, and sustainability is

Hsu L, Hutchison VB, Langseth ML (2019) Measuring sustainability of seed-funded earth science informatics projects. PLoS ONE 14(10): e0222807. https://doi.org/10.1371/journal.pone.0222807

Publications

Publications that are about a project that was supported by CDI, are about CDI, or has a reference to CDI in the text, are listed on the wiki at https://my.usgs.gov/confluence/display/cdi/Publications. This list has been maintained manually, but we are working on a way to automate updates to the list, and maintain a publication list on Zotero. 

Link to Grace's document content. 

In the project guidance, we ask that projects acknowledge CDI support with the statement: 

This work was supported by funding from the USGS Community for Data Integration (CDI).

Citations

Citations to CDI-funded projects have so far been done manually, using Google Scholar to check individually for citations to publications or software. This is an area for improvement, since citations to CDI work can be used as a simple and understandable metric for impact of CDI funding.

Action: Link to Citations data.

Membership Statistics

Grace Donovan has created a Jupyter Notebook that reads in a .csv of CDI member emails and reports statistics from the email list.

Metrics that give us good information for our programming include:

  • Total number of members (and quarterly growth).
  • Fraction of USGS members. (Helps to determine how much to focus on USGS-specific topics)
  • Fraction of .gov, .edu, .org, .com, and other domains.
  • List of emails for the previous categories. (Helps to identify particular members for requests for presentations and other tasks.)
  • USGS employee by USGS mission area, region, science center. (Helps to connect members.)

Other metrics

  • Monthly meeting attendance. (Helps to plan future meetings with popular topics.)
  • Webpage traffic, devices, clicks. (Helps to modify the webpage in a way to get users the information they need.)
  • Total funds contributed to funded projects and events.
  • Members that contribute a lot of content and why.
  • Questions and comments on wiki (not yet)
  • Questions and comments during monthly meetings (not consistently)
  • Twitter interactions (not consistently)
  • RFP metrics by mission area
  • RFP participants and comments

Not yet implemented:

  • Email link opens and clicks. (Helps to determine what content is useful to most members)
  • Wiki page statistics. (Helps to modify the wiki in a way to get users the information they need.)

How metrics are communicated

Annual reports

Annual Reports from FY10 to present can be seen on the wiki, if logged in, at

https://my.usgs.gov/confluence/display/cdi/CDI+Annual+Reports 

Since 2013, CDI Annual Reports have been published publicly as USGS Open File Reports. 

Funded Project reports

In FY17 we started publishing separate, stand-alone funded project reports. This information used to be included in the longer annual reports, but by creating a separate report, we allowed the teams more time to report links of their deliverables (especially reports or papers that had to make it through the peer review process). This results in a more complete and digestible summary of the funded projects. 

Funded Project reports can be found on the publications page of the public webpage.

https://www.usgs.gov/centers/cdi/publications 

Wiki

The CDI page aims to communicate metrics through the following pages

Products page  - a parent page that links to publications, project outputs, workshop posters, and other CDI products.

Publications page - a non-fancy text-based page of publications that were supported by CDI, about CDI, or have a reference to CDI in the text.

Membership page - Action: this page needs to be updated to reflect metrics about our members.

USGS Website

CDI Publications page - official feed of linked publications from the USGS Publications Warehouse

Annual Performance Metrics

The CDI coordinator, facilitators, their supervisors, and sponsors may have CDI-related metrics built in to their annual performance evaluation. These should be reviewed at the beginning of the year and on a quarterly basis to ensure achievement.

A list of past performance metrics is on the CDI wiki here. Action: create and link this content.


  • No labels