Metrics that are collected about the CDI, and their potential impact, are described below.
On the ScienceBase pages, under the external resources sub-section, all known projects are listed and categorized by output type. Output types are
Data Release: A formal USGS data release that will go through FSP review and approval
Mobile Application: Interactive application built specifically for a mobile device
Presentation: Slides, video, or other presentation media
Publication: Peer-reviewed publication (USGS or external journal publication)
Software: Executable or compiled code that can be downloaded
Source Code: A code repository for the project's source code
Web Application: Interactive application that runs on a web browser
Web Link: Project webpage, wiki page, white paper, or online resources that do not fit other categories
Web Service: A service endpoint URL where your service can be accessed by a client application
A publication about CDI funded projects, outputs, and sustainability is
Hsu L, Hutchison VB, Langseth ML (2019) Measuring sustainability of seed-funded earth science informatics projects. PLoS ONE 14(10): e0222807. https://doi.org/10.1371/journal.pone.0222807
Publications that are about a project that was supported by CDI, are about CDI, or has a reference to CDI in the text, are listed on the wiki at https://my.usgs.gov/confluence/display/cdi/Publications. This list has been maintained manually, but we are working on a way to automate updates to the list, and maintain a publication list on Zotero.
Link to Grace's document content.
In the project guidance, we ask that projects acknowledge CDI support with the statement:
This work was supported by funding from the USGS Community for Data Integration (CDI).
Citations to CDI-funded projects have so far been done manually, using Google Scholar to check individually for citations to publications or software. This is an area for improvement, since citations to CDI work can be used as a simple and understandable metric for impact of CDI funding.
Action: Link to Citations data.
Grace Donovan has created a Jupyter Notebook that reads in a .csv of CDI member emails and reports statistics from the email list.
Metrics that give us good information for our programming include:
Not yet implemented:
Annual Reports from FY10 to present can be seen on the wiki, if logged in, at
Since 2013, CDI Annual Reports have been published publicly as USGS Open File Reports.
In FY17 we started publishing separate, stand-alone funded project reports. This information used to be included in the longer annual reports, but by creating a separate report, we allowed the teams more time to report links of their deliverables (especially reports or papers that had to make it through the peer review process). This results in a more complete and digestible summary of the funded projects.
Funded Project reports can be found on the publications page of the public webpage.
The CDI page aims to communicate metrics through the following pages
Products page - a parent page that links to publications, project outputs, workshop posters, and other CDI products.
Publications page - a non-fancy text-based page of publications that were supported by CDI, about CDI, or have a reference to CDI in the text.
Membership page - Action: this page needs to be updated to reflect metrics about our members.
CDI Publications page - official feed of linked publications from the USGS Publications Warehouse
The CDI coordinator, facilitators, their supervisors, and sponsors may have CDI-related metrics built in to their annual performance evaluation. These should be reviewed at the beginning of the year and on a quarterly basis to ensure achievement.
A list of past performance metrics is on the CDI wiki here. Action: create and link this content.