Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Madison will lead a discussion about the proposed page on the Data Management Website about reviewing metadata.

Reviewed user stories for Reviewing Metadata page on DM Website:

    • As a technical reviewer of someone else’s metadata, I need to know where to start, how to approach the review, what to look for, so that I can  know that I’ve done a good enough job in my review.
    • As a dataset creator who is writing metadata for my dataset, I need to know what kinds of things the reviewer will be looking at, what they will want to see in the metadata, and how they will judge what I’ve written, so that I can do a good job and so that the review process doesn’t take too long.
    • As a manager who needs to make sure the data releases in my center are reviewed and approved well and in a timely fashion, I need to understand what “reviewing metadata” entails so that I can assign the right people to the job and so that I can have realistic expectations about how much time and effort that job will take.
    • As a member of a group developing policy for my organization, I need to know what aspects of metadata are consistent enough that they can be reviewed with rigor and which are harder to review, so that our policy can support a realistic work process for both producers and reviewers of metadata.

Current resources on Peter's site:

Discussion:

  • Group has experience with people needing reviewers and guidance on reviewing
  • Paul: Considered developing more specific checklist for sections, with boxes to tick
    • Ie. Is the date in the right format?
    • Dennis: Has to stay general - identify common things to address and include
    • Tom: Concerns that the checklist would get too long
  • Would it be worth it to put examples/checklists more specific to individual centers/mission areas on the website or should we keep it general?
    • Public facing website keeping it general, thematic topic lists “behind closed doors” on the confluence site
    • If some mission areas want to develop additional standards, they could be supplemental documents to be included on the wiki
  • Checklists for review on Confluence have not been updated since 2016 (at least for Woods Hole)
  • How to find reviewers/time taken is too specific for public-facing website
    • Agreed, and data review takes a substantial amount of time - try to emphasize that since the process can be lengthy, the scientist/reviewer should start early
    • Include this information in the Plan and Metadata Review section of the DM Website
  • Concern that content under data release is currently buried (result of Druple/wret migration)
    • Creating a new Reviewing Metadata page should help with making some of this content more accessible.
  • Different processes for metadata authors that are new and those that are experienced. Depends on the type of data release as well - different scientific subjects will contain different elements.
  • The level of detail of the review depends on the experience of the author
  • Providing tools/example workflows for metadata review?
    • Peter - converts XML to TXT and then pastes in a word processor and adds comments there
    • Sofia - looks in XML at text editor and metadata wizard - use screenshots to show errors and where edits need to be made
    • Colin - in the review tab of metadata wizard, a button will generate a “pretty print” (word document that is exported, opens in word directly from metadata wizard) of the XML, has a list of the schema errors. This is usually the document that goes into XML view. Combines structural review with content review.
    • Mikki - Review checklist (word document) has a place to make comments that looks like review memo, goes into IPDS along with word doc of metadata with comments
    • Andy: Preview of metadata wizard is copied and pasted into a word doc, have places to put comments and replies (reconciliation).
    • Kitty: Uses OME - when you download it comes out as a couple different formats, likes outline format that it creates. Writes comments and scans it back to the creator. Outline format is HTML.
    • Dennis: Copy/past one of the formats out of metadata parser and copy/paste into a doc, includes validation as part of it. Nice to provide mp as an option that you don’t have to download
  • Other tips/tricks for new reviewers
    • (difference between content and structural review)
    • Someone new might want to write a metadata record as a training method
      • Something that could be noted on the dm website
      • Notes for finding a reviewer - note that the reviewer should have written a metadata record and be familiar with the format
    • People who had never reviewed metadata were asked at speaker’s center - can have teams on a release that has people with different skill sets - some people review the GIS component, someone else with water quality experience, etc.
    • Emphasize that the metadata review can be split up by expertise (someone familiar with metadata, someone familiar with subject matter, someone familiar with GIS)

Mon. July 1, 2019, 2pm - 3 pm EST

...

  • Could be helpful for new reviewers: recommendations on how to start, what are the steps in the process. For example, copy/paste into Word doc for comments
    • Benefit of Word doc – can add comments, include doc in reconciliation materials in IPDS
    • Can be helpful to view with stylesheet as opposed to XML tags
  • A supervisor checking a data release wondered if there was a more specific checklist for supervisors
    • In addition: the supervisor wasn’t sure how to navigate multiple data and metadata files bundled together on landing page of ScienceBase data release.
    • Another meeting participant noted that supervisory review isn’t technically required, just data and metadata peer review.
  • A couple comments addressed the fact the doc is very general and high-level:
    • Meeting participant noted: splitting it into sections would make it FGDC-specific and document is meant to be broad and applicable to all metadata
    • MP validation is often the first step for metadata review – information about MP isn’t separated out from intro paragraphs or marked by a bullet – should it be more emphasized here?
    • Suggestion: maybe split bullets into categories by CSDGM section?
      • Meeting participant noted: splitting it into sections would make it FGDC-specific and document is meant to be broad and applicable to all metadata
  • A participant’s example of their process: they don't reference checklist for most reviews – instead, read metadata as a whole, check for coherence, completeness etc.
  • Comment: it would be nice to be able to actively use this doc as a checklist, with spaces for checks next to the bullet points
    • Maybe have a box at the end that says “see attached sheet for additional checks”? Or a note that says this is the start of a checklist?
    • Maybe an editable PDF?
    • Participant response: this might suggest to users that the bullet list is a comprehensive check. The language in the doc is important to notice: “For example, verify that:”
      • Maybe have a box at the end that says “see attached sheet for additional checks”? Or a note that says this is the start of a checklist?

















Specific Comments about bullet points in the document

...

  • Questions about recommended elements in the title and whether they are required
    • Note: important to keep in mind that there can be outliers
    • Example: can be helpful to include when there is cooperative work for various agencies
    • A few authors didn’t want to put the “when” in the title because they didn’t think it was necessary there (and might be unclear)
      • Note: important to keep in mind that there can be outliers
    • Why is “who” included in the title?
      • Example: can be helpful to include when there is cooperative work for various agencies
    • Possible solution: add “if applicable” to all the elements of the title (instead of just “scale”)
  • Question about coordinate system and datum – what to do if you have an aggregation over many years – do you generalize coordinate system information
    • Example from Woods Hole – data compilations of datasets going back 50 or 60 years. There’s a variety, can sometimes make guesses based on dates collected
  • Note about entity and attribute section review: you can use the Metadata Parser to output the entity/attribute section as a CSV table
  • Idea: create a 3rd document that contains tips and tricks that could be helpful for reviewers
    • Can contain: options of secondary validation in MP, different ways people can work on a review (e.g., Word doc methods)
  • Future discussion topic: keywords (wait until Peter Schweitzer can join us)
  • Should contact information check be included in the checklist? 
    • Content in these fields is variable, so may not fit in the list a bullet point check. 
    • Some centers have this standardized. 
    • Future topic of discussion?
  • Discussion topic: network resource name field – what do people use?
    • This can be helpful for large data releases with complex structures, multiple child items
    • Note: this is the default in the Metadata Wizard, so it's what folks often use
    • Woods Hole Coastal and Marine Science Center enters a direct download link to the data and the URL of the child item.
      • This can be helpful for large data releases with complex structures, multiple child items
    • Many others use data release DOI (that resolves to landing page) for the networkr field
      • Note: this is the default in the Metadata Wizard, so it's what folks often use
  • Question: how many child items do most data releases have?
    • key consideration: does adding child items/folders improve accessibility? Varies by case
    • Majority have none or only two/three
    • A few have subfolders, but this is rare
      • key consideration: does adding child items/folders improve accessibility? Varies by case


Mon. Oct.1, 2018, 2pm - 3pm EDT

...