CDI SSF Category : Computational tools and services
Applicants/Principal Investigator(s) : Glenn Guempel, USGS NGP Standards Architect; Rob Dollison, USGS NGP Delivery Services Lead
Abstract - At the first 2009 CDI workshop, scientists stated one of the top three goals was to make it very simple to access corporate databases. For example, taking a map session mashup they make in web views and move it easily into their advanced thick clients like ArcMap and Google Earth. It could take a scientist, planner, user, etc. a half-hour or more to create these unique views - zoom in, re-order services, change opacity, add services, turn on layers, etc. - and to have to recreate that in their environment, which could be one of the large reasons why services are not taking off - its too hard. The National Geospatial Program took lead and developed a capability to save the map session mashup setup or "context" in one-click in a standard map context then open it in web viewer APIs and popular thick clients like ArcMap and Google Earth. NGP has worked over 3 CDI years collaborating with CIDA, WiM, NEIC, CSAS, and EGSC on advancing this concept to work not just in The National Map applications and services, but the any web services and applications that can support saving as multiple standards and opening with a focus on being able to use this in any web viewer application in USGS. This year's proposal is to focus initial deployment of current SaveAs and OpenIn services and support continuous improvement on standardizing the context, hooking into OpenLayers and ESRI APIs.
% | Funding Source |
31% | Total funding amount requested from CDI |
69% | Total in-kind funding (NGP, OGC, CSAS) |
100% | Total Funding Needed |
Datasets | All Open-Standard Web Services available in USGS Web Viewers (i.e. TNM, NHSS, HDDS, Atlas, Water, MRDATA, NGMDB, CRC, MARIS, Multiple CSAS datasets |
---|---|
Geographic/geologic/ecosystem/habitat/taxonomic/other context | Any context requiring Spatial Visualization (i.e. Geographic, Geologic, Water, Ecosystem, Habitat, Topogrpahic, etc.) |
Type of Product(s) Generated |
|
The summary will repeat but provide more details about the information provided on the cover sheet including the following:
At the first 2009 CDI workshop, scientists stated one of the top three goals was to make it very simple to access corporate databases. In 2009, scientists and users made it clear to CDI - Make these setup mashup steps go away - 1-click. Our current viewing and app portfolio across USGS, DOI, and generally industry-wide does not meet this need. We've had many problems to solve such as we have 100s of viewer instances in 10s of flavors, there still isn’t a good standard for this, and we have yet to establish business case for programs and developers to move towards new approach.
This year's proposal is to focus initial deployment of current SaveAs and OpenIn services and support continuous improvement on standardizing the context, hooking into OpenLayers and ESRI APIs.
Name | Organization Affiliation | Mailing Address | Phone Number | Role | |
---|---|---|---|---|---|
Rob Dollison | USGS NGP | USGS HQ Reston, VA | 703-648-5724 | rdollison@usgs.gov | NGP Delivery Services Lead |
Glenn Guempel | USGS NGP | USGS HQ Reston, VA | gguempel@usgs.gov | NGP Standards Architect |
Name | Organization Affiliation | Mailing Address | Phone Number | Role | |
---|---|---|---|---|---|
Rick Brown | USGS NGP NGTOC | Rolla, MO | NGTOC Project Manager | ||
Dave Hughes | USGS NGP NGTOC | Denver, CO | NGTOC Engineering Projects Manager | ||
Robert Djurasaj | USGS NGP NGTOC | Denver, CO | NGTOC Developer Contractor | ||
Aaron Dandy | USGS NGP NGTOC | Denver, CO | NGTOC Developer Contractor | ||
Jim McAndrews | USGS NGP NGTOC | Denver, CO | NGTOC Developer Contractor | ||
Tim Kern | USGS CSAS Sciencebase | Ft Collins, CO | CSAS Sciencebase Developer Lead | ||
Natalie Latyshe | USGS CSAS | Denver, CO | CSAS Sciencebase Project Manager | ||
Sky Bristol | USGS AESIR | Denver, CO | CSAS AESIR Research lead/Sciencebase SME | ||
Mike Frame | USGS CSAS | Oak Ridge, TN | CSAS Hosting/Engineering Lead | ||
Tim Mancuso | USGS CSAS | Denver, CO | CSAS Hosting/Engineering Manager Contractor | ||
David Blodgett | USGS CIDA | Madison, WI | CIDA Architect/Developer | ||
Ivan Suftin | USGS CIDA | Madison, WI | CIDA Developer | ||
John Aguinaldo | USGS ESGC | Reston, VA | ESGC Developer Contractor |
Name | Organization Affiliation | Mailing Address | Phone Number | Role | |
---|---|---|---|---|---|
Multiple | ESRI | Technical Architects Points of Contacts | |||
Multiple | OpenGeo | Technical Architects Points of Contacts | |||
David Wesloh | OGC/NGA | St Louis, MO | OGC OWS SWG Chair | ||
Justin Haas | NGA Palanterra X3 | Fort Belvoir, VA | Palanterra X3 Product Manager |
Name | Organization Affiliation | Mailing Address | Phone Number | Role | |
---|---|---|---|---|---|
Dave Soller | NCGMP Sr. Leader Sponsor | ||||
Nate Booth | CIDA Sr. Leader Sponsor | ||||
Paul Wiese | 703-648-4543 | pmwiese@usgs.gov | NGP Sr. Leader Sponsor | ||
Matt Tricomi | USGS | Denver, CO | 720-244-3063 | mtricomi@usgs.gov | Architect Contractor |
Greg Smoczyk | USGS NEIC | Golden, CO | CDI SME Focus on ArcMap Integration | ||
Gary Latzke | USGS WiM | Madison, WI | WiM Projects Manager | ||
Jon Baier | USGS WiM | Madison, WI | CDI SME Focus on ESRI Flex API Integration | ||
Cassandra Ladino | USGS EGSC | Reston, VA | EGSC Projects Manager |
This project enables simplifying map service integration. The USGS provides many national, regional and local datasets for download, streaming interaction such as WFS/WCS, and analysis. Ultimately, most datasets are presented for visualization in "viewers" with basic navigation and interaction for inspection and even lightweight WebGIS like web service functions, annotations, etc. Many viewers – different APIs, clients, purposes, and niche functions – are invested in at USGS and DOI and the whole Federal Government. This project has the capability to save map contexts of the mashup a user makes in a viewer. Any context requiring Spatial Visualization (i.e. Geographic, Geologic, Water, Ecosystem, Habitat, Topographic, etc.)
Scientists, Analysts, etc. use many varying science and map tools with viewing capabilities to be used while working different parts of a problem. They’ll move from one tool to another: Finding - Use TNM to download foundational data, Discovering - Sciencebase to find thematic data, Processing - CIDA GDP for processing data, Analyzing - ArcMap for in-depth study, etc. For example, taking a map session mashup they make in web views and move it easily into their advanced thick clients like ArcMap and Google Earth. It could take a scientist, planner, user, etc. a half-hour or more to create these unique views - zoom in, re-order services, change opacity, add services, turn on layers, etc. - and to have to recreate that in their environment, which could be one of the large reasons why services are not taking off - its too hard. Meaning, each time they switch tools, they need to re-setup their session or “mash-up” which is not their priority and is a time-waster or nuisance
The solution is not "1 viewer" or "1 viewer API". We are stuck in a multiple viewer environment, we could recommend a few APIs, and restrict others at best. The problem with this is that when someone goes to a new viewer, they don't always have the same backdrop basemaps, common list of overlays, and most viewers don't have capability to add services into the viewer, except thick clients and The National Map. Essentially there is no way for a user to jump easily between viewers. The output of this project is to implement a web service that any viewer API can call that allows a user to save the last view of the multiple data map services. By having this capability become USGS/DOI wide, or even integrated late into the Geospatial platform, this should look to increase web service use by making easier to use between different web viewers and thick clients. |
The goals this year by August 31, 2013 are to:
Task | End Product/Service | Organization |
Geospatial - Save As Framework | Live Application Service with initial Transforms to Px3, ESRI, KML, and later, support more formats as source and target formats, such as new OGC beta | NGTOC/OGC |
Architecture, Design, Coordination | OGC OWS Context Feedback incorporated into standard | NGP/OGC |
Viewer Library API, SaveAs, and OpenIn Migration to SB/CSAS | Interim and Mid-term hosting decision made/planned - Explore Enterprise rollout discussions/decisions (communicating, training, hosting, investing, inventory) and moving towards program/developer adoption education, communication, and working with programs on where fits in priorities | CSAS/NGTOC |
Open In OpenLayers Framework (Read Px3 JSON, ESRI JSON | Beta OpenLayers submitted to Sciencebase | CSAS/CIDA |
Save to ArcMap (10 days) | Add transform to MXD | NGTOC |
Open in Flex (Convert Px3 JSON to ESRI JSON) | OGC ESRI/REST Context Feedback incorporated into standard | NCGMP |
The CDI Funding required would be for 30% of the effort for the non-NGTOC/NGP items. More details on the technical approach, level of effort commitment by organization, and quarterly timeline are provided below.
Active Discussion Threads
Background Architecture |
Task | Group | Acronym | $ Target | Govt Lead | Developer/SME | Developer Location | Experience |
Geospatial - Save As Framework | National Geospatial Technical Operations Center | NGTOC | NGTOC | David Hughes / Rick Brown / Rob Dollison | NGTOC Team led by Rick Brown | Denver Bldg 810 |
|
Architecture, Design, Coordination | National Geospatial Program | NGP | NGP | Glenn Guempel | Matt Tricomi (Contractor) | Denver Bldg 810 |
|
Viewer Library API, SaveAs, and OpenIn Migration to SB/CSAS | Core Science Analytics and Synthesis | CSAS | CSAS | Tim Kern / Natalie Latyshe | Tim Kern / Tim Mancuso and coordinating heavilly with NGTOC | Denver/Fort |
|
Open In OpenLayers Framework (Read Px3 JSON, ESRI JSON | Core Science Analytics and Synthesis | CSAS | CIDA/CSAS/EGSC | Natalie Latyshe / David Blodgett | Ivan Suftin / Tim Kern and input from John Aguinaldo and the Openlayers HTML5 library efforts | Fort/Madison |
|
Save to ArcMap (10 days) | National Geospatial Technical Operations Center | NGTOC | NGTOC | David Hughes / Rick Brown | NGTOC Team led by Rick Brown | Denver Bldg 810 |
|
Open in Flex (Convert Px3 JSON to ESRI JSON) | National Cooperative Geologic Mapping Program | NCGMP | NCGMP | Chris Garrity | Chris Garrity/ (Input from WiM Jon Baier) | Reston |
|
We will not be submitting bios, profiles, or CVs for this project.
Task | End Product/Service | LOE (Days) | % | Funding Source | Acronym |
Geospatial - Save As Framework | Live enterprise Application Service with initial Transforms to Px3, ESRI, KML which will be code maintained by NGP and hosted mid-term by CSAS | 90 | 46% | NGP and OGC Funding | NGTOC |
Architecture, Design, Coordination | OGC OWS Context Feedback incorporated into standard | 20 | 10% | NGP and OGC Funding | NGP |
Viewer Library API, SaveAs, and OpenIn Migration to SB/CSAS | Interim and Mid-term hosting decision made/planned | 15 | 8% | NGP and CSAS Funding | CSAS |
Open In OpenLayers Framework (Read Px3 JSON, ESRI JSON | Beta OpenLayers submitted for incorporation as part of Sciencebase code base | 30 | 15% | CDI Funding | CSAS |
Save to ArcMap (10 days) | Add transform to MXD as part of Save As application service | 10 | 5% | NGP and OGC Funding | NGTOC |
Open in Flex (Convert Px3 JSON to ESRI JSON) | OGC ESRI/REST Context Feedback incorporated into standard which would become one of the primary ESRI Flex libraries maintained by CSS. | 30 | 15% | CDI Funding | NCGMP |
Please provide a detailed budget for the project using Table 3. Include at least the following categories of information, separating the CDI funds from the in-kind match:
Included in official Proposal document
Included in official Proposal document
Included in official Proposal document
*Noting the Bureau’s goal of reducing travel by 30% in FY13.
Milestone | Geospatial - Save As Framework | Architecture, Design, Coordination | Viewer Library API, SaveAs, and OpenIn Migration to SB/CSAS | Open In OpenLayers Framework (Read Px3 JSON, ESRI JSON | Save to ArcMap (10 days) | |
---|---|---|---|---|---|---|
Q1 | Plan OGC Testbed with NGP EA Test ArcMap Initial Deploy on NGP HW | Re-organize all past 4 years of SaveAs/OpenIn Content Facilitate Transition of concepts/designs to New Teams and Design | Scope out FY12 scope issues Integrate into SB plans | Scope out FY12 scope issues Integrate into SB plans | Complete ArcPy Libraries Initial Deploy on NGP HW | Scope out FY12 scope issues
Integrate into SB plans |
Q2 | Complete KML Transform NGTOC begin working w/ OGC SWG on getting OWS JSON working in OGC Testbed (OGC Funding?) | NGP EA investigate REST SWG for context overlap and changes to ESRI OWF Context with OWS Context Confirm if direction for Context Standard should be OWS or REST OWF changes JSON | Test hosting ESRI JS Libraries Test hosting ESRI JS with Px3 Libraries Test hosting SaveAs Libraries Test hosting OpenIn Libraries | Complete OpenIn OL tasks Complete OL Library transitioning from CIDA Work Align communications to get other CSS OL libraries using such
| Coordinate SB CSAS Scope needs | NGMDB Develop SaveAs hooks for CSS Wide Flex Library |
Q3 | Complete TestBed and OWS JSON Feedback to OGC NGTOC Build OWS Reader with NGA Px3 Team (CDI Funding) NGTOC Write SaveAs w/ Source as OWS (currently is Px3) Coordinate Hosting Stack Transition to CSAS/Bldg 810 Stack | Support OGC Communications Document exploring REST JSON config changes to OWF JSON and further faciltate | Test hosting OL Libraries Test hosting ESRI Flex Libraries
| Complete SaveAs OL tasks CSAS coordinate with NGP and write OWS Reader with CIDA team for OL API | Transition to CSAS | NGMDB Complete libraries for Flex to be hosted CSS Wide Explore Writing OWS Reader for ESRI Framework (currently reading ESRI/OWF JSON) |
Q4 | Transition deployment from NGP to CSAS | Support Migration to CSAS for hosting | Deploy all above Libraries in CSAS for CSS wide use | Deploy OpenIn and SaveAs in SB CSAS | Transition deployment from NGP to CSAS | NGMDB Migrate Libraries to CSAS to hosting |
In the early 2000s, OGC sponsored the Web Map Context Profile XML standard. The idea was have a standard configuration file with the initial view setup for basemaps, URLs for legends, metadata and web map, which maps were turned on by default, what other overlay WMSs are available in a pick list, what is the default title, order, transparency and grouping. If you made a map, turned layers on/off, changed order, transparency, zoomed, pan, you could then save your new "mashup" as its called now as an OGC Context Profile for later on that site, or if another site supported reading that standard, could just drop it into another viewer. Also, this is not just a viewer reuse, but also analytical processing re-use.
Great concept, but didn't take off. Only Geosptial One-Stop on a large scale produced the file. Since then, all viewers support the user to make their own context like OpenLayers, ESRI JS or Flex API, Google Maps, Bing. And most also are trending towards JSON (Javascript Object Notation) as the context profile language of choice. But most developers are choosing their own context profile standard. This is very typical for developers to do without a standard to refer to.
In 2008, NGA Palanterra x3 went this direction and nicely documented their configuration JSON. USGS The National Map followed suit using that and extending with download framework. ESRI as a corporation came out with a 229-page paper for the REST specification, noting a configuration schema, and was released defaultly with Flex, ESRI JS API, and ArcGIS.com. It is similar to the Px3 implementation, but there are differences. USGS asked OpenGeo Openlayers if they would be up for a JSON context profile standard, and they said they were interested in collaborating as did ESRI ArcGIS / Rest Spec Team.
In 2011, OGC has advanced the context even further by creating map and processing context - more powerful than the original. This is up for standard acceptance in September 2011. *Check out the OWS Context Schema Experiment
In 2012, OGC completed a draft standards in XML and accepted to move to adding a JSON format support in 2013.