Confluence Retirement

Due to the feedback from stakeholders and our commitment to not adversely impact USGS science activities that Confluence supports, we are extending the migration deadline to January 2023.

In an effort to consolidate USGS hosted Wikis, myUSGS’ Confluence service is targeted for retirement. The official USGS Wiki and collaboration space is now SharePoint. Please migrate existing spaces and content to the SharePoint platform and remove it from Confluence at your earliest convenience. If you need any additional information or have any concerns about this change, please contact myusgs@usgs.gov. Thank you for your prompt attention to this matter.
Skip to end of metadata
Go to start of metadata

Title

National Geospatial Program, The National Map - Robert Dollison

You are looking at the 2012 view, 2013 proposal is at:

FY13 Proposal: Deploy Save As and Open In USGS Wide

2012 Final Presentation

For the meeting Webinar Meeting Sept05.2012, here is the presentation for this meeting CDI-TSWG Save Map As (Dollison) Status_v1.pptx

Description

Project Topic

Tasks

Resources Required

Major Outcomes

Total Funding Needed

Expand TNM Save As/Open In to USGS Wide

Vet the TNM Concept Discover issues for expanding Explore JSON Context Scope Explore Standardization Process Explore how have 1 Save As/Open In App service that USGS can use leveraging TNM efforts

Representatives from other USGS areas interested in Mashup Travel $ may be needed

USGS Wide approach to Mashup Same Context File Demos of Open Mashup in other Viewers Preferrably OpenLayers, Flex, Silverlight and JS API Mashup reader developed for reuse

- Travel $
- 140 days of labor .5 to .75 FTE for CDI Funding across LCAT (30), WIM(30), CIDA(10), and NGP (10)

Active Discussion Threads

Discussion on JSON and Security implementation
Context Security - Good threads on how we handle cross-domain JSON calls (client side or server side processing is the question)
Standard JSON Context
Context Standard - OGC has drafted the new OWS context in JSON encoding - now onto seeing the use cases and requirements come through.

Introduction

Context profile background:
The USGS provides many national, regional and local datasets for download, streaming interaction such as WFS/WCS, and analysis. Ultimately, most datasets are presented for visualization in "viewers" with basic navigation and interaction for inspection and even lightweight WebGIS like web service functions, annotations, etc. Many viewers – different APIs, clients, purposes, and niche functions – are invested in at USGS and DOI and the whole Federal Government.
The solution is not "1 viewer" or "1 viewer API" - see the "Viewer Explosion Conundrum" below. We are stuck in a multiple viewer environment, we could recommend a few APIs, and restrict others at best. The problem with this is that when someone goes to a new viewer, they don't always have the same backdrop basemaps, common list of overlays, and most viewers don't have capability to add services into the viewer, except thick clients and The National Map. Essentially there is no way for a user to jump easily between viewers.
In the early 2000s, OGC sponsored the Web Map Context Profile XML standard. The idea was have a standard configuration file with the initial view setup for basemaps, URLs for legends, metadata and web map, which maps were turned on by default, what other overlay WMSs are available in a pick list, what is the default title, order, transparency and grouping. If you made a map, turned layers on/off, changed order, transparency, zoomed, pan, you could then save your new "mashup" as its called now as an OGC Context Profile for later on that site, or if another site supported reading that standard, could just drop it into another viewer. Also, this is not just a viewer reuse, but also analytical processing re-use.
Great concept, but didn't take off. Only Geosptial One-Stop on a large scale produced the file. Since then, all viewers support the user to make their own context like OpenLayers, ESRI JS or Flex API, Google Maps, Bing. And most also are trending towards JSON (Javascript Object Notation) as the context profile language of choice. But most developers are choosing their own context profile standard. This is very typical for developers to do without a standard to refer to.
In 2008, NGA Palanterra x3 went this direction and nicely documented their configuration JSON. USGS The National Map followed suit using that and extending with download framework. ESRI as a corporation came out with a 229-page paper for the REST specification, noting a configuration schema, and was released defaultly with Flex, ESRI JS API, and ArcGIS.com. It is similar to the Px3 implementation, but there are differences. USGS asked OpenGeo Openlayers if they would be up for a JSON context profile standard, and they said they were interested in collaborating as did ESRI ArcGIS / Rest Spec Team.
In 2011, OGC has advanced the context even further by creating map and processing context - more powerful than the original. This is up for standard acceptance in September 2011. *Check out the OWS Context Schema Experiment
As well, USGS Community for Data Integration which brings together most viewer developers and managers, in FY11 agreed to piloting a context profile sharing concept, and in FY12, agreed to investigating in adopting a single JSON standard for corporate viewers and evangelizing and sharing code to support for project viewers. This goes for also sharing in the analytical processing context for re-use as well.
Given this momentum, the architecture proposal here-in is the starting basis for the conversation with a goal of not a "big bang" complete standard for all parts of the context, but a core part that is used commonly by all.

Benefit to FSP/Scientists/Mission Areas

See last years project
In FY10, Goal one focused on making it easier for ArcMap users to access Corporate databases. This was achieved by combining the recent service modernization of _The National Map (_TNM), and setting up Palette MXDs with all base geospatial layers from TNM Services pre-loaded along with Character of underlying rock units, and geochemical survey data (MRDATA), Biologigal GAP Land Cover (NBII), and denormalized Stream Flow information. As well, Hydrology build test Web API frameworks for easily adding/ setting up TNM Services in Water Web Application Viewers. %%
The FY10 goal of this would support scenarios such as bureau scientist studying multi-county region to locate sources of contaminants in water and soils.

In FY11, the team is proposing to continue supporting these two users – ArcMap and Web User – by combining the two user stories and automating the web transition to the heavy client. Supporting a user who is searching and exploring services on the web, mashing them up, and wanting to setup the same mashup context view in heavy clients like ArcMap or Google Earth with one-click and without needing technical support to do so.
Save in Single Click
Figure 1 - This continuation of FY10 Goal 1 and supporting scenario intends to deliver simpler usability for migrating contexts setup on the web with easy one-click saving and opening in the popular heavy clients of USGS users. As well, looking to save those contexts and make them registerable and thus searchable in key catalogs.*
Relevance and BenefitsFrom|https://my.usgs.gov/confluence/label/cdi/404040%7Dfrom] FY10 Goal 1:Well over a third of all USGS employees have access to or use enterprise GIS software. These scientists commonly and frequently overlay and integrate USGS collected or managed datasets to conduct their investigations within this GIS. Yet many corporate data systems within USGS do not provide ready and efficient access within a GIS environment.

Background Detail - Viewer Explosion Conundrum example

USGS alone has 100s of viewers on varying dozens of legacy and and dozens of current viewing platforms: dozens of homegrown Java or Javascript APIs, multiple hard-coded Google Maps, some Bing Maps, and several flavors of legacy, OpenLayers, variations on ESRI JS/DOJO API, The National Map Palanterra Extension to JS API, Flex, Silverlight and legacy ESRI viewers. As well, the community of thick client 2.5 dimension viewers such as Google Earth and there are well over 10,000 ArcMap licenses at DOI and a mixed audience of ESRI ArcGIS explorer and a growing interest in ArcGIS.com
This explosion of viewers could be viewed as a lack of governance discipline for USGS for managing software technology, but each viewer tends to more be an application with unique niche functions to that project (i.e. a specific water purity analysis function, a specific biota identification function, etc.). We do not one viewer as we'd create a swiss-army knife scenario. As well, 1 viewer technology does not make sense as to deploy something as reusable, with enterprise support is a lot of overhead and usually gains of such takes many years. Viewer Technologies change every 12 to 18 months, so this investment would be hard to get the reusable payback. Also, given the fast turnaround of viewers, people tend to learn web viewers quickly.
Also, some viewer APIs or clients are better for different situations – JS APIs are good for general viewers are no plug-ins are retired, homegrown APIs good for unique situations as generic COTS do not meet requirements, and slick looking flex and silverlight viewers are great when deployed in situations where plug-ins are issues. Also, thick clients are better for large feature sets as they are fast and make better use of memory.
Example of Viewers Flavors in Use at USGS for WebGIS

Project Management Details

Scope, Deliverables, Roles

Task

Geospatial - Save As Framework

Architecture, Design, Coordination

Open in OpenLayers in CIDA Portal

Save to ArcMap (10 days)

Open In OpenLayers Framework (Read Px3 JSON, ESRI JSON

Open in Flex (Convert Px3 JSON to ESRI JSON)

Deliverable

Develop SaveAs Server-Side Transform ModuleDevelop following transforms:Px3JSON2JSON, Px3JSON2KML, Px3JSON2GeoServicesJSONNote: Also developed session snapshot transforms for PDF, PNG, and JPG already

Key Arch Objectives:- Coordinating Context Standard- JSON Security- Linking 3 APIs to TNM after done - how to do "Open In"Support wiki updates, meetings, documentation

 

 

 

 

LOE Days Budgetted

90

20

10

10

30

30

Group

National Geospatial Technical Operations Center

National Geospatial Program

 

 

 

 

Acronym

NGTOC

NGP

CIDA

NEIC

 

 

 

https://my.usgs.gov/confluence/label/cdi/333333%7Dwim%7Bcolor%7D%7C\

]

Org Funds would go to

NGTOC

NGP

 

NEIC

EGSC,

 

https://my.usgs.gov/confluencenull\

] LCAT

 

 

https://my.usgs.gov/confluence/label/cdi/333333%7Dwim%7Bcolor%7D%7C\

]

Govt Lead

David Hughes

Kevin Hope

 

Lynda Lastowka

Cassandra Ladino

 

Funding Source

NGP

TBD

 

 

https://my.usgs.gov/confluence/label/cdi/333333%7Dcdi%7Bcolor%7D%7C\

]

 

https://my.usgs.gov/confluence/label/cdi/333333%7Dcdi%7Bcolor%7D%7C\

]

 

https://my.usgs.gov/confluence/label/cdi/333333%7Dcdi%7Bcolor%7D%7C\

]

 

https://my.usgs.gov/confluence/label/cdi/333333%7Dcdi%7Bcolor%7D%7C\

]

Developer/SME

Robert Djursaj (Contractor)

Matt Tricomi (Contractor)

 

 

 

 

Developer Location

Denver Bldg 810

Denver Bldg 810

 

 

 

 


As captured from original Scope of Deliverables, Roles, Tasks (EXCEL) reviewed on 4/17 (Sent out originally Feb 2012)

Milestones

Milestone

Geospatial - Save As Framework

Architecture, Design, Coordination

Open in OpenLayers in CIDA Portal

Save to ArcMap (10 days)

Open In OpenLayers Framework (Read Px3 JSON, ESRI JSON

Open in Flex (Convert Px3 JSON to ESRI JSON)

May-12

Security Decided as JAVA app for opening JSON cross-domain ;

Tech Design Review Rounds completed with NGTOC, NEIC and Openlayers group

Examine and Ramp-up on Px3 JSON Context and GeoServices JSONMeetup with Aguinaldo/ESCG to coordinate scope (avoid overlaps)

Wrote Px3 JSON string to convert into openlyaers object

Review the GeoServices_Context_Example.docx (From Standard Context Wiki page)Get familiar with GeoServices JSON

Draft Tight Requirements Doc for OpenGeoMeetup with Ivan/CIDAExamine Complex Base Layer concept

Rob to meetup with Gary to plan out this task

Jun-12
June 26th, 2012
Meeting Notes

Refactoring original SaveAs Service

First Demo of SaveAs Web Service
Testing SaveAs with OpenLayers calls

ProviDe Px3 code to Ivan/CIDA to compare code (Ivan is not rehosting or sharing code)

 

1. Completing non hardcoded Display of Px3 context including reviewing Px3 Code

2. Review re-using code for within the SaveAs mapper object working with NGTOC

Demo Px3 Content visualized in OpenLayers

 

See CIDA

 

Jul-12
July 31st, 2012

Work on Save of KML and ESRI (ESRI 1st)

 

3. Help testing SaveAs with providing JSON String to SaveAs function to test calls from other apps (Possibly in June as Openlayers is ready)

ESRI UC Meeting Data Gathering

See CIDA

ESRI UC Meeting Data Gathering

Aug-12

Next Mtg TBD by Rob D - Prepping for 9/5 Presentation

Solve ESRI JSON Context Standard to go with 1.0,5,7

Work on KML, but 2nd priority

OGC Debrief/Planning for FY13 with NGP Standards Architect

FY13 Scope planning with EA & Delivery lead

Work completed at this point (hours used up). Have made list of FY13 potential options

Experiment with ArcPy examples against TNM 1.5 example of ESRI JSON context

Help NGTOC determine standard version direction

See CIDA

Verify ESRI UC outputs

Help NGTOC determine standard version direction

Sep-12

 

9/5 Demo

Demo Save Px3 and ESRI (Maybe KML)

Demo Reading saving a user session over HTTP

Demo architecture goal

Demo Visualization of reading Px3 JSON

Demo ArcPy script reading ESRI JSON Context

See CIDA

Demo reading ESRI JSON Context in 2 or 3 ESRI Web APis

 

Additional Criteria

See Last Year Project wiki page https://my.usgs.gov/confluence/display/cdi/Web+Application+Integration+Framework

In kind funding and work leveraged

1 Travel $ possible
2 Partners after reviewing the concept may want development $ (i.e. Developer time salary contributions for partners like EGSC, WIM, CIDA time)
3 External ad-hoc ESRI talent reachback as needed or help broker the NGA Palanterra x3 and ArcGIS.com discusions as Px3 moves to the ESRI Context (Batten)

Interaction with DMWG

Ad-hoc - Likely more with sciencebase.gov (Kern, Bristol, Mack) on how we save the mashup contexts for discovery

Deliverable and its Measurable Benefit

Scope of Deliverables, Roles, Tasks
Here are the data flow and scope requirements :

Open In Context Reader in OpenLayers - By EGSC and CIDA
Open In Context Reader in ArcMap - By ESRI for ArcGIS 10.1
Open In Context Reader in KML - By NGP
Open In Context Reader in GoogleMap or BingMaps - Possible by DHS or EPA
Benefits and Reasons - https://my.usgs.gov/confluence/download/attachments/63832506/Saving+and+Opening+Mashups.pptx
Examples

Methodology (process)

Vet the TNM Concept

Meet with each partner, and get input, and clean up language, and concept
Here are the data flow and scope requirements :

Discover issues for expanding Explore JSON Context Scope

Explore Standardization Process

How we try to see if OGC or a group can come to a common standard?
Can we?
If not, what do we do USGS Wide?
Here is a sample thread we've had on this topic: https://my.usgs.gov/confluence/display/cdi/Web+Service+Contexts+-+Standards+and+Options

Explore how have 1 Save As/Open In App service that USGS can use leveraging TNM efforts

Do it

After understanding the standard, review the requirements, and agree to application interfaces, establishing communication internally and exernal plans, CODE!
This would also include agile demos with partners, talking about requirements with users, etc. (Fuller)

Partnerships

Scope of Deliverables, Roles, Tasks
140 days of labor .5 to .75 FTE for CDI Funding across LCAT (30), WIM(30), CIDA(10), and NGP (10)
(NGP has an additional 120 days of work to push to live and will be covering addiional 120 days as part of NGP budget)
Geospatial - Save As Framework (2PDF, 2JSON, 2KML, 2PNG, 2JPG, etc.) (90 days, but paid in NGP)
Robert Djurasaj (David Hughes)
Open In OpenLayers (10 days)
Water CIDA - Dave Blodgett (Nate Booth), Center for Integrated Data Analytics
Save to ArcMap (10 days)
Hazards - Greg Smoczyk
Open in OpenLayers (30 days)
Land Cover Analysis - Eastern Geographic Science Center, Land Cover Analysis Tool, Cassandra Ladino, John Aguinaldo
Open in Flex (30 days)
Water Information Management - Gary Latzke, Jon Baier, Nick Estes
Context Standards (60 days)
Doug Nebert, FGDC, OGC POC
Glen Geumpel, NGP
Communicating with User Partners - validating approach (15 days, but paid in NGP)
Tracy Fuller
JSON Context Reader in JS API (Ongoing by NGA)
NGA Palanterra X3
ESRI JSON Context Support (As needed, some meetings here and there, no cost)
ESRI Larry Batten, Jeremy Bartley, Charlie Frye
Consider BING player externally (TBD)
Rob to see if DHS (Booth) or EPA (Jerry) interested

  • No labels