Skip to end of metadata
Go to start of metadata

It's September 2017 and this month's CDI challenge is HOW to get started with new tools for reproducibility and visualization. 

Have you heard of Jupyter notebooks and Shiny apps? More and more people are starting to use these tools, but how does one dive in and get started?

Maybe you know of some great resources for learning, or you're an expert and want to share some of your work for others to learn from.

To start, Rich Signell is going to kick off a CDI Jupyter notebook series at the October monthly meeting.

Take a look at the slides below, then let us know in the comments what type of information you'd like the CDI to help gather and present so we can learn together.

Links:

7 Comments

  1. Some questions from the monthly meeting that we are working on getting answers to:

    What server(s) in USGS are used to host shiny apps?

    Will shiny server be purchased and made available to USGS and WRET?

    Is code.usgs.gov a good place to share Jupyter Notebooks? (see answer below)

    Are there any training resources for pySB? (see answer below)

  2. Is code.usgs.gov a good place to share Jupyter Notebooks?

    This is certainly possible and is being done by other projects already; for example: https://code.usgs.gov/usgs/whcmsc-rdc

    Keep in mind that code.usgs.gov is a Git hosting solution for versioning files. I am fine with using code.usgs.gov for Jupyter notebooks for version control and software release requirements. I would not use it more generally as a file storage solution.
    (answer from Eric Martinez)
  3. Are there any training resources for pySB?

    (I think this question came up during the Scientist's Challenge and is somewhat related.)

    https://my.usgs.gov/bitbucket/projects/SBE/repos/pysb/browse
    a good link to share

    https://www.sciencebase.gov/about/content/use-python-add-new-items-and-files
    (just routes to PySB)

    general notes on API use of REST services in ScienceBase (PySB uses these)
    https://my.usgs.gov/confluence/display/sciencebase/CRUD+Operations+on+Items

    And a good link here on the API
    https://www.sciencebase.gov/about/content/add-items-sciencebase-using-rest-protocols#overlay-context=content/add-sciencebase-items

    (answer from Drew Ignizio)


  4. Related to the questions: 

    What server(s) in USGS are used to host shiny apps?

    Will shiny server be purchased and made available to USGS and WRET?

    From Laura DeCicco at w_r_admin@usgs.gov:

    We've had success, and recommend, putting shiny apps in a custom R
    package, and having users run the app via the package directly in R.

    The "wateRuse" package has some very clear directions for app-users here:
    https://github.com/USGS-R/wateRuse/blob/master/README.md
    Using those directions, many "non-R users" have successfully installed
    and run the waterUse Shiny app. As a side bonus, a few of those "non-R
    users" have turned into "R users".

    Packaging and distributing the shiny app in this manner greatly reduce
    the overhead required to set up a shiny server. While it's not
    impossible to set up a Shiny Server, it's not a trivial solution and
    would need the help and support of the Infrastructure Team.  We
    explored the option of using shinyapps.io, but decided not to renew
    our license a couple of years ago. One issue was scaling, in a handful
    of use-cases, the apps did not scale to 10's to 100's of users well.
    Another issue was that the service was not completely FedRamp
    approved. While our "cloud expert" said it would most-likely not be a
    problem, we didn't want to start recommending products that weren't
    approved. Another consideration is that once the app goes to a public
    URL, there's a whole set of website government regulations that
    developers need to consider (the USGS has many great web developers
    that deal with issues I would never even begin to imagine). We also
    didn't want to recommend having users create personal accounts to
    shinyapps.io and linking that out from government sites.

    None of these issues arise if the users run the apps via a custom R
    package, and actually performance is greatly improved. There is a bit
    of a learning curve to the Shiny App developer (since they also need
    to create an R package), and there's a bit of set up for the Shiny App
    user (since they might not have R installed). That being said, the
    model has been used successfully in a few instances, and I'd be happy
    to answer questions if someone had an app and they need help packaging
    it up.

    A good resource we've put together for developing packages is here:
    https://owi.usgs.gov/R/training-curriculum/r-package-dev/

    The key to the package is to make sure you declare all the other R
    packages that your shiny app depends on...then when users install your
    package, all the dependencies are automatically installed with it. If
    there are CDI users interested in producing an app in this manner, I'd
    be happy to answer questions.

    (There's a least 1 other package that has deployed a shiny app to a
    bunch of non-R users as well:
    https://github.com/USGS-R/toxEval
    there might be others I'm not aware of)

    Hope that helps!

    Laura

  5. I'm wondering if there has been any change in the status of USGS hosting shiny apps on one of our servers. For my particular application, which would target wildlife managers, I think even the relatively small hurdle of installing R, Rstudio and the package would dramatically limit its usage.  By having the app on the web the manager could first see if this is something that will be useful to them and then go to the trouble of running it locally for better performance. 

    1. Cross, Paul C. , thanks for your comment. There has been recent discussion about R Shiny status, with varying answers in depending who you talk to.  Let me see if I can track down the latest! 

      Copying Bristol, SkyDeCicco, Laura A.,  Andrews, Caitlin MarieWieferich, Daniel JosephLoftin, Keith A.Hutchison, Vivian B.  so they can see what happens here...

      1. Cross, Paul C. , I asked Office of Enterprise Information what was the status, and Paul Exter said that he would add it to the list of things needing approval. However I don't know the timeline or how we will learn the outcome of it.

        There are recent USGS projects working with Cloud Hosting Solutions who have used RShiny, so you might want to inquire there if that is relevant. cloudservices@usgs.gov