Confluence Retirement

In an effort to consolidate USGS hosted Wikis, myUSGS’ Confluence service is scheduled for retirement on January 27th, 2023. The official USGS Wiki and collaboration space is now SharePoint. Please migrate existing spaces and content to the SharePoint platform and remove it from Confluence at your earliest convenience. If you need any additional information or have any concerns about this change, please contact myusgs@usgs.gov. Thank you for your prompt attention to this matter.
Skip to end of metadata
Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 69 Next »

Please note

The steps below are for a non-administrative install.

If you want to install and maintain Anaconda for all users, please go to this version of the instructions. (Admin access required)

What is the Anaconda distribution?

Anaconda is an open-source Python distribution that makes is possible to easily install and manage many pre-packaged third party Python modules.

It has some big advantages over using the Python Esri provides with ArcGIS:

  • Very useful tools not in Esri's standard distribution are available (IPython, Jupyter notebook, Spyder)
  • You can install and update the Python stack as a regular (non-pr) user
  • You can install and remove, and update third party packages easily, with Anaconda checking for compatibility for you 
  • Manage multiple environments (for example, one compatible with ArcMap, another compatible with ArcGIS Pro)

Workflow

The general workflow to make this happen is to:

  • Install Anaconda without fouling the Windows environment (paths, registry) to break Esri's python stack
  • Configure Anaconda with the particular add-ons you want, and
  • Configure ArcGIS's Python so that it is aware of the Anaconda environment (so you can import modules not in Esri's stack)

1) Download and Install Anaconda

The main distribution is pretty large. MiniConda downloads what you need to do to start. Choose the Miniconda that matches the main software you want to integrate with. (You can install both if you want.)

Run the .exe installers

Select install for a single user (Not "All Users")

  1. Install to a folder where there is going to be plenty of space (recommend the D drive, not the C drive)
  2. IMPORTANT: To avoid breaking ArcGIS, uncheck the checkboxes  (a) make Anaconda the default Python and (b) add Anaconda's Python to the PATH. 
  3. Here are screenshots to help guide you through the install process.

Anaconda first run

Open the Anaconda prompt window (Search in Windows and start the shortcut "Anaconda Prompt") and enter "python"

First Anaconda Session
(D:\Users\cprice\Miniconda2) C:\Users\cprice>python
Python 2.7.13 |Continuum Analytics, Inc.| (default, May 11 2017, 14:07:41) [MSC v.1500 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
Anaconda is brought to you by Continuum Analytics.
Please check out: http://continuum.io/thanks and https://anaconda.org
>>>

At this point, you  have a nice Anaconda Python setup (~ 225 MB) that's spiffy and new and totally useless with ArcMap.

2) Set up an SSL certificate for Conda

The Department of Interior is now requiring SSL encryption. This means you cannot download Anaconda packages without an SSL certificate in place.

  1. Download the DOI certificate:  https://internal.usgs.gov/oei/wp-content/itsec/DOIRootCA2.cer to your Downloads folder
  2. Open "Anaconda Prompt" window
  3. Set up the certificate to use it with Conda
mkdir %USERPROFILE%\.certificates
copy %USERPROFILE%\Downloads\DOIRootCA.crt %USERPROFILE%\.certificates
conda config --set ssl_verify %USERPROFILE%\.certificates\DOIRootCA.crt

3) Configure Anaconda To Work with ArcGIS

The following workflow will demonstrate how set up a custom Python environment within Anaconda that has the same modules installed as ArcGIS Python, and then add a compatible version of Spyder and Jupyter.

The following example is for ArcGIS 10.4.1, assuming you have successfully installed MiniConda 2 32-bit (as above).

A. Find the versions of numpy and matplotlib for your version of ArcGIS Desktop

The critical modules for ArcGIS compatibility can be determined from your Desktop or Pro Python command line.

(Or you can just look at the list below under B.)

# pyversions.py - report critical python stack for use with ArcGIS
# example output:
# ArcGIS install folder: C:\ArcGIS\Desktop10.4
# sys.executable: C:\Python27\ArcGISx6410.4\pythonw.exe
# matplotlib: 1.4.3
# numpy: 1.9.2
# scipy: 0.15.1
import sys
import os
ff = "{}: {}"
try:
    print("ArcGIS install folder: {}".format(os.environ["AGSDESKTOPJAVA"][:-1]))
    print(ff.format("sys.executable", sys.executable))
    import matplotlib
    print(ff.format("matplotlib", matplotlib.__version__))    
    import numpy
    print(ff.format("numpy", numpy.__version__))
    import scipy
    print(ff.format("scipy", scipy.__version__))
except:
    pass

B. Create an Anaconda environment for use with ArcGIS

We'll also include a few other modules that we know are shipped in the ArcGIS Python stack.

  1. Open the Anaconda prompt (32 or 64, depending on the ArcGIS environment you are integrating with)
  2. Create a compatible environment
    1. 32-bit (ArcMap, ArcCatalog)
      ArcGIS 10.2.2:  "conda create -n arc1022 python=2.7.5 numpy=1.7.1 matplotlib=1.3.0 pyparsing xlrd xlwt"
      ArcGIS 10.3.1:  "conda create -n arc1031 python=2.7.8 numpy=1.7.1 matplotlib=1.3.0 pyparsing xlrd xlwt"
      ArcGIS 10.4.1: "conda create -n arc1041 python=2.7.10 numpy=1.9.2 matplotlib=1.4.3 scipy=0.16.0 pandas pyparsing xlrd xlwt"
      ArcGIS 10.5: "conda create -n arc105 python=2.7.12 numpy=1.9.2 matplotlib=1.4.3  scipy=0.17.0 pandas pyparsing xlrd xlwt"
    2. 64-bit (Background Geoprocessing (x64), ArcGIS Pro
      ArcGIS 10.x x64 background processing: (same as above, from 64 bit prompt, for example:
          "conda create -n arc105x64 python=2.7.12 numpy=1.9.2 matplotlib=1.4.3  scipy=0.17.0 pandas pyparsing xlrd xlwt"
      ArcGIS Pro 1.2: "conda create -n arcpro12 python=3.4.3 numpy=1.9.3 matplotlib=1.4.3 scipy=0.16.0 pandas pyparsing xlrd xlwt"
      ArcGIS Pro 1.3: "conda create -n arcpro13 python=3.4.4 numpy=1.10 matplotlib=1.4.3 scipy=0.16.1 pandas pyparsing xlrd xlwt"
      ArcGIS Pro 2.0: "conda create -n arcpro13 python=3.4.4 numpy=1.10 matplotlib=1.4.3 scipy=0.16.1 pandas pyparsing xlrd xlwt"
  3. The conda tool will:
    • Determine that the specified package versions are compatible with each other.
    • Find any packages on which these depend and determine most recent versions that are compatible with what you've asked for, if not completely specified.
    • Show you what it plans to do and prompts you to continue.
    • Download (only) the packages you need to "overlay" onto the base environment to get the environment you have specified.
    • Set up a environment subdirectory, (D:\Users\cprice\Miniconda2\envs\arc1041 in our example) installing the downloaded packages into it. This adds up to about 1.5 GB in our ArcGIS 10.4.1 example.

C. Test the virtual environment

Open an Anaconda command window and load the virtual environment.

 

C:\Users\cprice> conda info --envs
# conda environments:
#
arc1041                  D:\Users\cprice\Miniconda2\envs\arc1041
root                  *  D:\Users\cprice\Miniconda2
D:\Users\cprice>activate arc1041
Activating environment "arc1041"...
[arc1022] D:\Users\cprice> conda list
# packages in environment at D:\Users\cprice\Miniconda2\envs\arc1041:
#
dateutil                  2.4.1                    py27_0
matplotlib                1.3.0                np17py27_0
numpy                     1.7.1                    py27_3
...

D. Add more packages

You can add more packages using conda install, but make sure you specify version numbers for these that won't change the Python modules required to stay compatible with ArcGIS's Python stack.

Let's add the Jupyter notebook and the Spyder IDE which are both popular additions to your toolbox.

Specifying the list of module versions (ARCLIST)  here ensures the environment will still work with ArcGIS 10.4.1.

set ARCLIST=python=2.7.10 numpy=1.9.2 matplotlib=1.4.3 scipy=0.16.0 pandas pyparsing xlrd xlwt
conda install -n arc1041 %ARCLIST% jupyter spyder
...
The following NEW packages will be INSTALLED:
... 

You can search for more packages available in conda with conda search.

You're not limited to adding only packages to which conda has access to your new environment. Here's an example on how to install a commonly desired GIS-related package (shapely) into a conda virtual environment using the python pip utility:

http://deparkes.co.uk/2015/01/29/install-shapely-on-anaconda/

3) Configure ArcGIS python to see Anaconda environment and vice versa

This can most easily be done (personal opinion) with a Python usercustomize.py startup script

  • Install Anaconda, setup environment to match your ArcGIS version
  • Download and edit the installation paths in this script (usercustomize.py.txt) to match your setup
Edit block in usercustomize.txt
###########################################
# Edit here match your setup
# These paths must match your Anaconda setup exactly.

# Anaconda home folders
conda_arcmap_home = r"D:\Users\cprice\Miniconda2"
conda_arcmap64_home =  r"D:\Users\cprice\Miniconda3x64"
conda_arcpro_home = r"D:\Users\cprice\Miniconda3x64"

# anaconda environments set up to match Desktop and Pro
conda_arcmap_env = "arc1041"
conda_arcmap64_env = "arc1041x64"
conda_arcpro_env = "arcpro13"

# Pro path must be included here - registry read not working
pro_path = r"C:\ArcGIS\Pro" + "\\"
pro_exe = os.path.join(pro_path, "bin\\ArcGISPro.exe")

# change to false after testing done
debug = True

# do not edit below this line
###########################################
  • IMPORTANT - Test and debug this script by running it from python.exe for each of your environments. (See examples in the script header.)
  • Place the script in your user site packages folder with the name usercustomize.py
    The user site-packages folder path can be found with:  python -m site --user-site
    Usually:  C:\Users\username\AppData\Roaming\Python\Python27\site-packages (ArcGIS Desktop)
           C:\Users\username\AppData\Roaming\Python\Python34\site-packages (ArcGIS Pro, Python35 for Pro 2.0)

Testing and Troubleshooting

ArcGIS

  1. Start Desktop or Pro, open the Python window
  2. print ("\n".join(sys.path))  -- you should see the Anaconda site-packages near the end of the list

Anaconda

  1. Start Anaconda 32-bit or 64-bit  command prompt
  2. activate arc1041  (or whichever environment in the list)
  3. import arcpy
  4. print ("\n".join(sys.path)) -- you should see the ArcGIS site-packages near the end of the list

References

http://www.continuum.io/blog/conda

How to do a separate Python installation with ArcGIS? (GIS Stack Exchange)

Using ArcPy with Anaconda (PyMorton)

  • No labels