Confluence Retirement

In an effort to consolidate USGS hosted Wikis, myUSGS’ Confluence service is scheduled for retirement on January 27th, 2023. The official USGS Wiki and collaboration space is now SharePoint. Please migrate existing spaces and content to the SharePoint platform and remove it from Confluence at your earliest convenience. If you need any additional information or have any concerns about this change, please contact myusgs@usgs.gov. Thank you for your prompt attention to this matter.
USGS OEI/Enterprise GIS
Space shortcuts

USGS EGIS web pages
EGIS Home
EGIS Support
Metadata
Training
Email Lists
Esri Events

External web pages
Esri Support
Esri Product Help
Esri GeoNet
GIS Stack Exchange

EGIS wiki space (here)

Page tree

Versions Compared

Old Version 76

changes.mady.by.user Unknown User (cprice@usgs.gov)

Saved on

compared with

New Version Current

changes.mady.by.user Unknown User (cprice@usgs.gov)

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Comment: fix typo

...

1) Download and Install Anaconda

The main Anaconda distribution is pretty large . MiniConda downloads what you need to do to start. Choose the Miniconda that matches the main software you want to integrate with. (You can install both if you want.)

...

(> 3 GB). MiniConda is a smaller version that just includes the Python standard library to start.

You can download and install either one, depending on whether you want to spend the time and disk space. Miniconda most efficient, as you can always download down anything in the conda distibution you want later.

...

  1. Install to a folder where there is going to be plenty of space (recommend the D drive, not the C drive)
  2. IMPORTANT: To avoid breaking ArcGIS, uncheck the checkboxes  (a) make Anaconda the default Python and (b) add Anaconda's Python to the PATH. 
  3. Here are screenshots to help guide you through the install process.
  4. Windows 10: after installing, rename the shortcut Anaconda Prompt to "Anaconda Prompt 32-bit" or "64-bit"

Anaconda first run

Open the Anaconda prompt window (Search in Windows and start the shortcut "Anaconda Prompt") and enter "python" in the command window:

Code Block
languagetext
titleFirst Anaconda Session
(D:\Users\cpricejwpowell\Miniconda2) C:\Users\cprice>pythonjwpowell>python
Python 2.7.13 |Continuum Analytics, Inc.| (default, May 11 2017, 14:07:41) [MSC v.1500 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
Anaconda is brought to you by Continuum Analytics.
Please check out: http://continuum.io/thanks and https://anaconda.org
>>>

...

  1. Download the DOI certificate:  httpshttp://internalsslhelp.usgsdoi.govnet/oei/wp-content/itsec/docs/DOIRootCA2.cer to   to your Downloads folder (details on SSL setup)
  2. Open "Anaconda Prompt" window
  3. Set up the certificate to use it with Conda
Code Block
languagetext
titleWindows Command Prompt
mkdir %USERPROFILE%\.certificates
copy %USERPROFILE%\Downloads\DOIRootCADOIRootCA2.crtcer %USERPROFILE%\.certificates
conda config --set ssl_verify %USERPROFILE%\.certificates\DOIRootCADOIRootCA2.crtcer

Set up SSL certificate for PIP (optional)

If you plan to use the pip utility to download and install packages not included in the Conda distribution, set up a certificate for pip: 

Code Block
languagetext
titleWindows Command Prompt
mkdir %APPDATA%\pip
pushd %APPDATA%\pip
(echo [global] >& pip.ini
echo cert=%USERPROFILE%\.certificates\DOIRootCADOIRootCA2.certcer) >>> %APPDATA%\pip\pip.ini
popd

...

 
3) Configure Anaconda To Work with ArcGIS
...
(Or you can just look at the list below under B.)
 Code Block
languagepy
titlePython script for ArcGIS Python window
# pyversions.py - report ArcGIS Python modue versions
# Example output:
# C:\ArcGIS\Pro\bin\ArcGISPro.exe
# Python 3.5.2
# matplotlib 1.5.3
# numpy 1.11.2
# scipy  |Continuum Analytics, Inc.| (default, Jul  5 2016, 11:41:13) [MSC v.1900 64 bit (AMD64)]
# matplotlib 1.5.3
# numpy 1.11.2
# scipy 0.18.1
import sys
import os
ff = "{} {}"
try:
    print(sys.executable)
    print(ff.format("Python", sys.version.split("|")[0]))
    import matplotlib
    print(ff.format("matplotlib", matplotlib.__version__))
    import numpy
    print(ff.format("numpy", numpy.__version__))
    import scipy
    print(ff.format("scipy", scipy.__version__))
except:
    raisepass
B. Create an Anaconda environment for use with ArcGIS
...
  1. Open the Anaconda prompt (32 or 64, depending on the ArcGIS environment you are integrating with)
  2. Create a compatible environmentStartDesktop or Pro, open the Python window
    1. 32-bit 2.7 (ArcMap, ArcCatalog)
      ArcGIS 10.2.2:  "conda create -n arc1022 python=2.7.5 numpy=1.7.1 matplotlib=1.3.0 pyparsing xlrd xlwt "console_shortcut
      ArcGIS 10.3.1:  "conda create -n arc1031 python=2.7.8 numpy=1.7.1 matplotlib=1.3.0 pyparsing xlrd xlwt"xlwt console_shortcut
      ArcGIS 10.4.1:  "conda conda create -n arc1041 python=2.7.10 numpy=1.9.2 matplotlib=1.4.3 scipy=0.1516.0 pandas pyparsing xlrd xlwt " console_shortcut *NOTE* Esri shipped 0.15.0, but I needed to use 0.16.0 to have conda work
      ArcGIS 10.5: "conda create -n arc105 python=2.7.12 numpy=1.9.2 matplotlib=1.4.3  scipy=0.17.0 pandas pyparsing xlrd xlwt"64-bit (Background Geoprocessing (x64), ArcGIS Pro
      ArcGIS 10.x x64 background processing: (same as above, from 64 bit prompt, for example:
          " xlwt console_shortcut
      ArcGIS 10.5.1: conda create -n  arc105x64 arc1051 python=2.7.12 13 numpy=1.9.2 3 matplotlib=1.45.3 2  scipy=0.17.0 pandas pyparsing xlrd xlwt" xlwt console_shortcut
      ArcGIS Pro 1.210.6.1: "conda create -n  arcpro12 arc1061 python=32.47.3 14 numpy=1.9.3 matplotlib=1.45.3 scipy2  scipy=0.1617.0  pandas pandas pyparsing xlrd xlwt"
      ArcGIS Pro 1.3: " xlwt console_shortcut
    2. 64-bit 2.7 ArcGIS Desktop Background Geoprocessing
      Same as above for each version, from Anaconda 64-bit prompt, for example:
          conda create -n  arcpro13 arc105x64 python=32.47.4 12 numpy=1.10 9.2 matplotlib=1.4.3 scipy3  scipy=0.1617.1 0 pandas pyparsing xlrd xlwt" xlwt console_shortcut
    3. 64-bit Python 3 ArcGIS Pro (from Anaconda 64 bit prompt).
      ArcGIS Pro 1.2.0: "conda create -n  arcpro20 arcpro12 python=3.54.2 3 numpy=1.119.2 3 matplotlib=1.54.3 scipy=0.1816.1 pandas 0 pandas pyparsing xlrd  xlwt"
  3. The conda tool will:
    • Determine that the specified package versions are compatible with each other.
    • Find any packages on which these depend and determine most recent versions that are compatible with what you've asked for, if not completely specified.
    • Show you what it plans to do and prompts you to continue.
    • Download (only) the packages you need to "overlay" onto the base environment to get the environment you have specified.
    • Set up a environment subdirectory, (D:\Users\cprice\Miniconda2\envs\arc1041 in our example) installing the downloaded packages into it. This adds up to about 1.5 GB in our ArcGIS 10.4.1 example.
C. Test the virtual environment
Open an Anaconda command window and load the virtual environment.
 
 Code Block
languagetext
C:\Users\cprice> conda info --envs
# conda environments:
#
arc1041                  D:\Users\cprice\Miniconda2\envs\arc1041
root                  *  D:\Users\cprice\Miniconda2
D:\Users\cprice>activate arc1041
Activating environment "arc1041"...
[arc1022] D:\Users\cprice> conda list
# packages in environment at D:\Users\cprice\Miniconda2\envs\arc1041:
#
dateutil                  2.4.1                    py27_0
matplotlib                1.3.0                np17py27_0
numpy                     1.7.1                    py27_3
...
D. Add more packages
You can add more packages using conda install, but make sure you specify version numbers for these that won't change the Python modules required to stay compatible with ArcGIS's Python stack.
Let's add the Jupyter notebook and the Spyder IDE which are both popular additions to your toolbox.
Specifying the list of module versions (ARCLIST)  here ensures the environment will still work with ArcGIS 10.4.1.
 Code Block
languagetext
set ARCLIST=python=2.7.10 numpy=1.9.2 matplotlib=1.4.3 scipy=0.16.0 pandas pyparsing xlrd xlwt
conda install -n arc1041 %ARCLIST% jupyter spyder
...
The following NEW packages will be INSTALLED:
... 
You can search for more packages available in conda with conda search.
You're not limited to adding only packages to which conda has access to your new environment. Here's an example on how to install a commonly desired GIS-related package (shapely) into a conda virtual environment using the python pip utility:
http://deparkes.co.uk/2015/01/29/install-shapely-on-anaconda/
4) Configure ArcGIS python to see Anaconda environment and vice versa
This can most easily be done (personal opinion) with a Python usercustomize.py startup script
  • Install Anaconda, setup environment to match your ArcGIS version
  • Download and edit the installation paths in this script (usercustomize.py.txt) to match your setup
 Code Block
languagepy
titleEdit block in usercustomize.txt
###########################################
# Edit here match your setup
# These paths must match your Anaconda setup exactly.

# Anaconda home folders
conda_arcmap_home = r"D:\Users\cprice\Miniconda2"
conda_arcmap64_home =  r"D:\Users\cprice\Miniconda3x64"
conda_arcpro_home = r"D:\Users\cprice\Miniconda3x64"

# anaconda environments set up to match Desktop and Pro
conda_arcmap_env = "arc1041"
conda_arcmap64_env = "arc1041x64"
conda_arcpro_env = "arcpro13"

# Pro path must be included here - registry read not working
pro_path = r"C:\ArcGIS\Pro" + "\\"
pro_exe = os.path.join(pro_path, "bin\\ArcGISPro.exe")

# change to false after testing done
debug = True

# do not edit below this line
###########################################
  • IMPORTANT - Test and debug this script by running it from python.exe for each of your environments. (See examples in the script header.)
  • Place the script in your user site packages folder with the name usercustomize.py
    The user site-packages folder path can be found with:  python -m site --user-site
    Usually:  C:\Users\username\AppData\Roaming\Python\Python27\site-packages (ArcGIS Desktop)
           C:\Users\username\AppData\Roaming\Python\Python34\site-packages (ArcGIS Pro, Python35 for Pro 2.0)
Testing and Troubleshooting
ArcGIS
    1. xlwt console_shortcut
      ArcGIS Pro 1.3: conda create -n arcpro13 python=3.4.4 numpy=1.10 matplotlib=1.4.3 scipy=0.16.1 pandas pyparsing xlrd xlwt console_shortcut
      ArcGIS Pro 2.0: conda create -n arcpro20 python=3.5.2 numpy=1.11.2 matplotlib=1.5.3 scipy=0.18.1 pandas pyparsing xlrd xlwt console_shortcut
      ArcGIS Pro 2.2.1: conda create -n arcpro22 python=3.6.5 numpy=1.14.2 matplotlib=2.2.2 scipy=1.0.1 pandas pyparsing xlrd xlwt console_shortcut
  2. The conda tool will:
    • Determine that the specified package versions are compatible with each other.
    • Find any packages on which these depend and determine most recent versions that are compatible with what you've asked for, if not completely specified.
    • Show you what it plans to do and prompts you to continue.
    • Download (only) the packages you need to "overlay" onto the base environment to get the environment you have specified.
    • Set up a environment subdirectory, (D:\Users\jwpowell\Miniconda2\envs\arc1041 in our example) installing the downloaded packages into it. This adds up to about 1.5 GB in our ArcGIS 10.4.1 example.
    • The console_shortcut conda package will adds a shortcut to your Windows start menu that directly starts up your environment. (Note, if you remove the environment, you will have to delete it yourself.)
    • Windows 10 note: 
      Windows 10 won’t show a start menu item if the name of an item/shortcut within the new group matches a shortcut name in another group and has the same target  (e.g. both ‘Anaconda2 (32-bit)’ and ‘Anaconda2 (64-bit)’ have a shortcut named ‘Anaconda Prompt’ with a target of cmd.exe which confuses Windows 10). So, when Miniconda2 32-bit (and/or 64-bit) is installed edit the ‘Properties’ of ‘Anaconda Prompt’ -> General tab -> rename to ‘Anaconda Prompt 32-bit’. Also, rename each shortcut that is created for environments for 32-bit or 64-bit similarly for consistency. For example, if you created a new virtual environment in Anaconda 32-bit named arc1041 then you would rename the shortcut from ‘Anaconda Prompt (arc1041)’ to ‘Anaconda Prompt 32-bit (arc1041).
C. Test the virtual environment
Open an Anaconda command window and load the virtual environment.
 
 Code Block
languagetext
titleAnaconda command prompt
C:\Users\jwpowell> conda info --envs
# conda environments:
#
arc1041                  D:\Users\jwpowell\Miniconda2\envs\arc1041
root                  *  D:\Users\jwpowell\Miniconda2
D:\Users\jwpowell>activate arc1041
Activating environment "arc1041"...
[arc1022] D:\Users\jwpowell> conda list
# packages in environment at D:\Users\jwpowell\Miniconda2\envs\arc1041:
#
dateutil                  2.4.1                    py27_0
matplotlib                1.3.0                np17py27_0
numpy                     1.7.1                    py27_3
...
D. Add more packages
You can add more packages using conda install, but make sure you specify version numbers for these that won't change the Python modules required to stay compatible with ArcGIS's Python stack.
Let's add the Jupyter notebook and the Spyder IDE which are both popular additions to your toolbox.
Specifying the list of module versions (ARCLIST)  here ensures the environment will still work with ArcGIS 10.4.1.
 Code Block
languagetext
titleAnaconda command prompt
set ARCLIST=python=2.7.10 numpy=1.9.2 matplotlib=1.4.3 scipy=0.16.0 pandas pyparsing xlrd xlwt
conda install -n arc1041 %ARCLIST% jupyter spyder
...
The following NEW packages will be INSTALLED:
... 
Pinning dependencies
Conda does provide a method for pinning dependencies to an environment so you don't have to specify them each time (as: ARCLIST above). You need to list these packages in a filed called pinned in the conda-meta folder. 
 
 Code Block
languagetext
titleAnaconda command prompt
C:\Users\jwpowell> activate arc1041
(arc1041) C:\Users\jwpowell>(echo python ==2.7.10 & echo numpy ==1.9.2 & echo matplotlib ==1.4.3 & echo scipy ==0.16.0) > %CONDA_PREFIX%\conda-meta\pinned
(arc1041) C:\Users\jwpowell>type %CONDA_PREFIX%\conda-meta\pinned
python ==2.7.10
numpy ==1.9.2
matplotlib ==1.4.3
scipy ==0.16.0
 
Now that this has been set up, conda install -n arc1041 scipy-0.18.1 will generate an error, and conda install six will update six to the highest version compatible with the pinned modules.
Finding more packages
You can search for more packages available in conda with conda search. There's a nice list on the Anaconda website that describes them all.
You're not limited to adding only packages to which conda has access to your new environment. Here's an example on how to install a commonly desired GIS-related package (shapely) into a conda virtual environment using the python pip utility:
http://deparkes.co.uk/2015/01/29/install-shapely-on-anaconda/
4) Configure ArcGIS python to see Anaconda environment and vice versa
This can most easily be done (personal opinion) with a Python usercustomize.py startup script
  • Install Anaconda, setup environment to match your ArcGIS version
  • Download (and rename from .py.txt to .py) this script:  (usercustomize.py.txt) -- and edit it to match your setup:
 Code Block
languagepy
titleEdit block in usercustomize.txt
###########################################
# Edit here match your setup
# These paths must match your Anaconda setup exactly.

# Anaconda home folders
conda_arcmap_home = r"D:\Users\jwpowell\Miniconda2"
conda_arcmap64_home =  r"D:\Users\jwpowell\Miniconda3x64"
conda_arcpro_home = r"D:\Users\jwpowell\Miniconda3x64"

# anaconda environments set up to match Desktop and Pro
conda_arcmap_env = "arc1041"
conda_arcmap64_env = "arc1041x64"
conda_arcpro_env = "arcpro13"

# ArcGIS Pro install folder
default_pro_path = r"C:\ArcGIS\Pro"
 
# change to false after testing done
debug = True

# do not edit below this line
###########################################
  • IMPORTANT - Test and debug this script by running it from python.exe for each of your environments. (See examples in the script header.)
Create a user-specific site packages folder and place the script in the folder with the name usercustomize.py. 
The user site-packages folder path can be found with:  python -m site --user-site
Usually:  C:\Users\username\AppData\Roaming\Python\Python27\site-packages (ArcGIS Desktop)
       C:\Users\username\AppData\Roaming\Python\Python34\site-packages (ArcGIS Pro, Python35 for Pro 2.0)

 Code Block
languagetext
titleCommand prompt
:: Python 2.7
mkdir %APPDATA%\Python\Python27\site-packages
copy usercustomize.py %APPDATA%\Python\Python27\site-packages
:: Python 3.4
mkdir %APPDATA%\Python\Python34\site-packages
copy usercustomize.py %APPDATA%\Python\Python34\site-packages
 
 
Testing and Troubleshooting
ArcGIS
  1. StartDesktop or Pro, open the Python window
  2. print ("\n".join(sys.path))  -- you should see the Anaconda site-packages near the end of the list
Anaconda
  1. Start Anaconda 32-bit or 64-bit  command prompt
  2. activate arc1041  (or whichever environment in the list)
  3. python
  4. import arcpy
  5. print ("\n".join(sys.path))  -- you should see the AnacondaArcGIS site-packages near the end of the list
Anaconda
  1. Start Anaconda 32-bit or 64-bit  command prompt
  2. activate arc1041  (or whichever environment in the list)
  3. import arcpy
  4. print ("\n".join(sys.path)) -- you should see the ArcGIS site-packages near the end of the list
References
http://www.continuum.io/blog/conda
How to do a separate Python installation with ArcGIS? (GIS Stack Exchange)
...
References
https://www.continuum.io/
How to do a separate Python installation with ArcGIS? (GIS Stack Exchange)
Using ArcPy with Anaconda (PyMorton)
Acknowledgements
Many thanks to Parker Norton for his reviews and excellent additions to this document!
Other approaches
Justin Mayers has posted a batch script to install and set up set up MiniConda and environments to interact compatibly with Esri desktop software (ArcMap and Pro).
USGS BWTST supports an Anaconda3 SCCM package. This could be installed and linked to Pro using the information in this document.
ArcGIS Pro 2.x and later sets up an Anaconda install of its own. This environment could also be leveraged in various ways as time goes on.