Using non-ESRI python modules via Anaconda

conda_arc1022.txt

The challenge here is getting the ESRI-provided version of Python and associated module s to work with non-ESRI provided modules that you gain via Anaconda. Generally speaking, when you download and install Anaconda, you will get newer versions of Python-related material (including standard modules like numpy) than what is in your ArcGIS "Python Stack," which will not work. You therefore must configure Anaconda to be compatible with the version of ArcGIS "Python stack" (10.1, 10.2, 10.3, Pro) that you have installed.  The instructions here describe how to install older versions of modules into Anaconda that will work with ArcGIS. 

What is the Anaconda distribution?

Anaconda is an open-source Python distribution that makes is possible to easily install many pre-packaged third party Python modules. It is a competing alternative to the Enthought Python Distribution (EPD).

It has some big advantages over using Esri's distributions:

• Very useful add-ons not in Esri's standard distribution are available (iPython, pandas, etc).
• You can install and update Python as a regular user
• You can install and remove, and update third party packages easily (and without admin access)
• A single Python install stack can be used to set up different "virtual environments" (including "ArcGIS-standard" ones)

Workflow

The general workflow to make this happen is to:

• Install Anaconda without fouling the Windows environment (paths, registry) to break Esri's python stack
• Configure Anaconda with the particular add-ons you want, and
• Configure ArcGIS's Python so that it is aware of the appropriate Anaconda content.

1) Install Anaconda (Single-User - no admin access needed)

1. Go to http://continuum.io/downloads
2. Download the 32-bit (x86) version of Anaconda (ArcMap [32-bit] can only directly work with 32-bit Python)
3. In the install dialog:
   1. Select install for a single user (Not "All Users")
   2. Install to a folder where there is going to be plenty of space (recommend the D drive, not the C drive)
   3. IMPORTANT: To avoid breaking ArcGIS, uncheck the checkboxes  (a) make Anaconda the default Python and (b) add Anaconda's Python to the PATH. 

At this point, you will have full Anaconda version of the python environment that's spiffy and new and totally useless with ArcMap. 

2) Configure Anaconda

The following workflow will demonstrate how to install the Pandas module for python as an example. Pandas is great for a whole variety of data processing tasks (just google "python pandas"). It is not a standalone module; it sits on top of numpy. A version of Pandas only works with a specific version of numpy. Since we don't want to mess with the version of numpy that comes with ESRI software, we need to match the version of Pandas that will be happy with the ESRI version of numpy.

The following example is for ArcGIS 10.2.2.

A. Find the versions of numpy and matplotlib Arc is using.

Open ArcMap and its Python window, and enter these commands:

>>> import sys, numpy, matplotlib
>>> print(sys.version, numpy.__version__, matplotlib.__version__)
'2.7.5 (default, May 15 2013, 22:43:36) [MSC v.1500 32 bit (Intel)]', '1.7.1', '1.3.0')

B. Create an Anaconda virtual environment (including pandas) that is compatible with ArcGIS 10.2.2.

Rather than modifying the basic Anaconda installation, you can create a custom environment within Anaconda and specify the version of Pandas you want there. To do this:

1. Get to the Anaconda Command Prompt (Start > Search > Anaconda, pick "Anaconda Command Prompt")
2. Type "conda create -n arc1022 python=2.7.5 numpy=1.7.1 matplotlib=1.3.0 xlrd=0.9.2 xlwt=0.7.5 pyparsing=1.5  pandas" 
   or, better yet, use this file (conda_arc1022.txt) that has a more complete list
   "conda create --file conda_arc1022.txt" 
3. Anaconda's conda command will then:

• Determine that the specified versions are compatible
• Find most recent compatible versions of packages not completely specified 
• Prompts you to continue
• Downloads (only) the pieces you need to "overlay" with the base environment to get the environment you have specified.
• Set up a subdirectory in the Anaconda install folder "arc1022" and downloads these packages into it.

C. Test the virtual environment

D:\Users\cprice>activate arc1022

Activating environment "arc1022"...

[arc1022] D:\Users\cprice>conda list
# packages in environment at D:\Users\cprice\Anaconda\envs\arc1022:
#
dateutil 2.4.1 py27_0
matplotlib 1.3.0
np17py27_0numpy 1.7.1 py27_3
pandas 0.13.0 np17py27_0
...

[arc1022] D:\Users\cprice>python
Python 2.7.5 |Continuum Analytics, Inc.| (default, Jul 1 2013, 12:41:55) [MSC v.1500 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>>

3) Configure the conda environment to access ArcGIS

Copy the .pth file to your Anaconda environments site-packages folder so Anaconda can "see" arcpy

[arc1022] (D:\Users\cprice>( copy c:\Python27\ArcGIS10.2\lib\site-packages\desktop10.2.pth
  D:\Users\cprice\Anaconda\envs\arc1022\lib\site-packages )
[arc1022] D:\Users\cprice>python

Python 2.7.5 |Continuum Analytics, Inc.| (default, Jul 1 2013, 12:41:55) [MSC v.1500 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import arcpy
>>> import pandas
>>> 

4) Configure ArcGIS to find conda packages

1. If you have admin access:

   1. Navigate to "C:\Python27\ArcGIS10.2\Lib\site-packages"

2. If you don't
   1. Run this command: C:\Python27\ArcGIS10.2\python.exe -m site --user-site
   2. Navigate to the reported path (you may have to create the folder)
3. Create and edit an ASCII file called "conda.pth" This file will have a single line, containing the path to the environment site-packages folder:
   For example: D:\users\cprice\Anaconda\envs\arc1022\lib\site-packages
4. You should now be able to import pandas from the ArcMap command prompt and it will also be available to script tools.

Testing and Troubleshooting

1. Start ArcMap
2. type "import pandas"
3. type "pandas."
   1. A popup menu with a list of pandas-provided functions is a pretty good sign the install succeeded.

If you get an error message when you try to import, the most common cause I've seen is that the version of the module is still too new. In the pandas example, version 0.10.1 also appears to use the same 1.6.1 version of numpy but for some mysterious ESRI-specific reason that we could not get support to solve or really talk about, pandas 0.10.1 does not work. The solution was to modify the Anaconda "esri" environment to hold 0.10.0 pandas and all worked. To do change this: 

1. Get to the Anaconda Command Prompt (via the Start menu on Windows, e.g.)
2. Type "conda install -n esri pandas=0.10.0"

References

How to do a separate Python installation with ArcGIS? (GIS Stack Exchange)

Using ArcPy with Anaconda (PyMorton)