Using non-ESRI python modules via Anaconda

(this page is under construction, feel free to add to it)

What is the Anaconda distribution?

Anaconda is an open-source Python distribution that makes is possible to easily install many pre-packaged third party Python modules. It is a competing alternative to the Enthought Python Distribution (EPD).

It has some big advantages over using Esri's distributions:

• Very useful add-ons not in Esri's standard distribution are available (iPython, pandas, etc).
• You can install and update Python as a regular user
• You can install and remove, and update third party packages easily (and without admin access)
• A single Python install stack can be used to set up different "virtual environments"

Workflow

The general workflow to make this happen is to:

• Install Anaconda without fouling the Windows environment (paths, registry) to break Esri's python stack
• Configure Anaconda with the particular add-ons you want, and
• Configure ArcGIS's Python so that it is aware of the appropriate Anaconda content.

The challenge here is that ESRI provides a very specific version of Python and libraries and you cannot easily get ArcGIS to use incompatible newer versions. This means that when exploiting non-ESRI provided modules that you gain via Anaconda, the Anaconda environment (including versions of standard library modules, for example, numpy) must be configured to be compatible with the ArcGIS "Python stack" for that release of the software (10.1, 10.2, 10.3, Pro).  

1) Install Anaconda

1. go to http://continuum.io/downloads
2. select for 32-bit python (because you should be compatible with the general/interactive version of python from ESRI)
3. IMPORTANT: when installing, you will be asked whether to make the Anaconda Python the default version of Python and whether to add this to the PATH environment variable. Decline both options (or the ESRI python will stop working).

At this point, you will have full Anaconda version of the python environment that's spiffy and new and totally useless with ArcMap. 

2) Configure Anaconda

The following workflow will demonstrate how to install the Pandas module for python as an example. Pandas is great for a whole variety of data processing tasks (just google "python pandas"). It is not a standalone module; it sits on top of numpy. A version of Pandas only works with a specific version of numpy. Since we don't want to mess with the version of numpy that comes with ESRI software, we need to match the version of Pandas that will be happy with the ESRI version of numpy. Generally, the ESRI numpy and the rest of the python environment is going to be older than what you might be able to pull from the internet, so you need to figure out which of the older versions of the module (Pandas, in this case) is a fit.

In this example, ArcGIS 10.2.0 is featured. This means that numpy version 1.6.1 is installed. This can be checked for your ArcMap Python window by typing:

import numpy

print(numpy.__version__)

As mentioned above, the installed version of Pandas must work with the ESRI numpy. It's a minor hassle to figure this out because there's no handy table (that I could find) listing this dependency for different Pandas versions. My totally inefficient strategy is to:

go to the Pandas web site

select the doc for a randomly selected version of Pandas (say, 0.10.0 for our example)

search that page for the word "depend" and follow the link

the dependency page will state the version of numpy needed (1.6.1 in our example version of ArcGIS 10.2.0)

Once the version of pandas that is compatible with the ESRI numpy is known, you can configure Anaconda accordingly. By default, Anaconda will come with the current stable release of Pandas, which will be too new to work with ESRI numpy. Rather than modifying the basic Anaconda installation, you can create a custom environment within Anaconda and specify the version of Pandas you want there. To do this:

1. Get to the Anaconda Command Prompt (via the Start menu on Windows, e.g.)
2. Type "conda create -n esri python=2.7.3 numpy=1.6.2 pandas=0.10.0"

This uses an Anaconda-provided piece of software, conda, to do some nice stuff for you. It creates a subdirectory beneath the Anaconda install area called "envs/esri" and puts the python modules of the version you ask for into it and allows you, within the Anaconda platform, to develop and run code in this new custom environment (instructions, information on that). To do so is different than accessing the extra module from ESRI python, but just so's you're aware of the possibility.

2) Configure ArcGIS

At this point, Anaconda has two versions of Pandas installed. The too-new default and the one you installed within the "esri" environment that should work with ArcGIS. ArcGIS, at this point, still has no way to understand that either version exists.  We hope to improve on the tactic described here in the near future but for now you will add a small ASCII file to a directory in the ESRI Python directories. This file, which specifies the path to the Anaconda "esri" environment, will be read whenever the ESRI Python interpreter is instantiated. This will enable ESRI python to find Pandas.

1. Navigate to the place ESRI python was been installed on your platform. On mine, it is "C:\Python27"
2. Navigate to "C:\Python27\ArcGIS10.2\Lib\site-packages"
3. Create and edit an ASCII file called "conda.pth" (note that the prefix can be anything, but the suffix must be .pth)
4. The file should specify the path to the Anaconda "esri" environment. Because I installed Anaconda at the C:\ prompt, the path for me is "C:\Anaconda\envs\esri\Lib\site-packages".
5. Save the file.

Testing and Troubleshooting

1. Start ArcMap
2. type "import pandas"
3. type "pandas."
   1. A popup menu with a list of pandas-provided functions is a pretty good sign the install succeeded.

If you get an error message when you try to import, the most common cause I've seen is that the version of the module is still too new. In the pandas example, version 0.10.1 also appears to use the same 1.6.1 version of numpy but for some mysterious ESRI-specific reason that we could not get support to solve or really talk about, pandas 0.10.1 does not work. The solution was to modify the Anaconda "esri" environment to hold 0.10.0 pandas and all worked. To do change this: 

1. Get to the Anaconda Command Prompt (via the Start menu on Windows, e.g.)
2. Type "conda install -n esri pandas=0.10.0"

References

How to do a separate Python installation with ArcGIS? (GIS Stack Exchange)

Using ArcPy with Anaconda (PyMorton)