Presentation @ GeoPython 2016 - Arcpy Overview

1. ArcPy Overview – Extending and
Automating ArcGIS with Python
Clinton Dow – Geoprocessing Product Engineer @ Esri
http://www.slideshare.net/ClintonDow/arcpy-overview-slides

2. Before we begin…
Free stuff!
• Free ArcGIS Pro
- http://bit.ly/ArcPyProTrial
• Free ArcGIS Online Account
- http://bit.ly/PythonAGOLTrial
• ArcPy Documentation
- http://bit.ly/ArcPyDocs
• ArcPy Resources
- http://bit.ly/ArcPyResources

3. About MeClinton Dow – Calgary, AB > Los Angeles, CA
• Product Engineer @ Esri
- ArcPy
- Conda
- Charts and Graphs
• GIS Developer @ Matrix Solutions Inc.
- GeoProcessing Tool Development
- Oil & Gas Applications
- Environmental Applications
- Civil Engineering Projects

4. esri
Environmental Systems Research Institute, Inc.
• Founded in 1969
• HQ in Redlands California
- 41 Offices Globally
- Employees from 67 Countries
• Supports Government, Military
and Civilian organizations
• Currently hiring Experienced Python Developers/Product
Engineers 
- http://bit.ly/ArcPyJobs

5. ArcGIS
Product Information
• Industry-leading Commercial GIS Software Suite
• ArcGIS for Desktop
- Initial release in 1999
- Current version of ArcMap 10.4
- ArcGIS Pro included since 10.3 – January 2015
- Current version 1.2
• ArcGIS for Server
- Current version 10.4
- Portal for ArcGIS included since 10.2
• ArcGIS Online

6. ArcPy
ArcGIS Python site-package
• Introduced in ArcGIS 10.0
- June 29, 2010
• Successor to arcgisscripting module
• Create tools leveraging ArcGIS
- In-Application Geoprocessing Tools and Scripting
- REST Web Services
- Standalone Python Programs
• 1,000+ ArcPy-based Tools Included in ArcGIS

7. Python at Esri
Other Uses of Python
• Test Framework
- Proprietary test-runner and reporting
• Queries in ArcGIS
- Tools not based on ArcPY
- Calculate Field Tool
- Map Label Expressions
• CityEngine
- Python Scripting
- Built-in script editor

8. Agenda
• Python in ArcGIS: Moving Forward
- Python 2 vs. 3
- ArcGIS 10.x vs ArcGIS Pro
• Tip #1 –ArcPy in an IDE
• Automating ArcGIS with Python
- Scripting Complex Workflows
- Service Endpoint Automation

9. Agenda
• Tip #2 – There must be a better way!
• Extending ArcGIS with Python
- Python Standard Library
- Tour of Complementary Python Packages
• Tip #3 – Comprehensions with Cursors
• The future of Python in ArcGIS
- Managing Packages and Environments with Conda
- Jupyter Notebook

10. ArcPy: Moving Forward
‘Python is the language of GIS.’ – Bill Moreland, Esri Dev Summit 2016
• Python 2 vs. 3 in ArcGIS
- Python 2 End of Support Countdown Clock!
- ArcGIS 10.4  Python 2.7.10
- ArcGIS for Server  Python 2.7.10
- ArcGIS Pro 1.2  Python 3.4.3
- ArcGIS Pro 1.3+  Decoupled Python!

11. ArcPy: Moving Forward
‘Python is the language of GIS.’ – Bill Moreland, Esri Dev Summit 2016
•ArcGIS 10.x to ArcGIS Pro
- ArcObjects SDK will not move to Pro
- Great time to be a Python GIS Developer!
- ArcObjects high cost of maintenance and very tightly
coupled to COM
- Rewrite code in Python 3 in Pro
- Platform agnostic vs. .NET code
- Web Services
- Vast library of Packages
- Improved features in Pro
- … it’s Python and not .NET 

12. Tip #1 – ArcPy in an IDE
GIS from the comfort of your development environment
• Integrated Development Environments
- Debuggers
- Test Frameworks
- Coverage
- PyLint/pycodestyle
- Version Control Integration
- Refactoring Tools
- Virtual Environment Support
- AutoComplete

13. Tip #1 – ArcPy in an IDE
GIS from the comfort of your development environment
• Recommended IDEs
- PyCharm
- Python Tools for Visual
Studio
- Spyder
- Eclipse with PyDev
- Wingware

14. Automating ArcGIS with Python
Toolboxes, Python Toolboxes, Interpreters, Command Line, Services…
• Multiple ways to create tools
- ArcGIS Toolbox (.tbx)
- Python Toolbox (.pyt)
- Stand-alone Scripts (.py)
- Interactive Python Window
• Decouple logic code from Toolbox
- Testing!!
• Distutils to create packaged code
Sharing
Install into tools as needed

15. Automating ArcGIS with Python
Modularized Code Demo

16. Automating ArcGIS with Python
Distutils demo

17. Extending ArcGIS with Python Modules
Python is a ‘Batteries Included’ Programming Language
• Python Standard Library
- Distutils
- Collections
- Itertools
- XML
- JSON
- UnitTest
- -asyncio

18. Tip #2 – There Must Be A Better Way!
Never Underestimate The Python Standard Library!
• Raymond Hettinger
- Transforming Code Into Beautiful, Idiomatic Python
- https://www.youtube.com/watch?v=OSGv2VnC0go
- Beyond PEP 8 – Best Practices for Beautiful, Intelligible Code
- https://www.youtube.com/watch?v=wf-BqAjZb8M

19. Extending ArcGIS with Python Modules
Utilize the Extended Python Ecosystem in your ArcGIS Tools
• Scientific
- Jupyter
- IPython
- NumPy
- SciPy
- Pandas
- PySAL
- SciKit Learn
• Utility
- Requests
- Arrow
- Xlrd/xlwt
- Chardet
• Visualization
- Bokeh
- Plotly
• Web-Scraping
- BeautifulSoup
- Scrapy
- Tweepy

20. Tip #3 – Filter & Map with Comprehensions
Using Pure Python Data Types to Work with Feature Classes
import arcpy
data_path = r"C:Pathtofeatureclass"
def cursor_comprehension_map(path):
# Map function
# m = map(lambda x: x[0] + 1.0, [row for row in arcpy.da.SearchCursor(path, ['OID@'])])
m = [row[0] + 1 for row in arcpy.da.SearchCursor(path, ['OID@'])]
return m
def cursor_comprehension_filter(path):
# Filter function
# f = filter(lambda x: x ==1, [row for row in arcpy.da.SearchCursor(path, ['OID@'])])
f = [row for row in arcpy.da.SearchCursor(path, ['OID@']) if row[0] == 1]
return f

21. Tip #3 – Comprehensions with Cursors
Using Pure Python Data Types to Work with Feature Classes
import arcpy
data_path = r"C:Pathtofeatureclass"
def cursor_list_comprehension(path):
l = [row for row in arcpy.da.SearchCursor(path, ["SHAPE@", "Language"])]
return l
def cursor_dictionary_comprehension(path):
d = {row[0]: row[1:] for row in arcpy.da.SearchCursor(path, ["OID@", "MsgText", "Language"])}
return d
def cursor_generator_comprehension(path):
g = (row for row in arcpy.da.SearchCursor(path, ["SHAPE@", "Language"]))
return g
def cursor_generator_statement(path):
with arcpy.da.SearchCursor(path, ["OID@", "MsgText"]) as cursor:
for row in cursor:
yield str(row[0])

22. Tip #3 – Collections Example
Using Pure Python Data Types to Work with Feature Classes
from collections import Counter
import arcpy
data_path = r"C:Usersclin8331DesktopTemptw.gdbtw_lang_europe"
def count_field_occurances(data_path):
c = Counter()
with arcpy.da.SearchCursor(data_path, ["OID@", "Language"]) as cursor:
for row in cursor:
c[row[1]] += 1
return c

23. The Future of Python in ArcGIS
Continuum Analytics’ conda VirtualEnv & Package Manager
• Conda integration into ArcGIS
- Advanced Python Package Management
- Curated channels of scientific Python packages
- Includes C and Fortran compiler
- Create and host your own conda packages
- Easily share your code with the GIS community
- Built-In Virtual Environment Support
- Manage multiple environments with access to ArcPy
- Easily share environments
- Trivializes dependency issues

24. The Future of Python in ArcGIS
Screenshots of conda UI

25. The Future of Python in ArcGIS
Jupyter Notebook/IPython Integration
• Interactive Scientific Computing with ArcGIS Support
- Interact with ArcGIS for Server and ArcGIS Online via Jupyter Notebooks
- ArcPy Integration coming
- Embed maps with simple, pythonic API
- Access scalable, web-based GeoProcessing Services
- GeoEnrichment Services
- GeoAnalytics Servers
- All the regular functionality of notebooks
- Pandas/matplotlib magic methods
- Github integration
- Easily Sharable

26. Help us make ArcPy better!
• GeoNet
- http://geonet.esri.com
- https://geonet.esri.com/community/developers/gis-developers/python
• ArcGIS Ideas
- http://ideas.arcgis.com
• Conda Support Email
- conda@esri.com
• ArcPy Café
- https://arcpy.wordpress.com/

27. Conclusion
• ArcGIS
- Industry-leading GIS Platform
- Big push to move towards Python
- Really big customers (NASA, ESA, Governments, Fortune 500 companies…)
- Lots of opportunity for Python programmers
• ArcPy
- Moving from Python 2 to 3
- Conda support
- Start in ArcGIS Pro if you’re learning
• We are hiring! Talk to me if you are interested in learning more.

28. Thank You!
Clinton Dow - Contact Info
cdow@esri.com
http://lnked.in/cdow
http://www.slideshare.net/ClintonDow/arcpy-overview-slides