Land Disturbance Editor ArcMap Add-In

One of my colleagues, TerraSystems Southwest, is assisting The Nature Conservancy in Tucson with the Sonoran Desert Disturbed Lands Analysis project. As part of their scope of work, they were asked to assess the level and type of land disturbance for over 100,000 1-square mile hexagons covering the Sonoran Desert.

As you might expect, editing the attributes of 100,000+ polygons is not a trivial task, and TerraSystems approached me to help find a way to make their editing task faster. The solution we came up with was a custom ArcMap add-in that would enable them to quickly edit the attributes for a selected polygon and move to the next feature for assessment.

The ArcGIS 10.0 and 10.1 add-in was developed in Visual Studio 2010 using esri ArcObjects, and contains a custom toolbar, tools, buttons, and dockable window forms. The Add-in leverages the ArcGIS add-in framework for simple deployment and updating.

TNC Custom Add-in Toolbar

The Add-in quickly edits the attributes of customizable fields to record the following information about each hexagon:

  • The disturbance amount (i.e. DistAmnt) of type Double
  • The disturbance type (i.e. DistType) of type String at least 2 characters wide. (The editor records an abbreviation of the disturbance type)
  • The order (priority) in which you want to edit the polygons (i.e. HexID), of type Integer
  • A flag (i.e. DistFlag) indicating that a polygon needs further review, of type Integer (stores a null, 0 or 1)

The Add-in allows users to identify the layer and fields in which to record these attributes, so it is not hard-coded to a particular layer or field name. In addition, the Add-in records the layer and field assignments to an XML file for reloading in future editing sessions, so users only need to configure the Add-in once.

TNC Custom Add-in Screen

Moreover, it was important to also ensure that the disturbance level and type attributes were not hard-coded, as it may be advantageous to add additional disturbance types in the future. In response, the Add-in was written to load both the disturbance levels and types from XML files that can be easily accessed from within the Add-in to make changes to the attributes the Add-in records for each feature.

XML Screenshot

The main strength of this Add-in is time savings during editing, users can select a feature and quickly edit multiple attributes using two different editing methods:

  • The Add-in dynamically* maps keys on their keyboard to enable power users to press a number key to set the feature’s level of disturbance, followed by a letter key to set the feature’s type of disturbance. Users can also press the space-bar to flag a feature as needing follow-up.
  • Alternatively, users can open the dockable Quick Edit window that contains dynamically* added buttons that represent disturbance levels and disturbance types. Users simply click one disturbance level button and another disturbance type button to set the feature’s attributes. Alternatively, users can click a button to flag the feature for follow-up.

TNC Custom Add-in Screen 2

* The buttons and keys are dynamically mapped when the Add-in loads. So if users edit the XML files to register new disturbance levels and types, the Add-in will dynamically map a new key and add new buttons to the QuickEdit Window.

Once both the feature’s disturbance level and type have been set (either by pressing custom mapped keys or using the Quick Edit Window), the Add-in will automatically* select the next feature in the edit series (using the field users selected as their Priority field when they configured the Add-in) and automatically* zoom to the next feature for inspection.

* Some times there are editing situations in which users prefer to not have the Add-in automatically select the next feature and/or zoom to the next feature. So both of these functions can be turned on or off in the Add-in’s configuration window.

Finally the Add-in contains help documentation showing users how to install, use and customize the Add-in.

TNC Custom Add-in Screen 3

By a conservative estimate, this Add-in shaves approximately 7 seconds off of each editing operation. Multiplied by the 100,000+ hexagons, the Add-in has saved nearly 200 hours on this project and countless hours on future projects that can leverage its customizable framework for future conservation assessment programs.

For more information about the Add-in please contact John Donoghue at: mail [at] johndonoghue [dot] net.

Recent Posts

Re-­engineering the Business Process of Desert Tortoise Data Collection

John Donoghue, Former Technology Manager at Ironwood Consulting

Ironwood Consulting Inc. (Ironwood) is a biological resources consulting firm in Southern California, and specializes in biological resource management, including desert tortoise permitting and mitigation. Ironwood is known for its successful work on large-­scale solar development projects in the Mojave Desert. Most recently, Ironwood was responsible for the pre-­project surveys, permitting, relocation, and monitoring of tortoises on four large-­scale solar energy projects: Desert Sunlight (3,000 acres), Silver State South (2,400 acres with 104 tortoises), Stateline (1,500 acres with 41 tortoises), and Dry Lake SEZ (serving three solar energy developer clients over a combined 2,000 acres with 45 tortoises). These projects combined required the successful translocation and monitoring of over 200 desert tortoises.

The problem
The large-­scale solar development projects Ironwood was managing required collecting a great deal of field data. During active desert tortoise field seasons, over 70 biologists would be collecting location, health, and other data on desert tortoises moving throughout project sites and offsite mitigation areas. This data was used to determine each tortoise’s territory, ascertain its health and identify other tortoises it may be interacting with. The information was necessary to help plan and ensure the successful translocation of tortoises from the project development site to nearby mitigation preserves, where tortoises would continue to be monitored to confirm they were adjusting to their new homes after the move.

Field biologists worked in remote locations that were often out of cellular voice and data access most of the time. Historically, field biologists collected data on paper forms that were later transcribed into Excel for analysis and reporting. Later, Ironwood adopted a system that used Pendragon Forms for offline field-­based digital data entry. This data was later synced through a server into a Microsoft Access database for storage and use. While the Pendragon Forms system accelerated the process of collecting and digitizing field data, biologists were aggravated by its antiquated interface and wanted forms that worked like the other iOS and Android applications they regularly used, with features that included scrolling pages, advanced data entry constraints on fields, photographs, GPS support and more.

Problems

  • Previous interface lacked modern features and user experience
  • Some data was entered into more than one form to record a single activity
  • Data entry errors were common
  • Data required time-­consuming and expensive manual QA/QC
  • Data available to system users was not relational and could not easily address questions asked by project managers
  • Projects hosted in separate databases with different schemas
  • Lack of standard database schema hampered efforts to gain efficiencies and prevented meta-­analyses across projects

The Pendragon/Microsoft Access system also required field biologists to enter some of the same data into different Pendragon forms, which resulted in many data entry errors and required a great deal of oversight and time-­consuming manual QA/QC. With Ironwood simultaneously managing multiple large-­scale solar projects, the time consuming manual processes that the Pendragon/Microsoft Access system required became very costly. Furthermore, the system wasn’t designed to support multiple end users, and the data products produced by the system were not designed to quickly answer the questions project managers were regularly asked by the solar development clients and agencies. Since each project was hosted in its own separate Access database with varying schemas to accommodate the peculiarities of each project, the absence of a standardized multi-­project database made analyses across projects impractical.

Developing a New Solution
To address these limitations, Ironwood decided to design an entirely new system that could support hosting multiple projects with multiple simultaneous users, while providing a greater variety of data products to address the different needs of field-­biologists, project managements, clients, and agencies. Ironwood began by meeting with their biologists in charge of desert tortoise management to review the shortcomings of the current system and discuss the envision the opportunities a new system could generate. The information from this and subsequent meetings resulted in a new database design that comprised a core set of tables and table fields that were designed to manage the data for multiple projects, and could accommodate all previously collected tortoise data, as well as incoming data for all upcoming desert tortoise projects. To ensure the database could support multiple simultaneous internal and external users working on different projects, the new database was implemented in Microsoft SQL Server. This also provided a foundation to support integrations with GIS other business systems.

iFormBuilder
In looking to replace Pendragon Forms, Ironwood reviewed a number of alternative iOS and Android applications for field-­based data entry. Ultimately, iFormBuilder was selected because it provided several benefits: there were iFormBuilder applications for both iOS and Android; the form building process in iFormBuilder was very intuitive, and the resulting forms had the modern user interface and user experience the field biologists desired. iFormBuilder forms also offered many features Ironwood needed, such as data entry constraints, data input masks, context sensitive field visibility, field validation, GPS support and in-form photos. Finally, the field biologist workflows were often best implemented as sub-­forms, and iFormBuilder sub-­forms were more intuitive to field biologists than the previous Pendragon Forms.

Integrating iFormBuilder and SQL Server
A fundamental goal for Ironwood’s new system was automating previously manual processes, and automating the transfer of data from iFormBuilder’ forms to Ironwood’s SQL Server database was a critical component. Fortunately, iFormBuilder offered an XML Post Data feature that pushes a copy of each record’s data from iFormBuilder to a user defined web page every time a new record is successfully uploaded to the iFormBuilder server.

Ironwood developed an ASP.net web application that contained a set of URL endpoints to receive XML data posted from iFormBuilder. The ASP.net application parsed the incoming XML data and inserted data as new records into Ironwood’s SQL Server database. During this process, the ASP.net application also performed QA/QC on incoming data, standardized data formats, and transformed the location information contained in the posted data into Open Geospatial Consortium (OCG) compliant spatial data for use in Ironwood’s ArcGIS system and other applications. Finally, the ASP.net application sent each user synching data to iFormBuilder an email that informed them of their successful data sync and identified any potential data errors found by the QA/QC processes.

Another goal of the new system was to re-­engineer the data collection business processes to remove duplicate data entry processes and reduce the number of digital forms needed. The previous Pendragon forms were designed to populate single database tables. Field biologists often had to enter some of the same data into different Pendragon forms as they performed a single activity, such as a health assessment – in which biologists completed one Pendragon form to record a tortoise location, and another Pendragon form to record tortoise health data.

In contrast, Ironwood designed its iFormBuilder forms around the activities biologists performed during their tortoise survey and monitoring work. As the data was received by Ironwood’s ASP.net application during syncing, the ASP.net application would automatically populate the appropriate tables. Therefore, a field biologist could use an activity specific iForm to record data for multiple associated activities, and the ASP.net application would sort the received data and populate the data into each relevant table.

Leveraging Better Data
To help provide more expedient and broader access to the data gathered in iFormBuilder, Ironwood also developed an ASP.net website that served as a central hub for field biologists, project managers, clients, and agencies. The resulting web portal supported multiple users and projects, and provided access to data as soon as it was synced from iFormBuilder and pushed into Ironwood’s SQL Server database. The web portal provided dashboards where users could view tortoise statistics and interactive maps of tortoise locations that were dynamically generated from the database using data from the most recent iFormBuilder application sync.

The web portal provided data in different formats for the various types of users the system supported. Field biologists and agencies accessed the portal to download dynamically generated Excel and GPX files that contained the most recently recorded tortoise locations and other information. Project Managers and Clients downloaded dynamically generated KML data for mapping in Google Earth, and viewed dynamic charts and statistics to review project’s status and look for potential issues. Unpublished URL endpoints allowed the data to be brought into ArcGIS Online, Google Earth and other systems.

In addition, by storing the location data as OGC compliant spatial points, the database could be directly accessed by GIS systems, and was persistently linked to Ironwood’s esri ArcGIS system where Python scripts automated daily cartographic quality maps used by field biologists, project managers, and clients.

Results
The better data input constraints provided in iFormBuilder significantly reduced Ironwood’s QA/QC needs. Despite having developed an automated backend QA/QC process, Ironwood later found that 95% of the data pushed into its database had no potential QA/QC issues identified by that process as the errors were constrained during data-­entry in iForms.

Field biologists were especially pleased with the new system. The iFormBuilder forms provided a modern user interface and user experience they expected, and iFormBuilder features such as designating the default device keyboard for fields, data input masks, and context sensitive field visibility, significantly increased the speed with which biologists could enter data into iForms, while being confident that data was accurate.

Results

  • Considerably fewer data errors
  • Increased adoption of digital forms
  • Greater trust in data
  • Much faster data turn-­around
  • Significantly lower data management costs
  • Improved client confidence in data management process
  • Increased client satisfaction
  • Improved agency cooperation
  • Stronger client relationships

The iFormBuilder XML Push workflow has greatly benefited field biologists by ensuring that the data the biologists recorded and synched was instantly added to Ironwood’s SQL Server database and biologists were immediately informed of its availability.

By developing a data portal website, Ironwood was able to leverage iFormBuilder and SQL Server to provide biologists, project managers, clients and agencies with access to all data anytime they needed it. Clients have praised the system and are using it regularly to keep informed on the progress of the mitigation programs for their projects. By making data collection processes transparent and providing clients with readily accessible data in the formats they needed, Ironwood was able to better communicate the data collection effort with its clients which reinforced the client’s trust in Ironwood’s data collection processes.

Though considerable effort was involved in reaching agreement with on a new database design and data input forms that bridged the gap between the needs of the field biologists and Ironwood’s end-­users, by implementing an integrated iFormBuilder, SQL Server and ASP.net solution, Ironwood was able to reengineer the business process of desert tortoise data collection. While the entire system took time to develop, the major benefits were realized when the system was fully implemented on multiple projects, where it essentially ran itself with very little intervention, resulting in significantly reduced data management and reporting costs, ultimately transforming a once cost-­intensive activity into a financial and practical benefit to Ironwood’s clients.

  1. Some Coveted Programming Skills Leave a reply
  2. Social Networks and Ecological Food Webs Leave a reply
  3. A Maxent Script Tool for ArcGIS 17 Replies
  4. Speedy Attribute Editing with a Custom ArcMap Add-In 2 Replies
  5. Landing a GIS Job and GIS Skills Development in 2013 39 Replies
  6. Les Miserables Look Down, Look Down the PhD Version 2 Replies
  7. Creating a Video of Vegetative Cover Sampling Leave a reply
  8. Desert Tortoise Training Leave a reply
  9. A Species Diversity Map for 88,000 New World Species Leave a reply