A Species Diversity Map for 88,000 New World Species

This is a small sample of a species diversity map that’s an end product of our models. This was created by overlaying 88,000+ shapefiles and represents the number of species potentially present in each cell. Red is low, dark blue and purple are high (over 8,000 species per cell).

Revisiting the North Etiwanda Preserve

Sage Scrub Chaparral

Sage Scrub Chaparral

When I studied population biology in junior college we conducted our field work in a large expanse of land along the alluvial fan that drained the San Gabriel Mountains to the Inland Empire valley below. Our specific field site was a 750+ acre area containing coastal sage scrub and coastal sage chaparral plant communities that were intersected by riparian corridors of alder, willow and occasionally sycamore trees.

Coastal sage scrub is a type of ecoregion that is located along the southern and central coast of California. It is important because is an endangered ecosystem that contains many unique species that occur nowhere else in the world (endemic), a number of which are endangered species. Coastal sage scrub is imperiled because it is also located on highly valued coastal real estate and threatened by human development.

Coastal sage scrub occurs from sea level to 1500 feet in elevation, along the coastal and inland valley foothills where coastal fog moderates the climate. Habitat contains sparse, low-growing soft, aromatic shrubs that range from dull-green to gray-green in color. In hot, dry summer months, shrubs often lose their leaves and become dormant as an adaptation to drought tolerance. They become green and vibrant again when winter rains arrive.

Some characteristics species in coastal sage scrub communities include:

Federally recognized threatened and endangered species include the California gnatcatcher (Polioptila californica), the San Diego banded gecko (Coleonyx variegatus abbottii ), the cactus wren (Campylorhyncus brunneicapillus), Merriam kangaroo rat (Dipodomys merriami), flannel-mouthed sucker (Catostomus latipinnis), western patch-nosed snake (Salvadora hexalepis), and cheese-weed moth lacewing (Chrysoperla spp.). Thirteen plant species of the coastal sage scrub are also recognized as threatened or endangered. (1)

A few years later the area began rapidly developing and our field area was threatened by a series of housing developments. Through the efforts of many people who fought to see the area protected to stem the decline of chaparral and coastal sage communities, a 762 acre section was set aside as a reserve. The area lies between the developed portions to the south and the national forest to the north and is protected on the southern end by a very wide power transportation corridor that serves as a buffer between the two land uses.

Welcome Sign

Welcome Sign

After several years of being away from this site, I recently revisited it yesterday afternoon and was surprised to see signs pointing the way to the preserve as I drove to the parking area. I was further surprised to see a dozen cars in the parking area at 3 pm on a Tuesday. When I last visited the site, it was protected, but still the completely undeveloped area that few people ever ventured into.

Trail Map

I quickly saw that the preserve has since been enhanced with formal trails, interpretive signs, and a few scattered picnic sites. It was now a popular place for hiking due to its cultural and recreational significance. Many people were taking a new trail that lead to a side canyon that contained a waterfall and shade beneath the alder trees. Others hiked around the lower portion of the preserve stopping at gazebos and interpretive signs.

While it was strange to see so many people in a place I used to walk around without seeing a soul, I was thrilled to see that the plant communities I fondly remembered were still there and appeared to be thriving. Walking through the scrub and chaparral I was transported back in time to my first population biology class, awestruck by the sight and smell of white sage and asking myself the same questions about the distributions of the plants I encountered along my hike.

Walking along one of the trails I saw massive shifts in the vegetation communities along a 1-mile length. Vegetation changed from stands that were dominated by white sage, to stands co-dominated by chemise and buckwheat containing little white sage. Walk a little further and the vegetation became co-dominated by black sage and sagebrush. Along this stretch I could not detect a difference in soil texture or aspect. So the sudden shifts of vegetation may the result of secondary successional processes where fire and floods disturb the vegetation and a different plant community begins growing in the areas that were disturbed. The overall pattern looked like waves of different vegetation along the hillside that was fascinating and worthy of more examination. Some of those communities are shown in the photos below.

Preserve Facing North West

Preserve Facing NW

White Sage

White Sage

White Sage Community

White Sage Community

Chemise and Buckwheat

Chemise and Buckwheat

Black Sage and Sagebrush

Black Sage and Sagebrush

Alder Riparian Woodland

Alder Riparian Woodland

Intersecting what I used to think was scorching hot chaparral (I’ve since lived in Arizona, so I have a new definition of hot) are wonderfully different riparian habitats containing cool, lush, shade tolerant species and – water. Water is a powerful force in these regions, winter rains and spring meltwater can move large amounts of water down the mountain through these canyons. Occasional fires remove vegetation and later rains bring floods that break trees and push large granite boulders downstream too.  These processes cause regular disturbances in the canyons and adjacent lands, and the vegetation is always in some state of recovery. The dense canopy of alder trees in the canyon bottoms is evidence of that those areas were previously disturbed, new trees grew to replace those that were killed, but self-thinning has not yet completed restructuring the tree stand.

Riparian Waterfall

Riparian Waterfall

Alder Woodland

Alder Woodland

The Southern California coastal sage scrub and chaparral communities are a wonderful place to explore with very interesting ecology that is worthy of protection and study. I’m happy that the North Etiwanda Preserve exists and look forward to continued visits.



For more information about the North Etiwanda Preserve see: http://sbcnep.org/

Photos of common coastal sage scrub and chaparral plants (from the nearby Cal State San Bernardino campus) can be seen at: http://biology.csusb.edu/PlantGuideFolder/

Finally, more information about coastal sage scrub and chaparral can be found at: http://www.eoearth.org/article/California_coastal_sage_and_chaparral

Happy exploring !

(1) From: http://www.eoearth.org/article/California_coastal_sage_and_chaparral

Night Blooming Cerus, Peniocereus greggii

The Night Blooming Cerus, or Queen of the Night, (Peniocereus greggii) is a Sonoran Desert cactus that most of the year would be virtually impossible to find. It’s main stem is a thin, dull-green to brown stalk that resembles the branch of shrubs like the palo verde, Ironwood, or creosote bushes that it normally grows beneath.

What makes this cactus special is that on certain summer nights the plants in each population produce large blooms synchronously – with some plants producing numerous flowers, some producing one and some none at all. The flowers open at sunset and last for one night, withering in the morning sun.

The flowers are produced only at night, are mostly white, about 3 inches wide and are strongly scented (though some people apparently cannot smell them). The flowers have long floral tubes and are pollinated by hawk moths (Sphingidae) which fly hundreds of yards between different plants in search of nectar. In return they exchange pollen between individuals.

The Night Blooming Cerus is actually not a cerus at all. When cacti were first collected and categorized they were lumped into one genus named ‘Cactus’. Later, taxonomists determined that there were several groups of cacti and split the plants into groups like Opuntia (beaver tail cacti), mammillaria (fish-hook cacti), Cerus (columnar cacti) and others. In the newer classification many columnar cacti were lumped into the Cerus genus, the Night Blooming Cerus being one of them – hence it’s name. However later work further subdivided plants in the Cerus genus into more groups. Columnar cacti like the Giant Saguaro remained in the genus Cerus, but vine-like cacti like the Night Blooming Cerus were moved into a new genus named ‘Peniocerus’, where the species is currently classified.

In Arizona and New Mexico, the Night Blooming Cerus is a protected or endangered species that has seen much of its habitat lost to agricultural and urban development. In addition, its future is dependent upon hawk moths which themselves are vulnerable to impacts from nearby development. For example, where pesticides are heavily used in agricultural areas adjacent to desert habitat, the hawk moth populations can be greatly reduced, such that few remain to pollinate the Night Blooming Cerus to help maintain the cactus’ population size.

Photos from the 2012 Night Bloom at Tohono Chul Park:

All images on this page copyright 2012 by John Donoghue II. All rights reserved.

For more information, see:

Mobile Apps for Naturalists

Project Noah LogoSince nearly everyone carries a smartphone these days, we can now carry a small library worth of field guides with us inside our phones so we never miss an opportunity to identify an interesting organism. With this in mind, here are a few mobile applications that are today’s field ecologist or naturalist should definitely consider having on your smartphone.

Merlin – Merlin is the Cornell Laboratory for Ornithology’s new bird identification app. Answer five simple questions about what bird you’re seeing and Merlin will come up with a list of possibilities based on both the answers to your questions and your location. Merlin is able to make highly educated guesses using its database of more than 70 million observations from the eBird citizen-science project.

Journey North – The Journey North app engages students in a global study of wildlife migration and seasonal change. Focused for K-12 students, users record and submit their observations while they are in the field and share their own field observations with classmates across North America. Students can use the app to track the coming of spring through the migration patterns of monarch butterflies, robins, hummingbirds, whooping cranes, gray whales, bald eagles— and other birds and mammals; the budding of plants; changing sunlight; and other natural events. (iPhone, Android).

BirdLog– I have been wanting a good mobile field notebook application for birding for quite a while. I even went so far as to build my own database to sync between my phone and laptop. BirdLog, is better. Developed in cooperation with the Cornell Lab of Ornithology, BirdLog helps you record and submit your latest bird sightings from the field. Moreover, your sightings contribue to the eBird database and provide data for thousands of amateurs and scientists who study birds. Among its many features, BirdLog allows you to create lists even when there is no cell coverage, keeps a running total of the number of species on your checklist, and plots the exact location of your sightings using your phone’s GPS. (iPhone, Android)

Nature’s Notebook – The National Phenology Network (NPN) has just released the mobile apps for Android and iPhone that let you record phenology (the annual timing of events such as flowering) observations in the field. Fill out a very short form and submit your information for use by scientists everywhere. While many people and applications are set to record when events like flowering start, knowing when they stop is just as important, as it helps tell scientists whether the growing season is getting shorter or shifting. So, one important component of the Nature’s Notebook application is that you can also use it to collect when plants have yet to flower or have stopped flowering, are showing new leaves or have dropped their leaves. (iPhone, Android)

Leafsnap – You can use Leafsnap to help identify plants. Simply use the application to snap a photo of a leaf and it will be uploaded to their servers where sophisticated image recognition software will combine the leaf’s photo with your location (from your phone’s GPS) to suggest possible matching species. (iPhone, Android)

iBird – iBird is the definitive application for identifying birds. It loads the contents of several bird guides onto your phone, combining drawings, photos, recordings of bird calls, range maps and other information to help you lookup any bird and identify its species. iBird loads all of this information on your phone, so you can use it in areas with no internet access. With iBird, there’s really no need to carry a thick bird guide around anymore. (iPhone, Android)

National Audubon Mobile Field Guide Apps – The National Audubon Society offers a suite of applications for smartphones that contain field guides to birds, mammals, reptiles, fish, insects, trees, wildflowers, and mushrooms. (iPhone, Android)

My Nature Animal Tracks – When all you have is evidence that an animal was present, you can use My Nature Animal Tracks to identify the animal by examining their tracks and scat using this handy app. (iPhone)

PhenoMap – Help scientists study changes in phenology (the annual timing of events such as flowering) by snapping photos of when natural plants in your area bloom. Fill out a very short form and upload your sighting to the University of Chicago and the National Phenology Network for inclusion into their phenology database. (iPhone)

ProjectNoah – Earn merit badges for being a citizen scientists and help the larger scientific community by logging your sitings of species in your neighborhood, on trails, etc. The information submitted by tens of thousands of citizen scientists is used to help study species diversity and distributions and other questions. (iPhone, Android)

These are just a few of the many exciting and helpful mobile applications that can assist your field excursions and reduce the number of field manuals you carry. While I’ve tried to introduce some very good and popular applications, this list is by no means complete. If you know an app that should be on this list, please add a comment below.

Create QR Codes Using the Google Chart API

For some of you this might be old news, but I wanted to create a QR Code for a poster presentation for an upcoming ecology conference (since all the cool people are using QR Codes these days). In searching for a way to generate one, I learned that you can easily create a QR Code using the Google Chart API.

To create a QR Code, simply enter the following URL into your web browser’s address bar:

(but don’t press enter yet, see the next step)

Before pressing enter, complete the API command by entering the URL of the location you want to generate a QR Code for after the chl property and equal sign (chl=) at the end of the link above.

For example, the complete API command you would enter into your web browser’s address bar to generate a QR Code for a link to this blog would be:


This creates a 300 x 300 pixel QR Code. To generate different sizes, simply modify the chart size property (chs) to a different pixel size. For instance if I wanted to generate a larger version of the same QR Code above, I could enter:


Happy QR Coding.

A Script Tool for Unzipping Files in ArcGIS

I was recently working with the SWReGAP data DVDs and wanted to develop an ArcGIS geoprocessing model to copy the DVD data, clip the rasters to the state of Arizona boundary and reproject them.

Because the DVDs contained a raster dataset for each species that was stored as a zip file, I wanted a way to decompress each raster within my model rather than simply decompressing all the zip files to a folder on my hard drive and using the uncompressed files in my model. (This is because each DVD was filled to capacity with 2+ GB zip files, so decompressing the DVD contents to my hard disk first filled up the remaining space on my hard drive. So I decided that I should decompress each file individually and delete it when I was finished.)

So I set about developing a script tool to decompress each zip file so that I could incorporate the script tool into my ArcGIS geoprocessing model. I tried using the zipfile libraries built into Python, but ran into problems decompressing these files because each zip file was too large for the readfile buffer used by the Python routines.

This led me to 7-Zip, an open source to WinZip. While 7-Zip contains a user interface, I was particularly interested in the 7-Zip Command Line Version as this made it possible for me to call the command line from my ArcGIS script too.

With the 7-Zip Command Line Version installed and a few lines of Python, I had a script tool that was capable of extracting a zip file and could be included in an ArcGIS geoprocessing model. I then developed a second script to decompress a whole folder of zip files, and a third script for zipping a feature class.

import os
import arcpy
infile = arcpy.GetParameterAsText(0)
outpath = arcpy.GetParameterAsText(1)

# create 7zip command
zipcommand = “7za e -tzip \”%s\” -o\”%s\”” % (infile, outpath)

      # execute command
except Exception as e:
      print e.message

I then added the script to a toolbox in ArcGIS as a script tool and set two parameters: 1) Zipfile Name as a file, and 2) Output Location as a folder. When clicked, the script tool appears like the one below:

I was pleased enough with the resulting tool that I developed a simple model that incorporated the tool into an iterator to loop through a folder of zip files and extract each one. That model is illustrated below:

Finally, I also created a second script tool that decompresses an entire folder of zipfiles to a specified location.

Since these have been helpful in automating my workflow I thought I would offer them to the GIS community as a zip file that contains a folder with an ArcGIS Toolbox, two script tools and a sample mode. You can download them from my website at: http://www.johndonoghue.net/ecology/resources.html

To use the ArcGIS Script tools:

There are a couple of things you need to do before the tools will work. Follow the simple steps below to obtain a copy of 7-Zip Command Line Version and modify your system paths.

  • Download and install 7-Zip Command Line Version. When you download the application it will consist of a zip file. decompress the zip file to a folder on your hard disk. I highly recommend extracting the files to a folder in your root directory named something like C:\7Zip or the default C:\7za920. If you extract them into a folder inside your program files folder on 64-bit systems, you may not be able to run the scripts as Python cannot find the executable even with the path statement.
  • Add the path to your 7-Zip installation to your Windows PATH environment variables – add it to both the System PATH variable and the User PATH variable . For example, my path was: C:\7za920; (see http://www.computerhope.com/issues/ch000549.htm if you need help doing this).
  • Finally,download the  Tools from my website and extract them to a folder on your hard drive. You should then be able to use them as you would any ArcGIS script .

I hope you find these tools helpful in your work.

Are Some Parts of ArcGIS Getting More Difficult to Use?

I got a call from a colleague who was having dificulty getting ArcGIS to calculate the value of a field in an attribute table. He was building a model in ModelBuilder and wanted to perform a conditional (If Then) calculatuion to evaluate whether a field’s value was Zero (0) before he tried to divide another value with it.

In response to his question, I quickly provided him the following VBA code for his field calculator tool:

Pre-Logic Script Code:

if ([SUM_weeks] <> 0) then
    newval = [SUM_deliv] / [SUM_weeks]
   newval = .001
end if

Assignment Code:

[NewField] = newval
Field Calculation Example with VBA

Field Calculation Example with VBA

While this worked fine, he mentioned that he would rather have Python code, since it is rumored that esri is removing VBA in future releases of ArcGIS. In response, I initially tried to simply convert the Pre-Logic code block to the following Python code:

if (!SUM_weeks! <> 0):
    newval = !SUM_deliv! / !SUM_weeks!
    newval = .001

Much to my surprise this code generated a parsing error, and try as I did, I could not change it in a way that would work, despit the fact that it appears to be perfectly valid Python code.

After reviewing some portions of the ArcGIS Desktop help file, I learned that in order for me to execute the same statement in Python, I needed to write a Python function and then pass the field values into that function. So I composed the following Python code to send my colleague:

Pre-Logic Script Code:

def DoMath(field1, field2):
   if (field2 == 0):
       newval = .001
      newval = field1 / field2
   return newval

Assignment Code:

DoMath(!SUM_delive!, !SUM_weeks_!)
Field Calculation Example with Python

Field Calculation Example with Python

This worked fine but I was really surprised that I could not just use the field names within the code block. We’ve always been able to do this in esri projects. We coudl do it in VBA and even Avenue (for those who remember the ArcView 3.x Query Builder), AML allowed this too.  But not anymore, why?

As a long time esri user, this kind of stuff concerns me. I’m not worried that they going to lose their customer base or market share. But it seems that as ArcGIS has grown more powerful some portions of it have grown more difficult for casual users. Replacing dialogs and functions that worked well for casual users with Python code blocks works well for programmers and power users, but it leaves casual users to struggle with stuff they used to know how to do.

Users shouldn’t have to write a Python function and then pass an attribute field value into that function to edit a field. This kind of editing used to be easier and I hope someone at esri figures this out and simplifies it again.

P.S.  Don’t get me wrong, it’s not that I hate change or dislike Python; I use it often. I should point out that there are some wonderful simplications to field editing in ArcGIS 9.x and 10 that esri programmers should be commended for. I greatly enjoy and make frequent use of the Calculate Geometry functions and appreciate not haveing to write the VBA code blocks that I used to write to add X and Y coordinates to attribute tables or calculate polygon areas. Actually I only wrote the code once and then saved the query expressions for subsequent uses, but still, it’s much easier to use the newer Calculate Geometry functions.

Dropbox – A Dissertation’s Best Friend

For the past few months I’ve been greatly enjoying the ability to sync files between my PC laptop, Mac laptop, iPhone and the web and can’t say enough about how excellent Dropbox is.

Dropbox installs on your computer and appears like a folder. You work with it like you would any folder. You can drag and drop files into it, save files directly to that folder, etc. However, when you put files into your Dropbox folder, those files are automatically synced to your secure and private Dropbox account where they are stored offline (and even versioned).

I’ve been using it to backup important documents and also store files I’m actively working on, such as my PhD dissertation – something I want to ensure gets automatically backed up with every change.

I’ve also noticed that I stopped putting files on thumb drives when going between my work, school and home computers. I just store the files I’m actively working on in my Dropbox folder; I load them and work with them from that location. When I save the file, the changes are automatically made available to me on any computer that I’ve installed Dropbox on. So I can start a document at school and then pick up where I left off at home, or vice versa.

It even came in handy once when I needed to show a PowerPoint presentation to a colleague. I didn’t have a copy with me but remembered that it was in my Dropbox folder. So I used the Dropbox iPhone app to load the presentation so I could show it.

Dropbox also allows you to share folders. I’ve started using Dropbox to exchange files with my ecology and informatics colleagues. What’s great about this option is that once someone shares a folder with you (and you accept their invitation), the shared folder appears in your computer’s Dropbox folder and the files any authorized user puts into that folder are automatically synced.

So if I want to share a file with my colleagues, I just save it to one of my shared folders and the file syncs to the web and then automatically syncs with each colleague’s’ computer when they are on the internet. Dropbox even displays a pop-up a message letting me know that new files were added or updated to my shared folders.

Dropbox will give you a 2GB account for free; no strings attached and no credit card required.

Once you sign up, you can invite your friends and colleagues to join and earn 250 MB of free bonus space for each person that signs up. You can do this to get a maximum of 8GB of free space (something that I’m obviously trying to do by posting about it).

But really, everyone I know who’s installed Dropbox has loved its ease of use and functionality. So I encourage you to try it and if you click one of the links here you can help me earn more free space 😉

Dear Census Bureau, Please Don’t Skip Me Again

Driving home from the office the other day, I heard a story on public radio about the accuracy of US Census data. The story provided an overview of several metropolitan cities that appeared to be shrinking according to US Census data. The story then went on to discuss demographic data providers who utiltize mutliple means of data collection to provide a much more accurate demographic picture of these areas.

There were several accounts of local government agencies who challenged the US Census conclusions using reports from third-party demographic data providers and won. Since many federal funds are tied to population statistics, these local agencies were set to receive less funding than they should have (according to the third-party demographics), so they challenged the Census figures to ensure that they received their fair share of federal monies.

The whole story got me thinking about how accurate the Census really is. I worked for a County government GIS department when the 2000 Census was being planned, and remember all the work that everyone in the department did to provide the Census Bureau with accurate and up-to-date address locations to help them plan their censusing efforts. We combined the results of a number of different data sources into a comprehensive list that was scrubbed using some AMLs, and in some cases manually checked for accuracy.

Since I was a Geographer by education and a GIS person by profession, I was thrilled about helping the US Census Bureau prepare for the Census. As it turned out, I never received a Census form or a visit from a Census taker. Was I missed again? I’ll never know. But I was disappointed that I never had the opportunity to participate in the actual Censusing either by completing the form or talking to an interviewer.

What made it most frustrating was that in 2000, I owned a home. So I thought “this time I’ll be sure to be included”. When the previous 1990 Census was taken, I was renting a condominium while attending college studying Geography. So I was excited about the Census and ready to participate. Collection came and went, no form and no interview. I asked my landlord if he received a form for the Condo I was renting, thinking that perhaps he completed the form on my behalf. He didn’t. Was I missed?.

So here were are in mid-2009, with the 2010 census being planned. I know local Census takers are running around town, though I’ve not seen any. No one has come to my door yet. Will I get a form this time? Hopefully. I would really be disappointed to be missed a third time.

My experience leaves me to wonder what exactly happens to the data us GISers provide to the Census Bureau? How is it used by the Bureau to determine who should receive forms and who should get a visit from a Census taker? The Census is not a survey, it’s supposed to be a complete Census of all individuals in the US. So ideally no one is skipped. So, how is it that I’ve been missed in the last two censuses?

Has the Bureau found another way to obtain demographic information on me such that they don’t need to send me a form or direct a Census taker to my house? Surely they can learn some information from County assessor records, but this doesn’t include the other sociological and economic information they collect. Are  they getting this from other data sources? What is the statistical possibility that I was missed in the last two censuses? What does that say about how accurate the data really is?

Any fixed-odds on me getting missed again this time around?

Social Guide to the ESRI User Conference

Okay, we’ve seen plenty of offical guides and articles offering advice on what to pack, how to get there, where to stay, how to choose sessions, etc. What none of these articles tell you are the key facts for the social side of the conference. Since networking is everything, it’s just as important to have a plan for the evening as it is for the conference day.

So, let’s start a discussion on all the important social events and customs a new ESRI User Conference attendee should know about. For example, where should I eat for lunch? Dinner? Which bar has the best beer? Where are the best industry parties? etc. I’ll start:

Dick’s Last Resort: Sure it’s a chain, but at some point in the conference, just about everybody eats here for lunch. So if you’re looking for someone to meet up with or spy on, this is the place. Occassionally you’ll also get the bonus of seeing some tourist get furious at the servers becuase they don’t understand that being treated rudely is part of the Dick’s experience.

Industry Parties: Get to know someone in the Electricity and Water/Wastewater User Groups. Tuesday and Wednesday nights are the vendor and industry sponsored parties; and the utilities always throw a good one. Attend the Electricity and Water/Wastewater User Group Socials in the evening and buddy up to a vendor to get a pass for the industry party that will happen afterwards.

Okay, now it’s your turn… add a comment with your advice.
(I received a number of suggestions by email, so I’ve added several in the Comments below)