Dear Census Bureau, Please Don’t Skip Me Again

Driving home from the office the other day, I heard a story on public radio about the accuracy of US Census data. The story provided an overview of several metropolitan cities that appeared to be shrinking according to US Census data. The story then went on to discuss demographic data providers who utiltize mutliple means of data collection to provide a much more accurate demographic picture of these areas.

There were several accounts of local government agencies who challenged the US Census conclusions using reports from third-party demographic data providers and won. Since many federal funds are tied to population statistics, these local agencies were set to receive less funding than they should have (according to the third-party demographics), so they challenged the Census figures to ensure that they received their fair share of federal monies.

The whole story got me thinking about how accurate the Census really is. I worked for a County government GIS department when the 2000 Census was being planned, and remember all the work that everyone in the department did to provide the Census Bureau with accurate and up-to-date address locations to help them plan their censusing efforts. We combined the results of a number of different data sources into a comprehensive list that was scrubbed using some AMLs, and in some cases manually checked for accuracy.

Since I was a Geographer by education and a GIS person by profession, I was thrilled about helping the US Census Bureau prepare for the Census. As it turned out, I never received a Census form or a visit from a Census taker. Was I missed again? I’ll never know. But I was disappointed that I never had the opportunity to participate in the actual Censusing either by completing the form or talking to an interviewer.

What made it most frustrating was that in 2000, I owned a home. So I thought “this time I’ll be sure to be included”. When the previous 1990 Census was taken, I was renting a condominium while attending college studying Geography. So I was excited about the Census and ready to participate. Collection came and went, no form and no interview. I asked my landlord if he received a form for the Condo I was renting, thinking that perhaps he completed the form on my behalf. He didn’t. Was I missed?.

So here were are in mid-2009, with the 2010 census being planned. I know local Census takers are running around town, though I’ve not seen any. No one has come to my door yet. Will I get a form this time? Hopefully. I would really be disappointed to be missed a third time.

My experience leaves me to wonder what exactly happens to the data us GISers provide to the Census Bureau? How is it used by the Bureau to determine who should receive forms and who should get a visit from a Census taker? The Census is not a survey, it’s supposed to be a complete Census of all individuals in the US. So ideally no one is skipped. So, how is it that I’ve been missed in the last two censuses?

Has the Bureau found another way to obtain demographic information on me such that they don’t need to send me a form or direct a Census taker to my house? Surely they can learn some information from County assessor records, but this doesn’t include the other sociological and economic information they collect. Are  they getting this from other data sources? What is the statistical possibility that I was missed in the last two censuses? What does that say about how accurate the data really is?

Any fixed-odds on me getting missed again this time around?

Social Guide to the ESRI User Conference

Okay, we’ve seen plenty of offical guides and articles offering advice on what to pack, how to get there, where to stay, how to choose sessions, etc. What none of these articles tell you are the key facts for the social side of the conference. Since networking is everything, it’s just as important to have a plan for the evening as it is for the conference day.

So, let’s start a discussion on all the important social events and customs a new ESRI User Conference attendee should know about. For example, where should I eat for lunch? Dinner? Which bar has the best beer? Where are the best industry parties? etc. I’ll start:

Dick’s Last Resort: Sure it’s a chain, but at some point in the conference, just about everybody eats here for lunch. So if you’re looking for someone to meet up with or spy on, this is the place. Occassionally you’ll also get the bonus of seeing some tourist get furious at the servers becuase they don’t understand that being treated rudely is part of the Dick’s experience.

Industry Parties: Get to know someone in the Electricity and Water/Wastewater User Groups. Tuesday and Wednesday nights are the vendor and industry sponsored parties; and the utilities always throw a good one. Attend the Electricity and Water/Wastewater User Group Socials in the evening and buddy up to a vendor to get a pass for the industry party that will happen afterwards.

Okay, now it’s your turn… add a comment with your advice.
(I received a number of suggestions by email, so I’ve added several in the Comments below)

Bing Maps?

Please note, the following text is an editorial comment. If you’re looking for information about Bing Maps, see http://www.discoverbing.com

 

Today Microsoft announces another name for Virtual Earth – Bing Maps. When they previously changed the name of Virtual Earth to Live Search Maps, I understood their intention of aligning their consumer mapping application (Virtual Earth) with the other consumer applications in the Microsoft Live Search family. However I personally disliked the name Live Search Maps; it was too long and unfriendly. So I’ve always continued to refer to the service as Virtual Earth (as did many other people).

This time Microsoft is attempting a much larger rebranding strategy around a new search platform Bing.com. Bing is an attempt to gain some ground on Google in the search arena. While Bing is a hip name, the rebranding does make me wonder what Microsoft is thinking. I’m sure a large part of the change is hope that Bing will become a household verb like Google. Somewhere a Microsoft ad agency is dreaming about people asking one another, “did you Bing it? What is it? Where is it?”

Microsoft is throwing g a lot of money ($80 to $100 million) at this rebranding campaign to get consumers to question whether Internet searching is giving them what they want. Unless they’ve got some fantastic features to show consumers an alternate reality that is markedly better, Bing will probably not work. Google achieved search dominance because they had a superior product. To supplant them in the search arena, Bing must be better, not just fancier. A fancy updated user interface with web site previews inside pop-up windows and a multi-million dollar marketing campaign did not help Ask.com. So what kind of bling does Bing have that is going to make me switch?

As an aside, while the name Bing is better than Kumo (another name in contention by Microsoft), when I read the single syllable word “Bing” after the word Microsoft, I instantly thought of another single-syllable word “Bob” – another famous Microsoft failure. I’m not trying to be funny; the association was instantaneous and completely uncontrived.

From the perspective of a geospatial professional I think Microsoft is trying to deal with an identity crisis. If we look at the suite of mapping related products/services offered by Microsoft there are:

• Live Search Maps (Formerly Virtual Earth)
• Microsoft Virtual Earth APIs – which were used to interact with Live Search Maps
• MapPoint Web Service

In the near future Bing rebranding will change:

• Live Search Maps = Bing Maps
• Microsoft Virtual Earth API = Bing Maps for Enterprise

Since the MapPoint Web Service name is not being changed, I’m guessing that this does not bode well for its future.

I am curious about whether there are plans to align all of the Windows Live products (Messenger, Mail, Calendar, Photos, Spaces, SkyDrive, etc.) around the Bing brand. Otherwise, it seems there will still be an identify crisis when it comes to consumer products. I am supposed to use Bing to search, but use Live for everything else.) Not to tout the obvious, but gMail, gTalk, Google Docs, etc. all unite around the name Google. How will Bing and Live become associated with Microsoft without millions of dollars of marketing?

I do think it’s a good idea for Microsoft to fix its geospatial product identify crisis and align their search related offerings around one brand. However, I think the product/service naming could be tighter. For example, Google’s mapping environment is named Google Maps. Their API is named the Google Maps API. Their 3D version is named Google Earth. It’s all very intuitive. It’s why I liked the name “Virtual Earth”, it worked.

If Microsoft wants to align their geospatial offerings around a fun (and potentially verb-able) product name like Bing, then I suggest they name their primary mapping products “Bing Earth” and “Bing Earth for Enterprise”. The API would then be called “Bing Earth API”. While they are at it, they could rename the desktop application MapPoint to “Bing Earth Studio” or “Bing Earth Desktop”. Sure Google would be mad, but is the name Bing Maps any different than Google Maps? Besides, Microsoft was already using the word Earth in Virtual Earth, so there’s already a precedent for some overlap.

Learning GIS and Getting a GIS Job – Some Tips and Tricks

I’ve seen some forum posts recently from folks wanting to know how to learn GIS, and many of the responses have had fairly typical information. So, this got me thinking about my own path to learning GIS and the atypical ideas and skills I learned and developed along the way that helped me excel in the industry. I used to teach some GIS course at the University of California Riverside Extension; and many of these ideas are drawn from my lectures in the practical class I co-taught with a GIS colleague. So, here is a rather lengthy discussion of how I got into GIS, what I did to set myself above the pack and what recommendations I give to aspiring GIS students. 

Getting Started
There are a variety of ways to get started learning GIS. You may be employed in a non-GIS field and are learning GIS as part of your job. Maybe you are switching careers or retraining yourself to find a new job in the growing geospatial industry. Perhaps you are a student who wants to learn GIS to support your field of study, or you are a student majoring in GIS or geography with aspirations of landing a job doing GIS.

Depending upon your goal, your pathway to get started learning GIS may vary. If you are a student intending to major in GIS or geography, your best bet would be to start at a community college or university where you can take accredited GIS courses that will apply to your major’s requirements. However, those seeking GIS skills to broaden their work-related knowledge or re-tool themselves for immediate employment may be better served by finding a professional certificate program in GIS at a local community college or university extension. These programs are usually non-accredited, which means that they will not likely count towards satisfying the requirements for a degree in GIS. However, professional GIS certificate programs are a great way to learn GIS quickly and gain readily employable skills to help you enter the marketplace. In addition, many professional certificate programs are taught by professionals working in the field, so you will get first-hand information about how GIS is used by the people who use it daily. These programs are also fantastic places to network with potential future employers.

My Experience
In my own career, I started learning GIS on my own without any training. Once I realized that I wanted to work with GIS as a career, I found a professional GIS certificate program and enrolled in classes. I was very fortunate to be in a region with a fantastic certificate program and excellent instructors from public and private industry. I took many classes to learn a variety of GIS topics, and I talked often with my instructors to get a feel for the industry and where the job potential was. I attended local and regional GIS User Groups to further network with peers and potential employers, see new product demonstrations, and learn tips and tricks that would help me in my GIS practice. I also read GIS industry magazines to learn how organizations were implementing GIS and get a sense of what direction the industry was moving. Finally I read ads for GIS jobs to see what employers were looking for and where the jobs were.

All of this helped me learn not only what minimum skills are essential, but what additional skills are valuable to employers. I quickly formed an idea of what skills make up a ‘great’ GIS, besides just knowing a lot about GIS. These other skills are those that provide a competitive advantage over others who just learned how to use a particular GIS software package such as ArcView.

The main skills that stood out were database management and application programming. Fortunately, I had some experience with programming before getting into GIS. In my previous jobs I wrote a few small Visual Basic programs to help automate certain tasks. Since I knew these skills would be extremely valuable to a GIS person, one of the things I did was find an opportunity to become a better programmer and database person while I was learning GIS.

So I took a job as a programmer for a start-up company. The job had nothing to do with GIS; and I was also struggling at first, because I’d never been employed as a full-time programmer. But it wasn’t too long before I was coding like a pro. The job also required me to learn SQL Server, so I was getting the database management experience I wanted as well. I was getting great programming and database management skills, so I wasn’t concerned about not working in GIS at that moment. I was learning GIS at night and on weekends in my certificate program classes, so I was making progress with my career goals. It was the combination of the programming skills I was learning on the job and the GIS knowledge I was learning in my classes that would pay off; and it did.

Within a year I was offered a job in government chiefly because of the connections I had made while networking with my instructors and at the local GIS user group. Working as a GIS person in government provided many opportunities to apply GIS to a variety of situations that involved crime mapping, land use planning, utilities, finance and others. In almost every case, the database management and programming skills I learned at my non-GIS jobs came in handy. From writing scripts to scrub and reformat data, to creating ad-hoc Microsoft Access databases to organize data for our GIS, these non-GIS skills helped the GIS team work faster. My colleagues who didn’t know how to write scripts or work with databases often did this work manually and talked about wanting to learn these other skills.

Today I’m a GIS consultant and ESRI Business Partner. I get to help agencies and organizations implement GIS in all kinds of environments and situations. The job is full of challenges as each client’s situation is different. I also continue to learn new skills. For example, I’m currently teaching myself to program in Microsoft Silverlight to see if it can help enhance our mapping applications.

So what are the take-home messages from my experience?
If you wan to learn GIS and you want to increase your chances of obtaining employment: 

Find a class
Ideally I think in-class learning is more valuable than online training, so I would first recommend that you look for GIS classes offered by your local community college, university or extended university. Some regions also have non-profit organizations that offer technical classes to train displaced workers for new careers. Some of these groups offer GIS classes too. If you can’t find a class in your area, then look online. There are a number of online GIS classes offered by ESRI and other organizations:

ESRI Virtual Campus (http://training.esri.com/gateway/index.cfm)
List of GIS Schools (http://www.gis.com/education/formal.html)

Attend regional and local user groups
I can’t stress enough how important this is. Virtually everyone I know working in GIS got their first or subsequent job as a result of networking at a local or regional user group. Find one and attend them often. Ask questions, talk to people – they will become your colleagues and peers. If there isn’t a local or regional User Group in your area, start one. There are probably lots of GIS people working in your area that would love to attend a user group. It’s just no one has the time or wants to responsibility of organizing one. If you have the time and don’t mind calling people, this is a great way to make contacts in the GIS industry very quickly. Talk to your ESRI regional representatives and they can provide names of people you can contact about speaking, attending or helping. Many agencies can get the use of their meeting rooms for free. Most local vendors will jump at the chance to market their product to a room full of potential customers, so vendors can be helpful in filling presentation slots and getting the word out. With a little bit of tact you can usually get a vendor to sponsor snacks if they are presenting. Whether your group starts out as a formal event or an informal lunch with colleagues, get together with GIS people and network.

Network with GIS professionals and classmates
If you are fortunate to be in an area where you can attend in-class training, be sure to network with those in your classes. In my experience while attending classes, those of us that talked with each-other shared information about job openings and tricks we learned. Those who didn’t network with us didn’t get this information. Be sure your class peers know who you are and admire your knowledge. You could share tricks you’ve learned, tell them about upcoming user group meetings, etc. It will impress them and impress your instructors.

Get real-world experience
You might be thinking about the double-edged sword: You can’t get experience without getting a job and you can’t get a job because you don’t have experience. The fact is I’ve been on a lot of hiring committees for entry-level GIS positions, and there are always TONS of resumes from candidates whose only GIS experience consists of course work. There are a lot of people out there who have taken GIS classes. Most of those introductory classes teach people how to push buttons to ‘operate’ the software. As a prospective employer, we want to see a candidate who can ‘use’ the software.

This is an important distinction. From your coursework I have no doubt that you know how to operate the GIS software. What I can’t see from your coursework is whether you can think on you own and whether you have solved real problems, using real data (classroom data always works well) in the real world using GIS. Unless I can see experience like that, or unless you’ve stood out in some other way (refer to the networking and user group items above), I’m probably not going to take a chance on you.

To get that GIS job you have to get some real world experience. So how do you do that? You volunteer. If you have the requisite GIS skills you get from a couple of introductory classes, there are plenty of government agencies who will take you on as a volunteer intern and build upon your knowledge in exchange for some volunteer labor from you. So, you give up some free time by helping them, and they teach you to use GIS in the real world, with real data, to solve real problems. What does this give you? It gives you another network opportunity and demonstrable experience to discuss at an interview. You can also show off copies of your maps and analyses from the portfolio of work you’ll develop during your volunteer efforts. Plus if you’ve proven to be a good asset, a good boss will try their best to keep you around by hiring you. Otherwise, they will let you know about opportunities with other agencies. It’s not limited to government agencies too. Look around for local non-profit agencies that might need help.

Learn auxiliary skills
I’ll say it again: learning to use ArcGIS Desktop is good. But a lot of people who want to work in GIS learned to use ArcGIS Desktop. What else can you do?

Databases
Any GIS person worth their salt knows how to set up and manage a database. Whether it’s Microsoft Access, SQL Server, Oracle, mySQL, PostGRESQL, etc. if you don’t know how to use a database, learn to. These days everyone stores data in a database. It might not be well organized, but it’s in a database; and you’re going to have to know how to work with it to make it useful in the GIS. My recommendation is that you learn Microsoft Access. It’s a good database system that’s easy to learn and use. From there you can graduate to the workgroup and enterprise systems such as Microsoft SQL Server, Oracle, mySQL, PostGRESQL, etc.

Scripting
Learn to program in a scripting language. Pick a scripting language such as Visual Basic for Applications (VBA), Python, or JavaScript and learn it. These scripting languages come in handy when you need to scrub huge databases to perform actions like correcting addresses for better geocoding accuracy, reformatting data, exporting and importing large datasets, and other fairly common activities GIS people get tasked with.

An easy way to start is learning Microsoft Visual Basic for Applications (VBA). It’s included in ArcGIS Desktop and it’s embedded in all Microsoft Office products. With a copy of Microsoft Access and a solid knowledge of VBA, you can convert anything into a format readily useful in a GIS. I use VBA and Microsoft Access to reformat data all the time. My data usually ends up in SQL Server, but it’s far easier to prepare it using Microsoft Access and some VBA scripts. ESRI has some resources to learn VBA and there are many websites that provide VBA resources for working with Microsoft Access. 

ESRI Virtual Campus VBA Classes
http://training.esri.com/gateway/index.cfm?fa=search.results&searchterm=VBA&softwaretype=All+Software&trainingformat=1%2C2&search=search

VBA Wikipedia
http://en.wikipedia.org/wiki/Visual_Basic_for_Applications

VBA is also your gateway to learning ArcObjects and object oriented programming for more advanced GIS development projects.

Crystal Reports
You know that CD that comes with ArcGIS Desktop that installs Crystal Reports? If you haven’t done so, install it. If you have installed it, play around with the Crystal Reports program. Make sure you figure out how to use the Crystal Reports report functions in ArcMap, but also play around with the core Crystal Reports program that you can launch without ArcMap. Crystal Reports is a powerful reporting tool that can read databases and spreadsheets, perform queries, sort and summarize data and generate nice looking reports.

Why learn this? Because one of the first lessons I learned working in GIS for government was that it’s not always about a map. In fact, it often isn’t about a map at all. Sure we GIS people got into GIS because we like to make pretty maps. But the mistake we make is in thinking that everyone else likes maps too. When I started working in government GIS, I was amazed at how often I was asked to create a report showing the results of my analyses. No map. No pretty aerial photo; just a table of numbers representing the results of a geographic question. Sure there was a lot of cool GIS analysis that went into generating the resulting table, but the deliverable product wasn’t a map. As you might guess, I often relied on Crystal Reports to generate that final product from the GIS analysis. I’m pretty sure you’ll find that as your analysis skills increase, the proportion of maps versus reports you generate will shift. So, learn Crystal Reports. You may find that in knowing it you’re valuable to non-GIS users as well. By learning Crystal Reports for GIS, I was able to help the finance department make a bunch of reports of data from the financial system. It had nothing to do with GIS, but I was pretty valuable to the team for my Crystal Reports skills.

In Conclusion
If you’re successful in getting a GIS job, you’re entering a great career at a time when GIS technology is advancing very rapidly. You’ve got to keep your knowledge up-to-date, so make sure you continue to learn new tools and techniques for working with GIS. Where can you do this? Most of the suggestions I’ve discussed are also appropriate for keeping your GIS skillset current. In addition, there are numerous online resources for learning about GIS techniques, and industry news. Social networking sites like Twitter offer the ability to tap into the thoughts of many great GIS people in real-time, while blogs provide in-dept articles about GIS topics (such as this one). In addition, nothing beats the software support forums. Chances are you are not the first person to encounter a problem or have a question about how to do something. The forums are your first line of support, use them. Along the way if you learn useful to the GIS community, share it and help others.

How I’d like ArcGIS to Work with SQL Server 2008

When Micosoft announced that SQL Server 2008 would include spatial data types I was excited. I’ve been a SQL Server user for a long time and have always preferred its management interface over those of the other DBMS I’ve tried. So the idea of spatial data types was exciting. Though there weren’t many tools out there to manage spatial data in SQL Server at first, I knew that ESRI would include support in the next update to ArcGIS, which they did at 9.3.

It seemed that GIS vendors such as Manifold beat them to it and included support for Katmai as soon as a pre-release build was available that included the spatial components. But I’m an ESRI user; and I’ve been one for a long time. So I knew that 9.3 would include support for SQL Server 2008.

And it did, except that it requires registering that SQL Server 2008 database in ArcSDE. Come on guys, why do we need ArcSDE to work with simple geometry? As a consultant I can’t always convince a client to go with ArcSDE for a variety of reasons:  they can’t afford it, they don’t have the IT resources to support it, etc. There are lots of times when adding the ArcSDE business tables to a SQL Server database schema is overkill to store some points lines and polygons. There are times when I’m lucky enough just to convince a client to add a couple of fields to a database. ArcSDE has a lot of benefits and there are plenty of times when I do recommend it and implement it in client sites, but ArcSDE shouldn’t always be necessary to work with simple spatial data in a relational database.

I’m a big ESRI fan and I strongly believe that they offer superior GIS tools. However, sometimes the simple approach is best. I’m not a Manifold GIS user, but I do like the way they’ve handled using SQL Server 2008 in their desktop GIS application. The video below shows the kind of integration I was hoping to see at 9.3 and at 9.3.1.

http://www.manifold.net/video/mfdsql08.wmv

Besides the obvious direct connection to SQL Server without RDBMS middle-ware. The connection supports read, write and even table modification from the client. I’d love to see this kind of support for SQL Server 2008 spatial data types in ArcGIS. Maybe 9.4?

Welcome to My Blog

Welcome to my blog. For years I’ve been using, developing, and helping clients implement GIS systems. Along the way I’ve made a number of contacts, shared a lot of information with colleagues, and developed some ideas about what I’d like to see in GIS. So, rather than complain to my friends and GIS peers, I’ve decided to join the Blogging community and share my ideas with the masses. My goal is to provide information on new technologies and techniques to help improve your GIS implementations, as well as create stimulating discussions on GIS topics of interest. Hope you enjoy the blog.