More data, yet we can see things better

24 March 2010

A colleague forwarded me the link to a great IBM ad that encapsulates what businesses are dealing with today as far as data. We don’t have that type of ad budget at SpatialKey, but if we did, our message would certainly be in line with IBM’s. The future is not less data, but more. And this applies to all kinds of industries, companies, and organizations. Yet instead of being overwhelmed by it, with the right tools, this wealth of data will actually allow us to see things better.

That’s the business SpatialKey is in. We’re allowing people who in the past may not have been on the front lines of data analysis to take their spreadsheets, CRM data, salesforce.com data and more, map it, analyze it via simple click and drag tools, and extract meaning from it. Our goal is to help people -regular business people, not just data specialists- ask questions of their data, understand trends, and make better decisions, faster.

Police report

The great advantage of working at SpatialKey is that our clients come from such a diverse variety of organizations- each with totally different problems to address, and data analysis needs. We’re helping agencies better understand war zones and address conflict, allowing pharmaceutical companies analyze vaccine distribution efficiencies, or sales departments gain deeper insights into their sales funnel. Our partner Social Compact and the Citi Foundation recently launched a new service based on SpatialKey to help drive investments to undeserved communities. We’re even helping police departments get more bad guys off the streets by allowing them to better understand and address crime patters. The list goes on. Right now for example,  I’m working on a case study with a client in the energy sector. They are a publicly traded company that buys excess electricity from schools, businesses, etc and sells it back to utilities or grid operators when energy use spikes. In their case, they use SpatialKey to better monitor and analyze energy capacity and therefore more quickly respond to energy need changes. SpatialKey provides them insight that they could only dream of in the past. All that with no programming, nor expensive hardware, training or software investment.

Sales team report

Marketing team report

The common thread among all our clients is that they have tons of data, most of it coming from a variety of sources. They know that they could do a better job putting the pieces of the puzzle together to make better decisions, but they don’t have the energy nor budget to invest in complicated tools. Each one of our clients were amazed when they saw their first demo of SpatialKey. They suddenly saw their data like never before and could not believe such a simple solution had so much power. The power they needed to drill down into their data and make better decisions.

Crisis mapping

The other thread we’re seeing is that once SpatialKey is used within an organization, it does not take long for others within the same company to want to start using it too. Take the example of our energy client. They immediately saw that they could use SpatialKey to track not only kilowatts, but also leads, prospects and more.

Our engineering team is hard at work developing new functionalities into SpatialKey. We all know that the future is about more data, not less, so we want to provide the tools needed to allow our users to see the pins in the haystack that will help guide their decisions. Looking forward to seeing what other client uses are coming our way. Want to find out more? Go to Spatialkey.com or contact us.

del.icio.us:More data, yet we can see things better digg:More data, yet we can see things better spurl:More data, yet we can see things better newsvine:More data, yet we can see things better furl:More data, yet we can see things better reddit:More data, yet we can see things better Y!:More data, yet we can see things better

Reading and writing and … location. Visualizing where different peformance metrics correlate.

5 February 2010

Our parent company, Universal Mind, was tasked by the Colorado Department of Education and Center for Assessment to visualize data from their innovative models for measuring student progress. The public version of that project is available at schoolview.org. SchoolVIEW has some great features to visually compare school performance in terms of proficiency and growth (improvement over prior years) in reading, writing, and math. (You can learn more about the project here.)

CDE SchoolVIEW

CDE's SchoolVIEW

SchoolVIEW data in SpatialKey

I was interested in seeing the correlation between these different metrics, and (since we’re obsessed with location) how that correlation relates to geography. So, I imported that data into SpatialKey.  The source file was a CSV with a row for each school.  Here’s what that data looks like:

SpatialKey’s bivariate renderer allowed me to quickly explore the data in just that manner. The bivariate renderer allows you to select two numeric attributes in your dataset, and an aggregate calculation for each. In the image below, I selected average math growth percentile and average math proficiency.  Each dot in the scatterplot legend at the upper right represents a colored location (grid cell) on the map.  The position of the dot represents its relative score for average math growth (y axis) and average math proficiency (x axis).  The color “behind” the each dot is the color used for the corresponding grid cell on the map.

This visualization shows the coorelation between math proficiency and growth, as it relates to location. (Click the image for a larger view.)

We can see there is a general positive correlation, where most locations have a similar relative score for math performance and growth: Most points on the scatterplot are along an imaginary diagonal line from the bottom left (low in both metrics) to upper right (high in both metrics).  What’s often interesting and informative is to see areas that deviate from the norm in terms of the correlation.  Areas with relatively high proficiency but low growth – “strong but losing ground” – are colored blue, while areas with low proficiency but high growth – “risin’ up” – are colored red.  These are both negative correlations.  Areas that score low on both metrics are shaded white, while those high in both attributes are shaded black – both positive correlations.  For this type of visual analysis, areas that fall toward the middle of both ranges are usually less interesting, and so those colors are more transparent to allow you to focus on the extremes.  It may take a few seconds to orient yourself to this view, but once acclimated it’s a powerful way to visualize some complex – and otherwise difficult to express – relationships.

You can correlate any pair of attributes by simply selecting from one of the axes in the scatterplot legend. This next image compares average math and reading proficiency.  First, notice there seems to be an even stronger correlation between these two variables than the previous set.  (The points line up even closer on the imaginary diagonal line.)   It’s also interesting to compare these two images; Notice how the schools in some locations are relatively strong (shaded black) or weak (shaded white) in both visualizations, while others show a particular weakness in one of the metrics.

Selecting a point on the scatterplot shows the corresponding location on the map. In this case, we've highlighted a school that is an outlier because it's relatively strong in math versus its perforrmance in reading, realtive to other schools. We can easily see this school is in Moffat County. (Click the image for a larger view.)

SpatialKey makes it easy to uncover and visualize these relationships, and to share them with others. From uploading the spreadsheet with school data to presentation, this only took a few minutes to create – without any programming or hassle. And, this is just the start. By adding filters we can see these trends for schools of certain sizes or types, or compare these trends over time.

Further Analysis

An interesting next step would be to see if there is any correlation between the areas that deviate from the norm school performance and property value changes. For example, are the “rising up” areas ones where real estate values have been growing faster than average, or gentrification is taking place. (Of course, determining causality is a whole different conversation!) One could bring additional real estate or demographic data into SpatialKey to help answer those questions. SpatialKey makes it easier to understand the relationships between disparate datasets.

Try it out for yourself

Don’t take our word for it. You can start uploading your own data and visually correlating it right away by signing up for the 30-day trial of SpatialKey. Or, contact us and we’ll be happy to walk you through the process.

del.icio.us:Reading and writing and ... location.  Visualizing where different peformance metrics correlate. digg:Reading and writing and ... location.  Visualizing where different peformance metrics correlate. spurl:Reading and writing and ... location.  Visualizing where different peformance metrics correlate. newsvine:Reading and writing and ... location.  Visualizing where different peformance metrics correlate. furl:Reading and writing and ... location.  Visualizing where different peformance metrics correlate. reddit:Reading and writing and ... location.  Visualizing where different peformance metrics correlate. Y!:Reading and writing and ... location.  Visualizing where different peformance metrics correlate.

Comparing Thematic Maps with Density Heatmaps

4 February 2010

Now that we’ve rolled out thematic mapping by state, county, and zip code in SpatialKey, you can produce some fantastic thematic maps with only a few mouse clicks. But it’s important to understand how these thematic maps represent your data, and when it might be appropriate to use thematic maps versus density maps. Both are useful, and SpatialKey makes switching between the two methods easier than it has ever been before.

We’ll compare a zip-code thematic map with a heatmap. Both maps show average home sale price by geographic area (either zip codes or clusters of points). The image below shows the two map types side by side.
thematic_heatmap_comparison

Now we’ll step through an analysis of these different map types to see why they produce different views of the same data.

Thematic map by zip code

First, let’s take a look at mapping home sales in Sacramento by zip code. The map below shows thematic zip codes colored by the average sale price. You can see the highest range is $400,000 and up and includes 3 zip codes in the image below. I want to focus on comparing the two labeled zip codes, 95818 and 95822. You can see that the 95822 zip code area has a much lower average sale price than 95818, which is immediately north of it.

sacramento_prices_zip_thematic

Density heatmap with zip-code boundaries

However, if we switch to a density heatmap we see a different picture. Switching from thematic zip codes to a density map takes literally 3 clicks in SpatialKey. The map below shows average sale price as a density map, with the boundaries of the zip codes overlaid in red. This is the exact same data showing the exact same attribute (home sales showing average sale price). But if you compare this image with the thematic map above you’ll notice that the hotspots tell a different story. A fluid area that overlaps both the zip codes we looked at above is actually the area with the high average prices. That area doesn’t cleanly fall into a single zip code.

sacramento_prices_heatmap_w_zip

This isn’t too shocking, since it intuitively makes sense that fairly arbitrary boundaries like zip codes wouldn’t directly map to more or less expensive areas of town. But it illustrates the difficulty of rendering your data thematically by certain shapes, like zip codes or counties.

Density heatmap with neighborhood boundaries

To further analyze the dataset I decided to load in the boundaries of the neighborhoods in Sacramento (the file was downloaded here). Now we see boundaries that come much closer to matching the home prices. Intuitively this also makes sense; if you think about home prices in your city you’ll likely think of expensive and cheap neighborhoods, not zip codes.

sacramento_prices_heatmap_w_neighborhoods

Everything has its place

Both thematic maps and density maps are useful when exploring geographic data. Both show you important aspects of your data, but it’s important to keep in mind the inherent limitations of the different methods. With SpatialKey, we provide you with the tools to easily switch back and forth between these rendering methods in seconds.

Try it out for yourself

You can start uploading your own data and making thematic maps right away by signing up for the 30-day trial of SpatialKey.

del.icio.us:Comparing Thematic Maps with Density Heatmaps digg:Comparing Thematic Maps with Density Heatmaps spurl:Comparing Thematic Maps with Density Heatmaps newsvine:Comparing Thematic Maps with Density Heatmaps furl:Comparing Thematic Maps with Density Heatmaps reddit:Comparing Thematic Maps with Density Heatmaps Y!:Comparing Thematic Maps with Density Heatmaps

Visual mapping and analysis for “regular” business users?

1 February 2010

We all know that a picture is worth a thousand words. Images from Tiananmen Square, September 11th, or the recent devastation in Haiti are universally understood and move people to action more than words ever could. Visualizing vs. reading about events is becoming more and more prevalent, with an increasing number of people receiving their information from the web or cell phone. In parallel with the upsurge in use of images and multimedia content to communicate information, the advent of Google Earth, online maps, or car and phone navigation tools has created an explosion in the use of visual maps in every day life. Instead of reading text, we are now provided maps to more easily see how to get from point A to point B, or where to find open homes in a specific neighborhood. For most of us, seeing is understanding and believing.

Photo courtesy of CNN and Google maps.

On the business side, 80% of business data has a location component which provides a goldmine of untapped information for marketing, sales and operations. But current visual mapping and analysis tools are expensive, can only be accessed by trained specialists, and require heavy IT involvement to set up and maintain. This is a big barrier to entry for most businesses. They want to “see”, understand and communicate data trends, but don’t have the time nor means to invest in yet another expensive infrastructure.

The businesses that already do leverage visual mapping and analysis can more effectively and more quickly see geographic or time-based data and trends critical to sales and operations. This provides them a real competitive advantage. Many oil and gas companies for example have invested in sophisticated Geographic Information Systems (GIS) and brought in GIS specialists to gain insight on their location intelligence via visual maps. This allows them not only to plot areas with the highest potential to drill in, but also better manage their pipelines, operations, retail facilities, and more.

…. or….    . …and…            … and..     

Thankfully, a revolution is taking place that allows “regular” business users -with no GIS training nor deep pockets-  to leverage the power of visual mapping and analysis. Enter Software as a Service (SaaS). SaaS is transforming mapping and data visualization in the business world the same way Google Maps revolutionized mapping for consumers. Using cost-effective, user friendly SaaS mapping and analysis applications, such as SpatialKey, organizations of all types and sizes can now import their business data, combine it with geographic or competitive information, and start visually analyzing trends critical to their business. Where are key customers located? How can they maximize results in their sales territories? How best to map their sales territories? Where should they open a new retail outlet? How does Q2 sales compare to Q1 on a geographic basis? What marketing campaign resulted in the highest ROI? And so much more.

Opportunities and threats previously hidden within row and column-based datasets are now clearly visible via interactive maps. Concepts difficult to explain in text or PowerPoint presentations can now also be shown and therefore easily understood resulting in better decision making. What’s more, since everyday decision makers can use these applications, “what if” questions can be answered on the fly versus having to wait for an analyst to do a new data query. Decision-making, communication, and collaboration are improved. After all, seeing is understanding and believing, even in the business world.

Note: we’ll be adding blog posts around visual mapping for sales and marketing users over the next few weeks. In the meantime you can find out more at our sales and marketing and/or enterprise solutions pages.

del.icio.us:Visual mapping and analysis for  digg:Visual mapping and analysis for  spurl:Visual mapping and analysis for  newsvine:Visual mapping and analysis for  furl:Visual mapping and analysis for  reddit:Visual mapping and analysis for  Y!:Visual mapping and analysis for