The great news is that analytical capabilities previously limited to larger companies with deep pockets are now available to a broader range of organizations. Plus, even larger companies are benefiting from new SaaS and cloud-based technologies that allow easier access to data often locked in legacy systems. More and more companies can therefore take a proactive approach to their decision making, an approach based on facts, vs the reactive approaches of the past.

Police report

Take law enforcement. When SpatialKey first launched, we immediately received interest from police departments looking into better approaches to crime fighting. Most already used sophisticated RMS or CompStat data systems, yet those systems did not take into account data from other systems or outside sources. For example gang data might be in one database, crime incidents in RMS, and offender information in a spreadsheet with no easy way to pull the information together to make better policing decisions. What they saw in SpatialKey was a way to get a better ROI on their investments by adding new dimensions to their data analysis capabilities.

With SpatialKey, a commander for example can bring in data from multiple sources and start looking for patterns: there are a string of robberies happening in a neighborhood- how can we curtail these? Who are the parolees or known offenders living within x miles of that area? Is there a specific time of the day that these crimes occur? In the police world, they call this Intelligence-led policing- the ability to leverage intelligence to guide crime fighting decisions. So if the commander sees a pattern in the time when crimes occur, increasing the number of patrol cars in the neighborhood at that time is a fact-based approach that hopefully will generate faster/cheaper results. And all it took was a couple minutes of analysis with SpatialKey vs randomly patrolling the area.

Well data is data is data. What we’ve seen in the past year is that a Sales VPs’ data analysis needs are actually not that different from a police chief’s (or any other decision maker for that matter). At the end of the day, what’s important for organizations is to find ways to pull together information critical to decision making and make it available to the people who need to actually make those decisions. So for one parolees, crime logs, or calls for service are key data points to look into to make fact-based decisions, for the other it might be sales leads, product sales, customer calls or customer demographics. And of course the types of data SpatialKey clients need to analyze is as varied as their industries.

The common thread we’ve found amongst all of our clients is that although they already have data analysis capabilities (most already use some form of BI or GIS system), those systems do not provide them the ad-hoc, “get the answer now” data analysis needs the addition of SpatialKey provides.

So although a Sales VP might be using Salesforce as a great CRM tool, there is no easy way to tie the salesforce data to other internal or external data sources to make better decisions to maximize sales opportunities. By incorporating SpatialKey with Salesforce, he/she can now bring in demographic information or competitive data into the picture. Are there patterns, geographic or otherwise, that can help guide sales staff deployment or lead closing activities? Is there a correlation between deal size and marketing activities? What works best to close leads in this geography vs the other?

In a fast-paced world when we all need an answer yesterday, the benefit they see in SpatialKey is that not only can they leverage knowledge they never had before, but they also don’t need to wait on data analysis gurus to get answers. They can do that analysis themselves- on the fly. Many actually use SpatialKey in meetings so that “what if” questions can be answered immediately. Maybe we should call this Intelligence-led sales…..

Please visit spatialkey.com for more information. We also offer a free trial if you want to take a spin of the app with your own data.

Analyzing conflicts via maps is not new. For centuries generals and politicians have moved pushpins on maps to help guide troop movements, understand enemy positions, or help avert conflict altogether. What’s new to conflict and crisis mapping are the tools now allowing military, politicians, and humanitarian groups better understand what drives political instability and violent conflict, and better address it.

One of the more rewarding uses of SpatialKey has been driven by a team led by Dr Clionadh Raleigh. Dr Raleigh and team members from the PRIO Center for the Study of Civil War created the ACLED (Armed Conflict Location and Event Data) database. The ACLED project team’s objective was to provide a better read of conflicts by understanding the relationships between combatants, social groups, economies, and even natural phenomena such as droughts or floods. Ultimately their goal is to achieve a more stable, just, and peaceful world.

Crisis mapping

Thanks in part to funding from the World Bank, ACLED provides up-to-date, immediately accessible analytics and maps for over 50 countries in the developing world to help academics, the World Bank, NGOs, aid agencies and more gain insights on civil war dynamics. The database includes for example the date and location of conflict events, event types, rebel and other groups involved, as well as specifics on battles, killings, riots, and recruitment activities by rebels, governments, militias, armed groups, protesters and civilians, and much more.

The difficulty of creating a central database for crisis mapping is that it needs to bring together vast amounts of diverse information coming from a wide variety of sources. In technical terms: a data mess. Since neither Dr Raleigh nor the users of the ACLED database are trained GIS (Geographic Information System) professionals, they decided to use SpatialKey to centralize and analyze the data. SpatialKey is web-based and does not require special training or programming. All users need is an internet connection to immediately create highly visual maps and reports.

The benefit of using SpatialKey is that each agency using ACLED data can now create maps to help answer totally different questions, no data specialists required. Some groups need to better understand how to mitigate conflict in a specific area, others want to find the safest zone to place a refugee camp, and yet others want to understand the impact of possible floods and droughts on a conflict so they can arrange their resources accordingly.

This has allowed researchers to analyze data with more precision, as well as create a more collaborative environment to help the researcher community create predictive models of civil war. It has also helped challenge assumptions. For example Dr Raleigh says that many people considered civil wars to be primarily rural events, but SpatialKey has showed that these conflicts tend to happen close to larger cities, as rebel groups attempt to engage with the military. She considers that the combination of ACLED and SpatialKey goes a long way toward advancing the field- it provides the next generation in conflict analysis and crisis mapping.

If you cannot attend Doug’s session, please read our case study on the use of SpatialKey by ACLED.

For more information on SpatialKey, or to start your free trial, please go to spatialkey.com.

That’s the business SpatialKey is in. We’re allowing people who in the past may not have been on the front lines of data analysis to take their spreadsheets, CRM data, salesforce.com data and more, map it, analyze it via simple click and drag tools, and extract meaning from it. Our goal is to help people -regular business people, not just data specialists- ask questions of their data, understand trends, and make better decisions, faster.

Police report

The great advantage of working at SpatialKey is that our clients come from such a diverse variety of organizations- each with totally different problems to address, and data analysis needs. We’re helping agencies better understand war zones and address conflict, allowing pharmaceutical companies analyze vaccine distribution efficiencies, or sales departments gain deeper insights into their sales funnel. Our partner Social Compact and the Citi Foundation recently launched a new service based on SpatialKey to help drive investments to undeserved communities. We’re even helping police departments get more bad guys off the streets by allowing them to better understand and address crime patters. The list goes on. Right now for example,  I’m working on a case study with a client in the energy sector. They are a publicly traded company that buys excess electricity from schools, businesses, etc and sells it back to utilities or grid operators when energy use spikes. In their case, they use SpatialKey to better monitor and analyze energy capacity and therefore more quickly respond to energy need changes. SpatialKey provides them insight that they could only dream of in the past. All that with no programming, nor expensive hardware, training or software investment.

Marketing team report

The common thread among all our clients is that they have tons of data, most of it coming from a variety of sources. They know that they could do a better job putting the pieces of the puzzle together to make better decisions, but they don’t have the energy nor budget to invest in complicated tools. Each one of our clients were amazed when they saw their first demo of SpatialKey. They suddenly saw their data like never before and could not believe such a simple solution had so much power. The power they needed to drill down into their data and make better decisions.

Crisis mapping

The other thread we’re seeing is that once SpatialKey is used within an organization, it does not take long for others within the same company to want to start using it too. Take the example of our energy client. They immediately saw that they could use SpatialKey to track not only kilowatts, but also leads, prospects and more.

Our engineering team is hard at work developing new functionalities into SpatialKey. We all know that the future is about more data, not less, so we want to provide the tools needed to allow our users to see the pins in the haystack that will help guide their decisions. Looking forward to seeing what other client uses are coming our way. Want to find out more? Go to Spatialkey.com or contact us.

CDE's SchoolVIEW

SchoolVIEW data in SpatialKey

I was interested in seeing the correlation between these different metrics, and (since we’re obsessed with location) how that correlation relates to geography. So, I imported that data into SpatialKey.  The source file was a CSV with a row for each school.  Here’s what that data looks like:

SpatialKey’s bivariate renderer allowed me to quickly explore the data in just that manner. The bivariate renderer allows you to select two numeric attributes in your dataset, and an aggregate calculation for each. In the image below, I selected average math growth percentile and average math proficiency.  Each dot in the scatterplot legend at the upper right represents a colored location (grid cell) on the map.  The position of the dot represents its relative score for average math growth (y axis) and average math proficiency (x axis).  The color “behind” the each dot is the color used for the corresponding grid cell on the map.

This visualization shows the coorelation between math proficiency and growth, as it relates to location. (Click the image for a larger view.)

We can see there is a general positive correlation, where most locations have a similar relative score for math performance and growth: Most points on the scatterplot are along an imaginary diagonal line from the bottom left (low in both metrics) to upper right (high in both metrics).  What’s often interesting and informative is to see areas that deviate from the norm in terms of the correlation.  Areas with relatively high proficiency but low growth – “strong but losing ground” – are colored blue, while areas with low proficiency but high growth – “risin’ up” – are colored red.  These are both negative correlations.  Areas that score low on both metrics are shaded white, while those high in both attributes are shaded black – both positive correlations.  For this type of visual analysis, areas that fall toward the middle of both ranges are usually less interesting, and so those colors are more transparent to allow you to focus on the extremes.  It may take a few seconds to orient yourself to this view, but once acclimated it’s a powerful way to visualize some complex – and otherwise difficult to express – relationships.

You can correlate any pair of attributes by simply selecting from one of the axes in the scatterplot legend. This next image compares average math and reading proficiency.  First, notice there seems to be an even stronger correlation between these two variables than the previous set.  (The points line up even closer on the imaginary diagonal line.)   It’s also interesting to compare these two images; Notice how the schools in some locations are relatively strong (shaded black) or weak (shaded white) in both visualizations, while others show a particular weakness in one of the metrics.

Selecting a point on the scatterplot shows the corresponding location on the map. In this case, we've highlighted a school that is an outlier because it's relatively strong in math versus its perforrmance in reading, realtive to other schools. We can easily see this school is in Moffat County. (Click the image for a larger view.)

SpatialKey makes it easy to uncover and visualize these relationships, and to share them with others. From uploading the spreadsheet with school data to presentation, this only took a few minutes to create – without any programming or hassle. And, this is just the start. By adding filters we can see these trends for schools of certain sizes or types, or compare these trends over time.

Further Analysis

An interesting next step would be to see if there is any correlation between the areas that deviate from the norm school performance and property value changes. For example, are the “rising up” areas ones where real estate values have been growing faster than average, or gentrification is taking place. (Of course, determining causality is a whole different conversation!) One could bring additional real estate or demographic data into SpatialKey to help answer those questions. SpatialKey makes it easier to understand the relationships between disparate datasets.

Try it out for yourself

Don’t take our word for it. You can start uploading your own data and visually correlating it right away by signing up for the 30-day trial of SpatialKey. Or, contact us and we’ll be happy to walk you through the process.

Photo courtesy of CNN and Google maps.

On the business side, 80% of business data has a location component which provides a goldmine of untapped information for marketing, sales and operations. But current visual mapping and analysis tools are expensive, can only be accessed by trained specialists, and require heavy IT involvement to set up and maintain. This is a big barrier to entry for most businesses. They want to “see”, understand and communicate data trends, but don’t have the time nor means to invest in yet another expensive infrastructure.

The businesses that already do leverage visual mapping and analysis can more effectively and more quickly see geographic or time-based data and trends critical to sales and operations. This provides them a real competitive advantage. Many oil and gas companies for example have invested in sophisticated Geographic Information Systems (GIS) and brought in GIS specialists to gain insight on their location intelligence via visual maps. This allows them not only to plot areas with the highest potential to drill in, but also better manage their pipelines, operations, retail facilities, and more.

…. or….    . …and…            … and..     

Thankfully, a revolution is taking place that allows “regular” business users -with no GIS training nor deep pockets-  to leverage the power of visual mapping and analysis. Enter Software as a Service (SaaS). SaaS is transforming mapping and data visualization in the business world the same way Google Maps revolutionized mapping for consumers. Using cost-effective, user friendly SaaS mapping and analysis applications, such as SpatialKey, organizations of all types and sizes can now import their business data, combine it with geographic or competitive information, and start visually analyzing trends critical to their business. Where are key customers located? How can they maximize results in their sales territories? How best to map their sales territories? Where should they open a new retail outlet? How does Q2 sales compare to Q1 on a geographic basis? What marketing campaign resulted in the highest ROI? And so much more.

Opportunities and threats previously hidden within row and column-based datasets are now clearly visible via interactive maps. Concepts difficult to explain in text or PowerPoint presentations can now also be shown and therefore easily understood resulting in better decision making. What’s more, since everyday decision makers can use these applications, “what if” questions can be answered on the fly versus having to wait for an analyst to do a new data query. Decision-making, communication, and collaboration are improved. After all, seeing is understanding and believing, even in the business world.

Note: we’ll be adding blog posts around visual mapping for sales and marketing users over the next few weeks. In the meantime you can find out more at our sales and marketing and/or enterprise solutions pages.

The first few rows of the Sacramento dispatch export

It took just a couple of minutes to import one of these files into SpatialKey, where we can produce rich interactive maps and reports related to dispatch activity in Sacramento.  The file from August 2009 contained about 30,600 records with location information.  Dispatches for Disturbance-Noise were the 7th most common type of dispatch in August.

Heatmap of the 1079 Noise Disturbances in Sacramento during August 2009.

By using the Temporal Heat Index and Timeline to inspect the date and time of occurrence, we get a better picture of when Disturbance-Noise calls occurred.  The Temporal Heat Index summarizes the number of crimes by hour of day and day of week.  Notice the dispatch volume is generally highest late at night – especially on Saturday and Sunday.

Most dispatches related to noise happen late at night on weekends.

So where are these early morning disturbances?  Simply select day/hour grids of interest and zoom in to see the detail.  Here’s a look at noise disturbances southeast of Capitol Park between 1 and 3am on weekends in August:

Southeast of Capital Park might be a good place to party but a hard place to sleep on weekends.

Try it for yourself

You can open up the sample report that we created to visualize these dispatches.  The report is fully interactive, so you can really explore the dispatch activity in Sacramento.  We saved the report with a filter for Disturbance-Noise. Try modifying this filter – and adding others – to see how SpatialKey works.  Interested in seeing where and when the 338 Drunk Suspects were encountered?  Or the 27 Shooting into Inhabited Dwellings?  Maybe you want to see where and when the 2246 Subject Stops occurred.  It’s simple with SpatialKey.

• Try Sample Report

Looking for easy-to-use location intelligence from your own data?  Get started with our free trial, and start visualizing your data in minutes without installing any software.

Notes

The images and reports in this post were created with publicly accessible data.  Check out Sacramento’s dispatch page to see their notes about what data is included.  We omitted data without address location in our SpatialKey report.

We have no association with the Sacramento police department (but we’d love to, so contact us if you’re from the Sacramento PD and want to use SpatialKey). SpatialKey does have specific features designed for Law Enforcement.

With the abundance of this raw data new challenges arise. How do cities display the information in meaningful ways without complex and costly software?  This data is typically shared in the format of CSV (comma separated value ), spreadsheet or XML files containing many thousands of rows. Extracting meaning from the data can be a daunting task requiring multiple pivot tables, graphs, and filters along with the expertise in doing so.  And assuming you are able to get this far, you will still be left without an easy solution in which to share this data with others.

How do you expose the location based information within the data?  It is possible take a handful of the items in the spreadsheet and plot them in a web based mapping solution but most web based maps fall short in their ability to plot thousands of points in a meaningful way.

With SpatialKey you can take nearly any of these data feeds and transform them into an interactive report in minutes. You don’t need to be a specialist to create and share time- and location-based analyses.

To demonstrate the power and flexibility of SpatialKey an example from the New York Department of Sanitation containing graffiti locations is shown below.  You can find this data in the NYC Data Mine by searching for the keyword “graffiti”.  This dataset contains requests to clean graffiti (other than bridges or highways) received from the public in the last 12 months. They include location information, open and closed dates, and details about the community.  A small snapshot of this data is shown below

Figure 1.0 - Sample of graffiti data from the NYC dataset

It is important to note that SpatialKey was not developed for this specific data and no programming was required to build the reports. It is as simple as exporting a CSV from excel and importing the data into SpatialKey. SpatialKey inspects the data during the import process and detects the data types (text, numbers and dates) and builds a custom user interface tailored to the data structure from the spreadsheet. SpatialKey also handles the geocoding as long as you have address information or X/Y in the data. The import process can be performed with thousands of rows in just a few minutes.

After importing the data a full screen map report is opened as shown in Figure 2.0. The report contains a timeline that highlights the trends of open graffiti reports over the last twelve months and the map highlights hot spots for reported graffiti locations. You can instantly identify these trends and hotspots quickly in SpatialKey then start to drill down to identify additional trends with the filtering tools.

Figure 2.0 - A map and timeline of reported graffiti

In Figure 2.1 a categorical pod for the “status” field in our data is opened and the data is aggregated by the unique statuses in that field.   By clicking on the “Closed” status you can filter out all closed incidents and the map reflects only open and pending incidents. In addition you can switch the timeline to show unfiltered data to see the trend of open/pending vs closed incidents.  Within the stacked bar chart the open incidents are displayed with the filled area and the closed incidents are shown in the unfilled area.

Figure 2.1 - Displaying the trend of open versus closed reports over time

A custom interface to display and filter data can be built in seconds with no programming or development needed. In Figure 3.0 four categorical pods have been added from different fields available in the graffiti dataset these pods can be used for both display and filtering.

Figure 3.0 - Adding pods for several fields from the graffiti dataset, pods can be used for display and filtering

Here are a few other  examples highlighting the power when you combine city data with SpatialKey.

• Crime in San Francisco’s Urban Renewal Area
• Visualizing SFPD’s “Operation Safe Schools”

Spreadsheets come to life and provide new meaning with just a few simple steps in SpatialKey. Try it out yourself with our 30 day trial.

While at ICCM2009, we’ll be discussing some of our work with ACLED (Armed Conflict Location and Events Dataset).  The ACLED project was founded by Dr. Clionadh Raleigh and team members from the PRIO Centre for the Study of Civil War, who set out to create a conflict database that would help answer researchers’ questions.  ACLED was subsequently funded by the World Bank with the aim of better understanding events in client states.

This video gives a brief overview of how SpatialKey is being used to investigate the relationship between this data and the location of refugee camps and environmental factors. What’s significant is that SpatialKey is making it easier for ACLED researchers to collaboratively bring together vast amounts of information from a wide variety of sources in a meaningful way – all without having to involve programmers or GIS experts to get their work done.

A recent Reuters article about trends in crisis mapping highlights the very issue SpatialKey is helping researchers solve:

Researchers have used maps to visualise crises for many years.

But there are drawbacks in the the use of highly-sophisticated, computerised Geographical Information Systems (GIS), which are usually used in such work — not least that they are expensive and difficult to operate.

Nor do these systems allow for much integration and collaboration, and due to their complexities they are not usually updated in real time.

For more details on our work with ACLED, read the ACLED crisis mapping case study.

It’s amazing to see how concentrated this data is around the Tenderloin and a few other areas (along Mission St near 16th also has a high concentration). Take a moment and look at the image above (click to get a larger view). If you were to draw a line around the worst parts of San Francisco what would that shape look like? Maybe something like this?

What’s amazing is that this shape was drawn by the city when they defined an “Urban Renewal Area.” The shape is available to download from the city’s GIS catalog. That shape was defined in 2002 as part of a federal Housing and Urban Development (HUD) program. One of the key goals in targeting that areas was to reduce crime. The following paragraph defines one of the five stated goals when the area was defined:

Reducing Crime
The City of San Francisco has a crime reduction strategy that is a multi-layered approach addressing: general crime prevention education; programs targeting at-risk youth (prevention); diversion programs; programs targeting incarcerated individuals to reduce recidivism; and post-incarceration programs to reduce recidivism.

That was almost 8 years ago, but the crime data in the image above is as recent as last week. There are of course many ways to interpret this data. It could indicate that the city is not making much progress reducing drug crimes within the Urban Renewal Area. Or it could mean that the city is actively targeting the area (which will lead to higher arrests) and that the indication of higher concentrations in the Urban Development Area is a good thing.

Comparing the Urban Renewal Area to the rest of SF

To put things in perspective we can compare the crime within the Urban Renewal Area with all the crime outside of the area. The image below shows the map filtered to include only crimes within the Urban Renewal Area. The report includes the total count of crimes within that area, as well as the total count of crimes in the city as a whole (including the area).

We can see that in the entire city there were 14,653 crimes. Out of those crimes, 4,291 occurred in the Urban Renewal Area.

29% of all crime in SF occurs in the Urban Renewal Area

The above statistic is for all types of crime (of which theft is the most common in the city overall). We can see by the distribution by crime type that there’s a difference in the ranking between the city as a whole and the Urban Renewal Area. Inside the Urban Renewal Area, narcotics crimes are the number one crime type (versus theft in SF as a whole). The report is even more interesting when we filter to only look at drug crimes:

Out of 2,673 total drug crimes, 1,875 occur in the Urban Renewal Area.

70% of all narcotics crimes occur in the Urban Renewal Area

Again, please keep in mind that the data only shows where the police are arresting people (see notes section below). And this is only one slice of time, from July 25 to September 20, so we don’t have the previous historical data to see any kinds of long term historical trends.

Try it for yourself

You can open up the sample report that we created to visualize these crimes. The report loads with the crime data on the map filtered to only include drug crimes within the Urban Renewal Area. It also includes another instance of the same crime dataset loaded in (but not shown on the map). That second dataset is used to show the total count of all crimes in San Francisco as well as the distribution by type of crime for the entire dataset.

• Try Sample Report

Notes on the data

The police department provides a feed of the last 90 days of crime incidents as a shapefile. The dataset used in the article was downloaded on September 21, 2009 and includes crime records from June 25, 2009 to September 20, 2009. Additionally, the city’s GIS department provides the shapefile for the Urban Renewal Area.

It is important to note that this is data of active police activity (ie arrests, citations). This means it is not a comprehensive dataset of where crime occurs. This is data about where the SFPD is arresting people. This is an important distinction to keep in mind. An increase in the number of arrests in a certain area may very well be a very good thing indicating that the police are focusing on certain problematic areas, not necessarily that actual crime in those areas is on the rise.

This is the second part of an ongoing exploration of publicly accessible San Francisco data from DataSF.org. Please see the other posts in the series.

The images and reports in this post were created with publicly accessible data. We have no association with the San Francisco police department (but we’d love to, so contact us if you’re from the SFPD and want to use SpatialKey).