That’s the business SpatialKey is in. We’re allowing people who in the past may not have been on the front lines of data analysis to take their spreadsheets, CRM data, salesforce.com data and more, map it, analyze it via simple click and drag tools, and extract meaning from it. Our goal is to help people -regular business people, not just data specialists- ask questions of their data, understand trends, and make better decisions, faster.

Police report

The great advantage of working at SpatialKey is that our clients come from such a diverse variety of organizations- each with totally different problems to address, and data analysis needs. We’re helping agencies better understand war zones and address conflict, allowing pharmaceutical companies analyze vaccine distribution efficiencies, or sales departments gain deeper insights into their sales funnel. Our partner Social Compact and the Citi Foundation recently launched a new service based on SpatialKey to help drive investments to undeserved communities. We’re even helping police departments get more bad guys off the streets by allowing them to better understand and address crime patters. The list goes on. Right now for example,  I’m working on a case study with a client in the energy sector. They are a publicly traded company that buys excess electricity from schools, businesses, etc and sells it back to utilities or grid operators when energy use spikes. In their case, they use SpatialKey to better monitor and analyze energy capacity and therefore more quickly respond to energy need changes. SpatialKey provides them insight that they could only dream of in the past. All that with no programming, nor expensive hardware, training or software investment.

Marketing team report

The common thread among all our clients is that they have tons of data, most of it coming from a variety of sources. They know that they could do a better job putting the pieces of the puzzle together to make better decisions, but they don’t have the energy nor budget to invest in complicated tools. Each one of our clients were amazed when they saw their first demo of SpatialKey. They suddenly saw their data like never before and could not believe such a simple solution had so much power. The power they needed to drill down into their data and make better decisions.

Crisis mapping

The other thread we’re seeing is that once SpatialKey is used within an organization, it does not take long for others within the same company to want to start using it too. Take the example of our energy client. They immediately saw that they could use SpatialKey to track not only kilowatts, but also leads, prospects and more.

Our engineering team is hard at work developing new functionalities into SpatialKey. We all know that the future is about more data, not less, so we want to provide the tools needed to allow our users to see the pins in the haystack that will help guide their decisions. Looking forward to seeing what other client uses are coming our way. Want to find out more? Go to Spatialkey.com or contact us.

The first few rows of the Sacramento dispatch export

It took just a couple of minutes to import one of these files into SpatialKey, where we can produce rich interactive maps and reports related to dispatch activity in Sacramento.  The file from August 2009 contained about 30,600 records with location information.  Dispatches for Disturbance-Noise were the 7th most common type of dispatch in August.

Heatmap of the 1079 Noise Disturbances in Sacramento during August 2009.

By using the Temporal Heat Index and Timeline to inspect the date and time of occurrence, we get a better picture of when Disturbance-Noise calls occurred.  The Temporal Heat Index summarizes the number of crimes by hour of day and day of week.  Notice the dispatch volume is generally highest late at night – especially on Saturday and Sunday.

Most dispatches related to noise happen late at night on weekends.

So where are these early morning disturbances?  Simply select day/hour grids of interest and zoom in to see the detail.  Here’s a look at noise disturbances southeast of Capitol Park between 1 and 3am on weekends in August:

Southeast of Capital Park might be a good place to party but a hard place to sleep on weekends.

Try it for yourself

You can open up the sample report that we created to visualize these dispatches.  The report is fully interactive, so you can really explore the dispatch activity in Sacramento.  We saved the report with a filter for Disturbance-Noise. Try modifying this filter – and adding others – to see how SpatialKey works.  Interested in seeing where and when the 338 Drunk Suspects were encountered?  Or the 27 Shooting into Inhabited Dwellings?  Maybe you want to see where and when the 2246 Subject Stops occurred.  It’s simple with SpatialKey.

• Try Sample Report

Looking for easy-to-use location intelligence from your own data?  Get started with our free trial, and start visualizing your data in minutes without installing any software.

Notes

The images and reports in this post were created with publicly accessible data.  Check out Sacramento’s dispatch page to see their notes about what data is included.  We omitted data without address location in our SpatialKey report.

We have no association with the Sacramento police department (but we’d love to, so contact us if you’re from the Sacramento PD and want to use SpatialKey). SpatialKey does have specific features designed for Law Enforcement.

It’s amazing to see how concentrated this data is around the Tenderloin and a few other areas (along Mission St near 16th also has a high concentration). Take a moment and look at the image above (click to get a larger view). If you were to draw a line around the worst parts of San Francisco what would that shape look like? Maybe something like this?

What’s amazing is that this shape was drawn by the city when they defined an “Urban Renewal Area.” The shape is available to download from the city’s GIS catalog. That shape was defined in 2002 as part of a federal Housing and Urban Development (HUD) program. One of the key goals in targeting that areas was to reduce crime. The following paragraph defines one of the five stated goals when the area was defined:

Reducing Crime
The City of San Francisco has a crime reduction strategy that is a multi-layered approach addressing: general crime prevention education; programs targeting at-risk youth (prevention); diversion programs; programs targeting incarcerated individuals to reduce recidivism; and post-incarceration programs to reduce recidivism.

That was almost 8 years ago, but the crime data in the image above is as recent as last week. There are of course many ways to interpret this data. It could indicate that the city is not making much progress reducing drug crimes within the Urban Renewal Area. Or it could mean that the city is actively targeting the area (which will lead to higher arrests) and that the indication of higher concentrations in the Urban Development Area is a good thing.

Comparing the Urban Renewal Area to the rest of SF

To put things in perspective we can compare the crime within the Urban Renewal Area with all the crime outside of the area. The image below shows the map filtered to include only crimes within the Urban Renewal Area. The report includes the total count of crimes within that area, as well as the total count of crimes in the city as a whole (including the area).

We can see that in the entire city there were 14,653 crimes. Out of those crimes, 4,291 occurred in the Urban Renewal Area.

29% of all crime in SF occurs in the Urban Renewal Area

The above statistic is for all types of crime (of which theft is the most common in the city overall). We can see by the distribution by crime type that there’s a difference in the ranking between the city as a whole and the Urban Renewal Area. Inside the Urban Renewal Area, narcotics crimes are the number one crime type (versus theft in SF as a whole). The report is even more interesting when we filter to only look at drug crimes:

Out of 2,673 total drug crimes, 1,875 occur in the Urban Renewal Area.

70% of all narcotics crimes occur in the Urban Renewal Area

Again, please keep in mind that the data only shows where the police are arresting people (see notes section below). And this is only one slice of time, from July 25 to September 20, so we don’t have the previous historical data to see any kinds of long term historical trends.

Try it for yourself

You can open up the sample report that we created to visualize these crimes. The report loads with the crime data on the map filtered to only include drug crimes within the Urban Renewal Area. It also includes another instance of the same crime dataset loaded in (but not shown on the map). That second dataset is used to show the total count of all crimes in San Francisco as well as the distribution by type of crime for the entire dataset.

• Try Sample Report

Notes on the data

The police department provides a feed of the last 90 days of crime incidents as a shapefile. The dataset used in the article was downloaded on September 21, 2009 and includes crime records from June 25, 2009 to September 20, 2009. Additionally, the city’s GIS department provides the shapefile for the Urban Renewal Area.

It is important to note that this is data of active police activity (ie arrests, citations). This means it is not a comprehensive dataset of where crime occurs. This is data about where the SFPD is arresting people. This is an important distinction to keep in mind. An increase in the number of arrests in a certain area may very well be a very good thing indicating that the police are focusing on certain problematic areas, not necessarily that actual crime in those areas is on the rise.

This is the second part of an ongoing exploration of publicly accessible San Francisco data from DataSF.org. Please see the other posts in the series.

The images and reports in this post were created with publicly accessible data. We have no association with the San Francisco police department (but we’d love to, so contact us if you’re from the SFPD and want to use SpatialKey).

The following video walks through how quickly we were able to use SpatialKey to filter down over 14,000 crime records from the past 90 days to isolate these exact drug crimes. By the end of this short 5-minute video we produce a list of the schools that have these particular narcotics incidents near them. This is an example of using the powerful proximity filtering in SpatialKey to filter one dataset by buffering around another, as well as filtering by type of crime, day of week, and hour of day.

Data sources

Both the crime data and the school locations were downloaded from San Francisco’s new data warehousing website: datasf.org. The police department provides a feed of the last 90 days of crime incidents as a shapefile. The dataset used in the video was downloaded on September 21, 2009 and includes crime records from June 25, 2009 to September 20, 2009. Additionally, the city’s GIS department provides the files for the locations of all the private and public schools in San Francisco (both as shapes and as points).

How we did it: step-by-step

I’ll walk through a visual explanation of how we filtered down the 14,653 crimes. These steps are the ones you see performed in the video. You can click each screenshot thumbnail on the right to view the full screenshot.

Step 1: Load in the crime data

We imported the shapefile that we downloaded from the SFPD’s website into SpatialKey and opened a new full-screen map template. This initial map shows all 14,653 incidents. The heatmap shows the distribution of crimes in the city. It is very clear that the most crime activity occurs downtown.

Step 2: Buffer around the schools

We then brought in our second dataset, which was the shapes of all the public and private schools in San Francisco. We used the Proximity Filter capabilities of SpatialKey to create a 1,000 foot buffer around each school and only include the crimes within that buffer. You can see the filter in action and how it removed all the crimes that are not near schools. This filtered out approximately 6,000 crimes and brought our total count down to 8,417.

Step 3: Only show drug crimes

Since we were only interested in narcotics crimes, we opened up a pod to show the type of crime (this was a field called “Category” in the data provided by the SFPD). We filtered out all crimes except “Drug/Narcotics,” which left only 1,849 crimes.

Step 4: Filter by day of week and hour of day

Then we filtered by Day of Week to only include crimes occurring Monday through Friday. We also opened up a histogram showing the distribution by hour of day and we selected a range from 7 am to 4 pm. This is an approximation of school hours. Obviously all schools will have slightly different exact hours of operation.

Step 5: Only show heroin/crack/meth

The Juvenile Drug Trafficking and Schoolyard Act is specifically for certain types of drug crimes, so we needed to filter out the other drug crimes that we weren’t interested in. Luckily, the SFPD provides a field in the dataset called “Description” that provides more granular detail about the type of crime, including what type of drug was involved. We filtered out any crimes that did not include the sale of heroin, crack cocaine, and methamphetamine. This brought our total dataset down to 166 crimes.

So in literally under 4 minutes we were able to go from over 14,000 crimes to a very targeted list of 166 and from 243 schools down to a list of 21 that we would want to act on.

Try it for yourself

You can open up the sample report that we created to visualize these crimes. We saved the report with all the filters in place (buffered around schools, during school hours, only certain drug crimes). You can modify these filters if you want to see how the filtering in SpatialKey works. The report is fully interactive, so you can really explore the crime activity in San Francisco.

• Try Sample Report

Notes

This data included crimes during the summer when it is likely that most schools are not in session. The purpose of this example is not to find the exact crimes that occurred during the actual school hours, but instead to find the geographic locations where crimes tend to occur during school hours. The areas we highlighted tend to have drug crimes occur at the right time of day, so we are extrapolating that during the school year this trend would continue.

It is also important to note that this is data of active police activity (ie arrests, citations). This means it is not a comprehensive dataset of where crime occurs. This is data about where the SFPD is arresting people. This is an important distinction to keep in mind. An increase in the number of arrests in a certain area may very well be a very good thing indicating that the police are focusing on certain problematic areas, not necessarily that actual crime in those areas is on the rise.

The video, images, and reports in this post were created with publicly accessible data. We have no association with the San Francisco police department (but we’d love to, so contact us if you’re from the SFPD and want to use SpatialKey).