We mapped the concentration of where these permits were issued. The different types of buildings, such as apartments versus office buildings, have very different distributions throughout the city. Some of these distributions are expected, such as the high concentration of permits for offices in the downtown area of San Francsico. But some of the distributions are more interesting and tell a story about the urban makeup of the city. Notice that apartments are much more concentrated closer to downtown in neighborhoods like the Tenderloin, Nob Hill, and Hayes Valley, most heavily around the eastern and northern areas surrounding the financial district. Two-family homes (ie duplexes) have a different concentration that includes neighborhoods like Cow Hollow and the Mission. And one-family homes are in neighborhoods like Pacific Heights, Noe Valley, and Twin Peaks.

The maps here show the number of building permits by the type of building.

Here are some alternate screenshots that are at a more granular resolution, so you can see a bit more detail on the different areas of the city. Click each thumbnail for a much larger version:

Try it for yourself

You can open up the sample report that we created to visualize these building permits.  The report will load with two layers: the building permits and the neighborhood boundaries of San Francisco. You can change which types of building permits are shown by selecting items in the “Proposed Use” filter pod that is open in the report.

• Try Sample Report

Looking for easy-to-use location intelligence from your own data?  Get started with our free trial, and start visualizing your data in minutes without installing any software.

Notes on the data

It’s always important to remember what data we’re looking at. This is the number of building permits issued between September 2008 and August 2009. A single building might have multiple permits issued, which could be everything from renovations or re-roofing to a change from residential to commercial, etc.

This is the third part of an ongoing exploration of publicly accessible San Francisco data from DataSF.org. Please see the other posts in the series.

The images and reports in this post were created with publicly accessible data. We have no association with the city of San Francisco (but we’d love to, so contact us if you’re from the San Francisco government and want to use SpatialKey).

It’s amazing to see how concentrated this data is around the Tenderloin and a few other areas (along Mission St near 16th also has a high concentration). Take a moment and look at the image above (click to get a larger view). If you were to draw a line around the worst parts of San Francisco what would that shape look like? Maybe something like this?

What’s amazing is that this shape was drawn by the city when they defined an “Urban Renewal Area.” The shape is available to download from the city’s GIS catalog. That shape was defined in 2002 as part of a federal Housing and Urban Development (HUD) program. One of the key goals in targeting that areas was to reduce crime. The following paragraph defines one of the five stated goals when the area was defined:

Reducing Crime
The City of San Francisco has a crime reduction strategy that is a multi-layered approach addressing: general crime prevention education; programs targeting at-risk youth (prevention); diversion programs; programs targeting incarcerated individuals to reduce recidivism; and post-incarceration programs to reduce recidivism.

That was almost 8 years ago, but the crime data in the image above is as recent as last week. There are of course many ways to interpret this data. It could indicate that the city is not making much progress reducing drug crimes within the Urban Renewal Area. Or it could mean that the city is actively targeting the area (which will lead to higher arrests) and that the indication of higher concentrations in the Urban Development Area is a good thing.

Comparing the Urban Renewal Area to the rest of SF

To put things in perspective we can compare the crime within the Urban Renewal Area with all the crime outside of the area. The image below shows the map filtered to include only crimes within the Urban Renewal Area. The report includes the total count of crimes within that area, as well as the total count of crimes in the city as a whole (including the area).

We can see that in the entire city there were 14,653 crimes. Out of those crimes, 4,291 occurred in the Urban Renewal Area.

29% of all crime in SF occurs in the Urban Renewal Area

The above statistic is for all types of crime (of which theft is the most common in the city overall). We can see by the distribution by crime type that there’s a difference in the ranking between the city as a whole and the Urban Renewal Area. Inside the Urban Renewal Area, narcotics crimes are the number one crime type (versus theft in SF as a whole). The report is even more interesting when we filter to only look at drug crimes:

Out of 2,673 total drug crimes, 1,875 occur in the Urban Renewal Area.

70% of all narcotics crimes occur in the Urban Renewal Area

Again, please keep in mind that the data only shows where the police are arresting people (see notes section below). And this is only one slice of time, from July 25 to September 20, so we don’t have the previous historical data to see any kinds of long term historical trends.

Try it for yourself

You can open up the sample report that we created to visualize these crimes. The report loads with the crime data on the map filtered to only include drug crimes within the Urban Renewal Area. It also includes another instance of the same crime dataset loaded in (but not shown on the map). That second dataset is used to show the total count of all crimes in San Francisco as well as the distribution by type of crime for the entire dataset.

• Try Sample Report

Notes on the data

The police department provides a feed of the last 90 days of crime incidents as a shapefile. The dataset used in the article was downloaded on September 21, 2009 and includes crime records from June 25, 2009 to September 20, 2009. Additionally, the city’s GIS department provides the shapefile for the Urban Renewal Area.

It is important to note that this is data of active police activity (ie arrests, citations). This means it is not a comprehensive dataset of where crime occurs. This is data about where the SFPD is arresting people. This is an important distinction to keep in mind. An increase in the number of arrests in a certain area may very well be a very good thing indicating that the police are focusing on certain problematic areas, not necessarily that actual crime in those areas is on the rise.

This is the second part of an ongoing exploration of publicly accessible San Francisco data from DataSF.org. Please see the other posts in the series.

The images and reports in this post were created with publicly accessible data. We have no association with the San Francisco police department (but we’d love to, so contact us if you’re from the SFPD and want to use SpatialKey).

The following video walks through how quickly we were able to use SpatialKey to filter down over 14,000 crime records from the past 90 days to isolate these exact drug crimes. By the end of this short 5-minute video we produce a list of the schools that have these particular narcotics incidents near them. This is an example of using the powerful proximity filtering in SpatialKey to filter one dataset by buffering around another, as well as filtering by type of crime, day of week, and hour of day.

Data sources

Both the crime data and the school locations were downloaded from San Francisco’s new data warehousing website: datasf.org. The police department provides a feed of the last 90 days of crime incidents as a shapefile. The dataset used in the video was downloaded on September 21, 2009 and includes crime records from June 25, 2009 to September 20, 2009. Additionally, the city’s GIS department provides the files for the locations of all the private and public schools in San Francisco (both as shapes and as points).

How we did it: step-by-step

I’ll walk through a visual explanation of how we filtered down the 14,653 crimes. These steps are the ones you see performed in the video. You can click each screenshot thumbnail on the right to view the full screenshot.

Step 1: Load in the crime data

We imported the shapefile that we downloaded from the SFPD’s website into SpatialKey and opened a new full-screen map template. This initial map shows all 14,653 incidents. The heatmap shows the distribution of crimes in the city. It is very clear that the most crime activity occurs downtown.

Step 2: Buffer around the schools

We then brought in our second dataset, which was the shapes of all the public and private schools in San Francisco. We used the Proximity Filter capabilities of SpatialKey to create a 1,000 foot buffer around each school and only include the crimes within that buffer. You can see the filter in action and how it removed all the crimes that are not near schools. This filtered out approximately 6,000 crimes and brought our total count down to 8,417.

Step 3: Only show drug crimes

Since we were only interested in narcotics crimes, we opened up a pod to show the type of crime (this was a field called “Category” in the data provided by the SFPD). We filtered out all crimes except “Drug/Narcotics,” which left only 1,849 crimes.

Step 4: Filter by day of week and hour of day

Then we filtered by Day of Week to only include crimes occurring Monday through Friday. We also opened up a histogram showing the distribution by hour of day and we selected a range from 7 am to 4 pm. This is an approximation of school hours. Obviously all schools will have slightly different exact hours of operation.

Step 5: Only show heroin/crack/meth

The Juvenile Drug Trafficking and Schoolyard Act is specifically for certain types of drug crimes, so we needed to filter out the other drug crimes that we weren’t interested in. Luckily, the SFPD provides a field in the dataset called “Description” that provides more granular detail about the type of crime, including what type of drug was involved. We filtered out any crimes that did not include the sale of heroin, crack cocaine, and methamphetamine. This brought our total dataset down to 166 crimes.

So in literally under 4 minutes we were able to go from over 14,000 crimes to a very targeted list of 166 and from 243 schools down to a list of 21 that we would want to act on.

Try it for yourself

You can open up the sample report that we created to visualize these crimes. We saved the report with all the filters in place (buffered around schools, during school hours, only certain drug crimes). You can modify these filters if you want to see how the filtering in SpatialKey works. The report is fully interactive, so you can really explore the crime activity in San Francisco.

• Try Sample Report

Notes

This data included crimes during the summer when it is likely that most schools are not in session. The purpose of this example is not to find the exact crimes that occurred during the actual school hours, but instead to find the geographic locations where crimes tend to occur during school hours. The areas we highlighted tend to have drug crimes occur at the right time of day, so we are extrapolating that during the school year this trend would continue.

It is also important to note that this is data of active police activity (ie arrests, citations). This means it is not a comprehensive dataset of where crime occurs. This is data about where the SFPD is arresting people. This is an important distinction to keep in mind. An increase in the number of arrests in a certain area may very well be a very good thing indicating that the police are focusing on certain problematic areas, not necessarily that actual crime in those areas is on the rise.

The video, images, and reports in this post were created with publicly accessible data. We have no association with the San Francisco police department (but we’d love to, so contact us if you’re from the SFPD and want to use SpatialKey).