Background

Massachusetts is a small state, but it has 351 cities and towns. The large number of jurisdictions grants the state its incredible civic diversity, ranging from rural villages to small towns, to beach towns, to metropolises.

But the multiplicity of jurisdictions also results in an extremely tricky coordination problem: although it is broadly understood that the state’s real estate and housing prices are too high, no individual jurisdiction wants to permit housing supply sufficient to accommodate the demand, and so housing prices rise further. The high degree of mobility between cities and towns, particularly in the Boston Metro area surrounded by I-93, I-95, and I-495, means that residents can easily relocate if their city government implements any policy they are unhappy with (for instance because they believe it threatens the long-term value of their home).

Among the reasons given for low permitting are that older residents worry population growth will require increased taxation to fund local schools (ie they will have to pay more for a public good they will not consume) and that white residents oppose new units because they worry that the new entrants might not be white. Another explanation is that owners of valuable property don’t like new construction because they fear the units will be small, requiring the new owners to pay less property tax. I wondered how valid these explanations were in explaining permitting for new units around the state.

Research Question

How do demographic factors interact with permitting in cities and towns across Massachusetts?

I explored four hypotheses:

1. Permits issued will be negatively correlated with higher percentages of residents identifying as white
2. Permits issued will be negatively correlated with higher median age
3. Permits issued will be negatively correlated with median home price
4. Global autocorrelation statistics will identify clustering across the state and local autocorrelation statistics will show high clusters of permitting in high age low clusters in low-age areas

Data

Dependent variable: Perm_pthou – Permits issued per thousand residents (continuous variable)

Independent variables:

• Median home price
• Percentage of residents identifying as white only
• Median Age

Geography: TIGER/Line Shapefile, 2015, state, Massachusetts, Current County Subdivision State-based

Exploratory Data Analysis

First, I created a series of maps to explore trends in the dependent variable, Permits issued per thousand residents. I started with a Natural breaks (6 categories) choropleth (Figure 1).

There were interesting variations on the map, but they weren’t in the places I expected. I was expecting to see a lot of permitting in the Boston metro area and around some of the second-tier cities like Worcester, Springfield, Lowell, and Fall River. Instead, all of those regions had moderate-to-low permitting activity, while permitting per thousand residents was much higher in areas of Cape Cod and the Islands, as well as some of the more suburban areas of Springfield and Worcester.

I was curious about these unexpected outliers, so I produced a standard deviations map (Figure 2). The standard deviations map makes it even clearer where the areas of high permitting are. Among them (in red) are Middlefield in Western, MA; Westminster in north-central MA; Hopkinton on the outskirts of Worcester; and Arlington on the outskirts of Boston; Lakeville and Plymouth on the South Shore, and various communities on the cape and islands as mentioned above. These are not cities that most people would think of as producing relatively high numbers of units; indeed most people in Massachusetts would not even recognize some of the names of these cities and towns. Of course, we need to keep in mind that the metric is “permits per thousand residents,” so the actual quantity of units produced in these towns might not be large.

Next, I moved on to exploring the dependent variables. First, I examined median home price (Figure 3).

The map produced some interesting patterns. As I expected, high median property values cluster in the Boston Metro area. But interestingly, the highest median property values are not in Boston proper, but rather in the bedroom suburbs of metro-west, like Newton and Lexington. The beach towns of the Cape and Islands and North Shore also have high property values. Property values are lower in Western Massachusetts, although there is a group of towns in Western Massachusetts (highlighted in red) that roughly corresponds to some high permitting towns in the previous maps; these are the college towns of Northampton and Amherst (and neighboring towns). There is another group of high property value towns in the southeast corner of the state, an area known for being remote but picturesque.

Next, I used a standard deviation map to examine the areas by percentage white (Figure 4). This map generally confirmed my prior understanding of the distribution of race in the state: extremely high percentages of white residents in the western half of the state, interrupted by cities like Pittsfield and Springfield. Per the boxplot, MA cities and towns have high proportions of white residents. The Eastern part of the state is much more diverse, with relatively high proportions of nonwhite residents clustered in Boston and the “gateway cities” which include Lowell, Lawrence, Worcester, and Fall River.

Finally, I created a standard deviation choropleth for age. I was not surprised to see clusters of youth in many of the same dense urban areas with higher nonwhite populations. What was somewhat interesting is that there were areas of central Massachusetts and the Cape and Islands where median age and housing permitting were both high.

This was not what I was expecting.

Multi-Dimensional Analysis

I started my multidimensional analysis by running a series of regressions of my dependent variable on the independent variables. The regressions revealed that while the percentage of white residents did not have a significant effect on permitting, both Median age and median value did have significant effects at the p<0.01 level. What’s more, contrary to my hypothesis, their effect was positive. Every 1-year increase in median age was associated with a 0.071 increase in housing permits issued. Every $1,000 increase in median house price was associated with a 0.004 increase in the number of housing permits issued (technical note: because Geoda couldn’t give me the precision I needed, I defined a new variable to generate this estimate: the median home value divided by 1,000).

• 

• 

• 

Seeing my counterintuitive finding regarding the positive relationship between median age and permits per thousand residents by a regression, I wanted to learn more about that relationship. I selected some of the permitting outliers to learn more, which highlighted some scattered communities in Central MA and several communities in the Cape and Islands region (Figure 7). 

• 

• 

To learn more, I created a conditional plot with standard deviation choropleths (Figure 8). The plot revealed that the areas where age and permitting were highest were in non-urban areas of Western MA and the Cape. Meanwhile, the dense cities actually featured both the youngest populations and lowest permitting rates, suggesting that urban areas simply aren’t producing housing commensurate with their population sizes and the young people trying to move in.

Figure 8: Median Age on the horizontal axis, Permits per Thousand Residents on the vertical

Global Autocorrelation

Having identified a potentially interesting relationship between age and permitting in my multivariate analysis, I now began to examine clustering more closely for the two variables. My exploratory data analysis maps were highly suggestive of clustering, but I now sought to use Moran’s I to quantify it.

In selecting weights, I felt most interested in distance measures (distance band – 20 mi and K (10) nearest neighbors); with the small areas and irregular shapes of Massachusetts towns, I felt that first-order queen contiguity measures might not be inclusive of nearby towns. Another benefit of distance measures was that they allowed me to include Nantucket in the analysis, which would be an isolate in first-order queen contiguity. However, I did include a second-order queens (exclusive) contiguity, because some of my maps indicated that there might be a relationship between core cities (most notably Boston) and its second-ring suburbs. This was most notable in the median value map, but I wondered whether it might also be true in permitting and age. It is consistent with my anecdotal experience of Boston that the city is surrounded by an inner ring of less “desirable” communities, which are in turn surrounded by the second ring of tony suburbs.

Figures 9-11 give a general idea of the extents of the weights.

• 

• 

• 

Global Moran’s: Each test done with 99,999 permutations

Variable Name	Weight Used	Pseudo P Value	Moran’s I	Z-Score	Mean	SD
Permits per thousand	KNN – 10	0.000010	0.1826	8.3110	-0.0028	0.0223
Permits per thousand	Distance Band 20 mi	0.000010	0.2587	20.8238	-0.0028	0.0126
Permits per thousand	Queens 2nd Order Exclusive	0.000010	0.2305	9.5917	-0.0027	0.0243
Median Age	KNN – 10	0.000010	0.4621	20.5320	-0.0029	0.0226
Median Age	Distance Band 20 mi	0.000010	0.4105	32.6553	-0.0029	0.0127
Median Age	Queens 2nd Order Exclusive	0.000010	0.4358	17.7685	-0.0028	0.0247

For the permits per thousand variable, the global autocorrelation was strongest with the distance band (20 mi) weight. The Queen’s 2^nd order exclusive weight reflected slightly less global clustering. Interestingly, the global autocorrelation was weakest for the geographically restrictive KNN-10 specification.

The Median Age variable showed even more pronounced global autocorrelation. Every specification I looked at produced a Moran’s I in the 0.4-0.5 range. However, while the permitting variable showed stronger autocorrelation for the more geographically inclusive weights, the opposite was true for median age. For age, the global clustering was strongest for KNN 10, somewhat weaker for the Queens 2^nd order exclusive, and weakest for the Distance Band (20 mi) weight.

In summary, both variables show clustering, although it is more pronounced for median age than for permits per thousand.

Local Autocorrelation

To better understand the spatial distribution of this clustering, I applied a series of local autocorrelation measures. In the following tables, I narrowed down the communities of interest by only including communities in those clusters that are significant at the pseudo p<0.01 level in the Distance Band weighting (this specification produced the most inclusive clusters). The 0.01 significance level seemed to produce the right balance of clustering; more stringent filters produced very few clusters. I then found both Moran’s I and Geary’s C for my three weights and note a Y if the community was clustered and significant at the P<0.01 level and Y* if it was significant at the pseudo P<0.001 level. I highlight communities where the clustering whose significance remained robust despite the change in specification.

Low-Low

	KNN – 10 I	KNN-10 C	Distance Band 20 mi I	Distance Band 20 mi C	Queens 2nd Order Exclusive I	Queens 2nd Order Exclusive C
Boston city			Y	Y*
Burlington town			Y	Y
Chelsea city			Y	Y	Y
Cohasset town			Y*
Danvers town			Y	Y
East Longmeadow town	Y	Y	Y	Y
Everett city			Y	Y
Hampden town	Y		Y	Y
Hingham town			Y
Holland town			Y*
Hull town			Y*	Y
Leyden town	Y		Y
Longmeadow town	Y	Y	Y	Y
Lynn city	Y		Y*	Y*
Lynnfield town			Y	Y*
Malden city	Y	Y	Y	Y*
Marblehead town			Y	Y		Y*
Medford city			Y	Y*		Y
Melrose city	Y	Y*	Y	Y*
Monson town			Y*	Y
Nahant town			Y	Y*	Y	Y
Northfield town	Y		Y
Palmer Town city			Y
Peabody city			Y	Y
Quincy city			Y	Y
Reading town			Y	Y
Revere city	Y		Y	Y	Y	Y
Salem city			Y	Y
Saugus town	Y*	Y*	Y*	Y*
Somerville city			Y	Y
Stoneham town			Y	Y*
Wakefield town			Y*	Y*
Wales town			Y	Y
Weymouth Town city			Y
Wilbraham town			Y
Winchester town			Y	Y*
Woburn city			Y	Y

There are two low-low clusters that seem to be robust to the different specifications I used. The first was a cluster of cities in the North Boston Metro area (Lynn, Malden, Melrose, Revere, Saugus, and Nahant). These cities have large populations but not much permitting. The second notable cluster centered on the Springfield suburbs of Longmeadow and East Longmeadow. A possible explanation is that suburbs outside the biggest cities try to keep the amount of permitting low to preserve the low population densities and suburban lifestyles that attracted their residents in the first place.

High-High

	KNN – 10 I	KNN – 10 C	Distance Band 20 mi I	Distance Band 20 mi C	Queens 2nd Order Exclusive I	Queens 2nd Order Exclusive C
Aquinnah	Y*	Y	Y*	Y
Chilmark	Y	Y*	Y*	Y*	Y	Y
Dennis	Y		Y
Eastham		Y	Y	Y	Y	Y
Edgartown	Y*	Y*	Y*	Y*
Harwich	Y		Y		Y	Y
Mashpee			Y
Nantucket	Y*	Y*	Y	Y	Y*	Y*
Oak Bluffs	Y*		Y*			Y
Orleans	Y*		Y*
Tisbury	Y*		Y*		Y*
Wellfleet	Y		Y
West Tisbury	Y*		Y*

The high-high clusters are restricted to the Cape and Islands. This is not unexpected given the mapping exercises above, but it is interesting that the strongest proportional permitting in the state is happening in the Cape, which historically has had small populations (although that has been changing in recent years, as recently installed sewer lines facilitate higher density). Another possible explanation is that many of these permits are for vacation homes; perhaps this increases the number of permits without increasing the census population count, giving the region a big advantage on the independent variable I chose.

High/Low and Low/High (Potential Outliers)

	KNN – 10 I	KNN – 10 C	Distance Band 20 mi I	Distance Band 20 mi C	Queens 2nd Order Exclusive I	Queens 2nd Order Exclusive C
Andover town			Y
Arlington town			Y
Belchertown town			Y
Brewster town	Y*	Y	Y*	Y	Y	Y*
Brimfield town	Y		Y*
Colrain town			Y
Gosnold town	Y	Y	Y*	Y*
Middleton town			Y
North Reading town			Y		Y
Provincetown town	Y*	Y	Y*	Y*
Swampscott town	Y		Y*		Y
Winthrop Town city	Y		Y		Y
Yarmouth town	Y		Y

The local autocorrelations identified several low/high outliers on the cape (Brewster, Gosnold, and Provincetown), indicating that these towns produce lower than expected permitting. It also identified some high/low outliers, towns that were issuing more permits than expected: Swampscott and Winthrop in the North Boston Metro Area and Brimfield, which is an extremely rural town along I-90.

For age, the pseudo P < 0.01 level seemed to be too inclusive. Instead, I filtered to the 0.001 level. In the tables that follow, a Y represents significance at the 0.001 level, and Y* represents significance at the 0.0001 level. In this case, the distance band weight turned out to be extremely inclusive even at the 0.001 significance level, identifying massive high-high clusters in the west end of the state and Cape Cod, and a low-low cluster throughout a large area around Boston, roughly corresponding to area encompassed by I-495. Therefore, in constructing the tables, I limited my reporting to clusters significant in the KNN specification.

Low-Low

	KNN – 10 I	KNN – 10 C	Distance Band 20 mi I	Distance Band 20 mi C	Queens 2nd Order Exclusive I	Queens 2nd Order Exclusive C
Arlington town	Y		Y*	Y*	Y
Belmont town	Y*		Y*	Y*
Boston city	Y*	Y*	Y*	Y*		Y
Brookline town	Y	Y	Y*	Y*	Y
Cambridge city	Y*	Y*	Y*	Y*
Chelsea city	Y*	Y*	Y*	Y*	Y
Everett city	Y	Y	Y*	Y*
Malden city	Y*	Y	Y*	Y*
Medford city	Y*	Y*	Y*	Y*
Melrose city	Y		Y*	Y*
Newton city	Y		Y*	Y*
Revere city	Y		Y*	Y*	Y
Somerville city	Y*	Y*	Y*	Y*
Watertown Town city	Y		Y*	Y*

Age is highly segregated in the state between the relatively young Boston Metro Area and the aging Center-west and Cape. All of the core communities of the Low-Low clusters were in the Boston metro-north area: places like Cambridge, Somerville, Malden, and Brookline: All places with large universities that attract thousands of young people or MBTA access: perfect bedroom communities for young professionals who commute to Boston. Interestingly, the Queens 2^nd Order exclusive returned few significant clusters in the Boston area, meaning that I did not find evidence for clustering between cities and their 2^nd ring suburbs.

High-High

	KNN – 10 I	KNN – 10 C	Distance Band 20 mi I	Distance Band 20 mi C	Queens 2nd Order Exclusive I	Queens 2nd Order Exclusive C
Alford	Y*	Y*	Y*	Y*	Y*	Y*
Barnstable Town	Y		Y*
Becket	Y*		Y*	Y*		Y
Brewster	Y*	Y*	Y*	Y*	Y
Chatham	Y*	Y*	Y*	Y*	Y	Y
Cummington	Y	Y*	Y*	Y*		Y*
Dalton	Y		Y*		Y*
Dennis	Y*	Y*	Y*	Y*	Y
Eastham	Y*	Y*	Y*	Y*	Y*	Y*
Egremont	Y*	Y	Y*	Y*	Y*
Great Barrington	Y*		Y*		Y*
Harwich	Y*	Y*	Y*	Y*
Hinsdale	Y	Y*	Y*	Y*		Y*
Lenox	Y	Y	Y*	Y*	Y*	Y*
Monterey	Y*	Y*	Y*	Y*	Y*	Y*
Mount Washington	Y*	Y*	Y*	Y*	Y
New Marlborough	Y*	Y*	Y*	Y*	Y*	Y*
Orleans	Y*	Y*	Y*	Y*
Otis	Y*		Y*	Y*
Peru	Y	Y	Y*	Y		Y
Provincetown	Y*	Y*	Y*	Y
Richmond	Y*	Y*	Y*	Y*
Sandisfield	Y*		Y*
Sandwich	Y	Y		Y
Sheffield	Y*		Y*	Y*	Y*
Stockbridge	Y*	Y*	Y*	Y*	Y*	Y*
Truro	Y*	Y*	Y*	Y*
Tyringham	Y*	Y*	Y*	Y*	Y*	Y*
Washington	Y	Y	Y*	Y*	Y*	Y*
Wellfleet	Y*	Y*	Y*	Y*
West Stockbridge	Y*		Y*	Y*	Y*	Y
Yarmouth	Y*	Y*	Y*	Y*		Y

There are two large High-High clusters: one along the state’s western border and another on the Cape and Islands. This aligns with my anecdotal understanding of populations dynamics in these regions: Western Mass has been deindustrializing for decades, so many working-class families have left. Meanwhile, the Cape attracts retirees hoping to enjoy their golden years on the beach (as well as vacationers and the extremely wealthy).

Low/High (Potential Outliers)

	KNN – 10 I	KNN – 10 C	Distance Band 20 mi I	Distance Band 20 mi C	Queens 2nd Order Exclusive I	Queens 2nd Order Exclusive C
Heath	Y		Y*
Lee	Y*		Y*	Y*	Y*
Nantucket	Y				Y*	Y*
Pittsfield	Y*		Y*	Y*

Lee and Pittsfield have notably lower populations than their neighbors. I’m not sure why this is true of the small town of Lee, but Pittsfield is the largest city in the western part of the state, which may explain why it has different population dynamics and hosts a younger demographic than its neighbors.

Bivariate Autocorrelation

Finally, I brought permitting per thousand people and median age together for a joint autocorrelation. For joint autocorrelation, I used multivariate Geary’s C at the pseudo p <0.0001 level (which was very close to the Bonferroni bound of 0.0001424).

This specification showed clusters in the Boston Metro area (including Boston, Malden, Cambridge, Chelsea, and Somerville) that the univariate models showed have low permitting and also low median age. Meanwhile, in the far west and Cape, there was also clustering of permitting and age – this time of high permitting rates and also high median age.

The bivariate Geary’s C detected a cluster in Central MA around Warren and West Brookfield, which are both small towns in between Springfield and Worcester. There is also a cluster that my univariate analyses had not picked up southwest of Boston (Norfolk, Medfield, Medway, Plainville, and Wrentham). This was an interesting cluster: brushing the region on a bivariate Moran’s scatterplot suggested this cluster was low on median age, but high on permitting (Figure 16). In other words, these outliers might be the places policymakers should look for best practices of how to increase housing permitting.

Positive

County	KNN – 10	Distance Band 20 mi	Queens 2nd Order Exclusive
Alford	Y	Y	Y
Boston	Y	Y
Cambridge	Y	Y
Chelsea	Y	Y
Concord		Y
Eastham	Y	Y
Edgartown	Y	Y
Everett	Y	Y
Malden	Y	Y
Medfield	Y	Y	Y
Medway	Y	Y
Monterey	Y	Y	Y
Mount Washington	Y	Y
Norfolk	Y	Y
Reading		Y
Richmond	Y	Y
Somerville	Y	Y
Stockbridge	Y	Y	Y
Truro	Y	Y
Tyringham	Y	Y
Warren		Y
West Brookfield		Y
Wrentham	Y	Y

Discussion and Conclusion

This exploration of the predictors of permitting in Massachusetts surprised me. I expected the dense cities to be the biggest producers of housing, but that was not the case. As a result, my hypothesis that young, racially diverse areas would produce the most per capita permits turned out to be wrong. Instead, high age areas like Western MA and the Cape have the highest rates of permitting when adjusted for population.

Hypotheses:

1. Permits issued will be negatively correlated with higher percentages of residents identifying as white – No relationship found in basic EDA and regression analysis
2. Permits issued will be negatively correlated with higher median age – Reject hypothesis; Permitting is positively correlated with higher median age
3. Permits issued will be negatively correlated with median home price – Reject hypothesis; Permitting is positively correlated with higher median home price
4. Global autocorrelation statistics will identify clustering across the state and local autocorrelation statistics will show high clusters of permitting in high age low clusters in low-age areas – Reject hypothesis; Although global autocorrelation statistics did show clustering across the state, local multivariate autocorrelations showed that high age and high permitting often cluster together.

One major takeaway is that when selecting variables, there are many choices you can make in constructing spatially intensive variables. My independent variable has the number of permits in the numerator and the population in the denominator. My reasoning was that areas with large populations need more housing to fulfill their demand than small areas. On the other hand, the weakness of this metric is that since the population is in the denominator, cities with large populations perform poorly on the measure even though they produce relatively large absolute quantities of housing. As a result, some of the cities one might guess would guess produce “a lot of housing” were not necessarily high-valued on the permits per thousand people metric.

This was not the only variable construction I could have used. Another potential setup might have measured the extent to which permits fulfilled the demand for housing in the community by deriving a demand metric from housing price changes (permitting in the numerator and the demand metric in the denominator). Another approach might have been to consider whether the housing supply is increasing fast enough to absorb population growth (permitting in the numerator and population growth in the denominator). I think there are reasonable arguments for those alternatives, and they might have produced a very different analysis.

In short, it is not clear a priori which setup is best. It’s a question of analytical technique, but it’s also a question of political philosophy: how should policymakers think about which communities are permitted their “fair share” of housing units? Who defines the quantity that we consider a fair share, and what goes into that calculation? Who decides?

Furthermore, once the analysis is complete, how do you evaluate its validity? My findings are counterintuitive: I would not normally think of the Cape and Islands as satisfying the state’s housing needs. But on the other hand, the purpose of data analytics is to generate findings that go beyond what the analyst can perceive through anecdote, and it is true that the Cape and Islands are developing new amenities to host year-round populations, which is a relatively recent phenomenon. It is also true that the state’s housing supply crisis is based around the Boston Metro area, and that many of the suburbs I identified have extremely restrictive zoning ordinances that prevent supply expansion. So although it’s possible to quibble with the parameters of my analysis, it has also picked up on some real phenomena that policymakers ought to incorporate into their approach to the state’s housing affordability crisis.

---