S&P Case-Shiller housing index probably overweights foreclosure sales

Thursday, July 2, 2009

The monthly S&P Case-Shiller home price index has been around since the 1980's. And it seems to be the most-followed index of housing values and is said to be the superior measure of home values in current market conditions. The S&P Case Shiller index matches sales (tracks the prices) across all types of mortgages, conforming and non-conforming. The monthly FHFA house price index (formerly OFHEO) tracks just home sales of mortgages that conform to standards (limits) set by Fannie Mae and Freddie Mac.

But it does look like the S&P Case Shiller index is becoming increasingly focused on foreclosures. Despite the fact that the metro weights for the index remain the same, the sale-pair counts are focused in those high-foreclosure areas. Hence, there is a downward bias on the index overall.

The chart above illustrates how the sales pair counts compare to their sample average (y-axis) and the associated loss in home values from the peak through April 2009 (x-axis) across the 20 metropolitan areas that go into the monthly composite-20 S&P Case Shiller home price idex. You can get the sales pairs (the matching home sales) and home value data here and see a short description of the full methodology here. Let me be very clear about the y-axis: it is the average share of sales pair counts for 2009 (January to April) minus the average share of sales pair counts across the entire sample of data for each metropolitan area in the composite 20.

What I notice is that the sales pair counts are becoming increasingly weighted toward the biggest bubble - i.e., foreclosure - metro areas: LA, San Diego, Phoenix, etc. Sales in these areas are really dragging down the overall value of the index. Presumably, the foreclosure sales are weighted less heavily, and the metro area weights are fixed in the overall index. But it is somewhat suspect to me that the share of Phoenix's housing market, for example, has increased its share of pair counts by 8.7% over its sample average, 5.4%. Therefore, there are a lot of foreclosure sales biasing the index downward.

Below is a table that relates the share of each state's stock of housing (out of the total) to the share of pair counts included in the S&P Case Shiller composite 20. For example, California owns 10.4% of the total housing stock, but currently receives a 25% sale-pair weight in the Case Shiller index.

The FHFA index is much broader geographically and incorporates the whole of the census regions (see the release here). In contrast, the monthly S&P Case Shiller index's geographic net is thinner.

The S&P Case Shiller index is most certainly a step up from the median or the mean (reported by the National Association of Realtors). But I do wonder what information it is really giving us. S&P Case Shiller is not the end-all, be-all to home price values, and probably overweights the index on foreclosure sales.

Rebecca Wilder


James July 3, 2009 at 10:56 AM  

Remember that there are multiple Case-Shiller indexes. The 20-city index only measures 20 metro areas. The national index measures two-thirds of U.S. housing.

Also, the table in your post isn't quite right, even for the 20-city index. The 10- and 20-city indexes measure entire metro areas, not just cities. So, for example, the table indicates that the District of Columbia makes up 5.73% of the 20-city index's weight, while Virginia and Maryland make up zero percent. In fact, Virginia and Maryland make up the vast majority of the Washington, DC metro area's population, so the table is incorrect. One third of Virginia's entire population is in the DC metro area, so Virginia is significantly measured by the 20-city index. Also, the vast majority of the foreclosures in the area are in the outer suburbs in Virginia and Maryland. DC itself has few foreclosures.

A similar thing occurs in the NY metro area. The table says New York makes up 8.25% of the index, while New Jersey and Connecticut make up zero percent. But, New Jersey and Connecticut make up significant parts of the NY metro area and are measured by the 20-city index.

Rebecca Wilder July 3, 2009 at 11:17 AM  

Hi James,

I agree, the index is "not quite right" (as you say); I have corrected the chart to better represent the facts (as best I can).

Although the point that I was trying to make still holds Phoenix metro area holds 14.12% of the sale pair counts, while the state Arizona accounts for just 2.1% of the total housing stock.

Thanks, Rebecca

G2 July 5, 2009 at 7:29 PM  

during times of high foreclosures,such as now, doesn't the below methodology affect what actually is being represented ?

"When they can be identified from a deed record2, non-arms-length transactions are excluded from the pairing process. The most
typical types of non-arms-length transactions are property transfers between family
members and repossessions of properties by mortgage lenders at the beginning of
foreclosure proceedings. Subsequent sales by mortgage lenders of foreclosed properties
are included in repeat sale pairs, because they are arms-length transactions."

fair to say that the statistic totally excludes "foreclosures" but are indicative of re-sales (auctions, distressed sales etc) by mortgage lenders ?

Anonymous July 7, 2009 at 9:52 PM  

What is the relationship, if any, between the C-S sales count pairs and pricing structure? There is none.

Do you really know how the indices work with real money? You can put your bucks on UMM or DMM, Rebecca. I'm sure it will be almost as popular as the GLD ETF in 5 years. It is a thin market today but it's the greatest financial innovation for the typical American household since sliced Wonderbread.

I believe that you are confounding data counts (if you didn't know, Phoenix sales, using your example, are on fire this year...well actually not, as paired sales are down since the pre-pricing peak) with valuation weights.

If anything, the higher the sales pairs, the greater the accuracy of pricing. And the C-S index is value weighted. A 10% loss on a $500,000 property is weighted more than a 10% loss on a $250,000 property.

And just to further nitpick, the FED does not use the FHFA data for Flow of Funds. But you could email Casey Mulligan for confirmation on that issue.

Rebecca Wilder July 8, 2009 at 7:52 AM  

Hello Anonymous,

You say: What is the relationship, if any, between the C-S sales count pairs and pricing structure?

The sale pair counts most certainly affect the S&P index; looking at the resale price (sale pairs) is the mechanism through which the index is constructed. Presumably, the foreclosures get a smaller weight in the index (i.e., in Phoenix), but nevertheless, if there are a growing population of sale pair counts focused in key areas, foreclosure prices are likely driving the index.

I am confused how the market’s macro shares DMM has anything to do with the fact that the index may be undervaluing the overall market? Not to mention, I see that the composite 10 will be followed.

Finally, you are correct - the fed no longer uses the FHFA. Late in 2008, it adopted the LoanPerformance HPI. I have corrected this in the text.


John July 9, 2009 at 10:59 AM  


Thank you, thank you!

We have been analyzing the Case Shiller data for years and as you have indicated very concisely, the index does have a potential for bias.

Your direct comment about Phoenix and Arizona hits the mark. Case Shiller refers to the overall sales price and does not consider the fundamentals of the sale price such as square footage, age of home and land area. Nor, in our opinion does it recognize the weight of the general population size to sales activity size.

Our analysis indicates that the price decline, if developed on a sales price per square foot basis, is not as drastic. Furthermore if you establish a weighting influence for the impact of foreclosures, the decline would be even less.

Maricopa County (Phoenix) had a study completed to determine the influence of foreclosures for assessment purposes. This report clearly shows a differential of 10% to 15% (depending on level of foreclosures). This report is based on the sales price per square foot. So if 1.5 million homes in the 5th largest city are valued recognizing these influences, why isn’t Case Shiller recognizing this.

Many will argue that foreclosures are market value and hence are the market. In my opinion this is not true. Yes foreclosures influence the market, but adjustments must be considered for these sales under market conditions and financing in an appraisal.

In addition to your comments on all of the regions, I will clearly state this: More weight should be developed based on the general population of properties, not the sales activity alone.

Kudos’s for pointing this out. Case Shiller represents two-thirds of the housing market? Well what is it leaving out? An index that tracts Metro Market should be based on a consistent parameter (say 60 mile radius) and it should identify the total properties covered, residential properties and annual sales activity. This index should also categorize properties into groups. I find it very difficult to accept that a 1,000 square foot house has the same economic and demographic buyer as a 2,500 square foot house.

Yes, I understand the need for a general overview, but the resources exist for a more expansive index and it should be developed. This is no easy task. Our current tracking covers 45 million properties versus Case Shiller at 70 million. Our analysis clearly indicates that Case Shiller, while being a benchmark is not a true indicator of the overall market.

Based on this past real estate crisis, we simply cannot afford to use indexes that lack total coverage and stratification levels which will enable financial institutions, realtors, builders and the general public the ability to understand a very complex financial market.

John Watch

Blog Archive

Search News N Economics

  © Free Blogger Templates Columnus by 2008

Back to TOP