How to obtain truth in forecasting ...

| 2 Comments
In the comments, Yonah Freemark says

"But I don't think the evidence is there right now to portray CAHSR's estimates as "unreasonable.""

Now I don't know what truly constitutes unreasonable, it depends on how you reason. By my reasoning, the forecasts are high compared to reference-class lines. Of course we do not know for certain what the actual ridership will be in 30 years, just as we don't really know anything 'for certain' about the future. I would however lay money they miss their claimed forecast. This is something that is missing in travel demand forecasting, accountability.

To that end, I will bet $1000 that in 2030 CA HSR doesn't meet current ridership projections as posted currently on their website: "88 - 117 million passengers annually by 2030 for the entire 800-mile high-speed train network". Any takers? Maybe on http://www.longbets.org/ ...
Contingent only on their completion of the project.

(I suppose we could have another bet on whether the line actually gets completed by 2030. I would bet no, but am less certain about this folly being avoided.)

My friend PB asked "can you think of some way to bet [as in to make money, not claim "I told you so"] the CA High Speed Rail (if built) will be a fiasco? Fiasco in the sense of waste of money/albatross on the state due to operational costs, cost overruns, level of subsidy per usage unit etc ... so again not fiasco as in safety, operational downtime etc."

Aside from one-on-one bets, about which PB notes " I dont like making multi-year bets because of the mental overhead and some other weird asymmetries", one would have to set up a betting pool on various operational characteristics, e.g. Policy Analysis Markets.

I have never done that, but am seriously considering setting something up for travel demand forecasts.

Unfortunately there is no obvious company to short (except maybe CAHSR bonds when they come out depending on whether the state is guarantor) (I assume you can short bonds like stocks, but I don't know if the market exists for 20 years out). The consultants and contractors will all make out like bandits. If someone is fool enough to build it privately, short them.

My sense is forecasters should be required to post bonds about the accuracy of their forecast. Either that, or they should have to be paid in stock for the facility they are forecasting. This would greatly reward honesty and punish optimism bias and strategic misrepresentation.

2 Comments

This is an interesting idea, and seems like it would likely have the effect of discouraging overly enthusiastic ridership forecasts. It seems like it also might have the unintended consequence of stymieing all transit projects, as few consultants would be willing to take risks when the future is so uncertain.

I assume that in fairness you'd also like to set up a futures market for road projects (perhaps on a regional scale) to see whether expansions really reduce time spent in peak congestion and actual commute times over the long term. If not, traffic engineers and forecasters don't get paid.

In response to Brendon's comment: This should apply to flows or travel times, depending on what the critical variable is. It is of course a multi-modal idea.

Induced demand literature suggests elasticities below 1, so expansions should on average reduce travel time, though perhaps not by very much. Generally though they do serve more traffic, so from the point of view of maximizing travel demand (which a normal business would do), highways have success. If the aim were to reduce costs for users, then the success is less.

Toll roads generally also have an optimism bias problem,un-tolled roads less so (see our paper
http://nexus.umn.edu/Papers/ForecastAccuracy.pdf "Post-construction evaluation of traffic forecast accuracy" for an examination of how good/bad forecasts have been in the Twin Cities region. Overall freeway traffic was underestimated, while lower class road traffic was over-estimated.)

David Levinson

Network Reliability in Practice

Evolving Transportation Networks

Place and Plexus

The Transportation Experience

Access to Destinations

Assessing the Benefits and Costs of Intelligent Transportation Systems

Financing Transportation Networks

View David Levinson's profile on LinkedIn

Subscribe to RSS headline updates from:

About this Entry

This page contains a single entry by David Levinson published on October 5, 2010 4:28 PM.

A Tale of Two Ridership Forecasts was the previous entry in this blog.

Should All Public Transit Be Free? is the next entry in this blog.

Find recent content on the main index or look in the archives to find all content.

Categories

Monthly Archives

Pages

Powered by Movable Type 4.31-en