weMove Massachusetts: A disappointing, flawed plan for performance metrics that will repeat the mistakes of the past

This month, MassDOT announced a new plan called "weMove Massachusetts" which introduces the idea of using performance metrics to help plan investment in transportation. Sounds good, right? I was looking forward to reading about a scientific approach towards selecting projects that help people access the opportunities in their lives, while steering towards overall goals like "GreenDOT." But instead of that, when I read the report, I found the same bad old attitude that you also find at the Texas Transportation Institute. The main automobile measure is regressive. The sole metric for bicycle usage is almost completely useless. The transit metrics range from irrelevant, to poorly designed, down to outright worthless. There are no walking metrics in the report. Many of the metrics don't even involve actually measuring anything in the real world!

Metrics have consequences

Before I go into the specifics, let's take a quick step back and think about what we actually want out of our transportation metrics. There are two main ways of looking at transportation broadly, and both have their uses, depending on context: mobility vs accessibility.

Mobility measures the amount of traveling you are able to do: e.g.

• How fast can you go?
• How far can you go?
• How easy is it to move around?

Accessibility measures the amount of opportunity that is open to you because of transportation: e.g.

• How many jobs/needs/activities can you reach in a reasonable amount of time?
• How many choices do you have about getting to your destinations?
• How is your quality of life improved by transportation?

These are not completely distinct viewpoints. For example, you could increase the number of jobs available to you by improving the speed of transportation. On the other hand, you could also achieve the same by having the economy add more jobs within your existing range, or by moving to a region with more job density. The latter is, broadly and superficially, the way cities form.

However, choice of viewpoints does matter when trying to put together cost/benefit analysis for new transportation projects, and it is very important to choose carefully. A metric is a convenient number which tells you something useful, but it is also an intentional oversimplification which can lead you astray if not used judiciously. Unfortunately, our MPO has a history of abusing metrics, and traffic engineers are infamous for applying the wrong tools to our cities, causing terrible destruction in the process.

Is there an ideal general metric at all? I don't think so, but the closest one I believe is the last example I cited above: "How is your quality of life improved by transportation?" Ultimately, transportation is about giving you the ability to live your life as you see fit, to take advantage of the opportunities open to you, and to keep you safe while doing it. The problem with this formulation is that it is very vague and broad, making it difficult to quantify. That's intentional: "quality of life" means as many different things as there are different people, so there really is no one way to interpret this. Instead, when designing performance metrics that are more easily quantifiable, this principle is what you should be striving to justify your decisions by: how does your work help people live the life that they want to live?

Without further ado, I will jump into the "weMove Massachusetts" report and its many flaws:

Pavement, Bridges

I've grouped these two together because they have a similarity: obviously you don't want roads to crumble or bridges to collapse. But, the metrics assume that the status quo is sustainable and desirable, and all that is needed is an influx of money to repair what we have. A very, very large influx of money: $544 million a year on Pavement and $447 million a year on Bridges. That's a billion dollars a year for what will basically only benefit motor vehicle modes. This is not to deny the need for such repairs or such infrastructure. We need some of it. But do we need ALL of it? There should be some cost/benefit analysis applied to the two largest line items in the transportation budget. These metrics do not enable that. There is no consideration given to whether a lane diet might be a better way to go, for safety and money-saving purposes. And there is no consideration given to whether an overpass should be removed instead of rebuilt, even though 1950s traffic levels on it long ago disappeared. Is every square inch of asphalt sacred? Must we continue to perpetuate the mistakes of our forebears, at any cost, without thought?

Mobility

This metric is one of the worst in the document. It's basically lifted straight from the highway-crazed Texas Transportation Institute. Sure, this metric is fairly easy to quantify, but it reduces the whole notion of "mobility" down to one, highly confounded number: hours of delay per 1,000 miles traveled in a private vehicle. To understand why this is a bad metric, even from a driver's standpoint, consider the following tale of two commuters, Alice and Bob:

Alice lives in Waltham and works at an office near Rt 128 in Waltham, about 2 miles away. Under ideal conditions, her commute travel time is 5 minutes by car. However, under typically congested circumstances, it usually takes her about 10 minutes by car.

Bob lives in West Stockbridge and works at a shop in Westfield, about 40 miles away. Thanks to the sparse population, Bob can almost always drive 60 mph from start to finish, for a commute travel time of 40 minutes each way.

I cannot speak for the relative happiness of Alice and Bob, since it would depend on their (fictional) personalities. However, I can point out that Alice only spends 20 minutes a day commuting, while Bob spends 80 minutes a day just getting between home and work. Yet, according to this "mobility" metric of MassDOT, Alice is suffering terribly from delays, while Bob is doing great. Assuming 250 working days per year, Alice puts in 1,000 VMT/year for her commute, and experiences "delay" of 41.6 hours/year.

Viewed through the prism of this mobility metric, the DOT would rate Alice's commuting experience as being miserable, slow, and hellish. But that doesn't make sense from a common sense perspective. Alice only has a 10 minute commute! I bet most people would love to be in her situation, rather than being stuck in a car slogging it out 40 minutes each way. The DOT's metric is completely at odds with the principle of "improving quality of life." Not only that, the metric completely ignores all other modes of transportation. Two miles is not that far. That could even be walkable on a daily basis. It's easily bikeable within 15 minutes. If Alice is the sort of person who values getting exercise to and from work, it could easily be made possible. Or perhaps she would rather take a bus, the 70A, and do some reading on the 15 minute trip. Bob has none of those options: he is a captive car commuter. The only option he has is to car pool with someone else. But the metric doesn't even care about that: since it measures vehicle-delay rather than person-delay, your passengers may as well not exist.

The story of Alice and Bob is a bit too neat, obviously: in the real world, there are many other factors. But it's meant to illustrate a point: the mobility metric chosen by MassDOT is aimed in the wrong direction. The metric of "delay per vehicle-mile" is not designed to produce good transportation outcomes for people, but rather, to produce more funding for expensive road-widening projects which line the pockets of connected contractors and please the egos of highway engineers. The footnote to this metric tells it all:

Funding for mobility projects includes any projects that will increase roadway system capacity such as additional lanes, bottleneck removal, intersection improvements, etc.

MassDOT will claim that Alice is "suffering" from tremendous delay as justification for widening the streets in her neighborhood, installation of fancy traffic engineering techniques such as grade separation and complicated traffic signal phases. All of which will degrade the quality of life in her neighborhood. To be fair, in this day and age most of that will be shouted down by commuter opposition, but this is the same kind of metric that was used to shove these kinds of "improvements" down the throats of our communities in the past. More insidiously, planners will use this delay metric as justification for furthering suburban sprawl, instead of creating compact, walkable neighborhoods. The "delay per vehicle-mile" metric is complete garbage and thoroughly inappropriate for an agency which claims to be about promoting "smart growth" and "green travel" mode shift.

Safety

Safety is important. Everyone knows that. But, this particular way of doing it is an example of a metric which sounds good at first, but is actually quite mediocre, or downright dangerous, when you look at the details of how it is actually implemented. Why is there a focus only on intersections and interchanges. That seems strange. Although those are certainly higher risk locations, crashes can happen at any place. Why the restriction? The answer is found in another section which fills in some of the details:

Roadway safety is typically measured by the number of crashes. Crash rates cannot be easily projected into the future using the types of performance curves applied to other highway metrics. Therefore, safety performance was measured by the number of high-crash locations that can be improved over time for an average cost of $500,000/intersection.

So basically, they claim that they cannot get a good metric for "crashes prevented per dollar" and therefore, as a proxy, they will just count the number of intersection projects that can be funded, assuming half-million dollars each. Here's a litany of problems with this approach:

1. There's nothing about "intersection improvements" which necessarily implies better safety.
2. DOTs have a history of making "improvements" that make streets more dangerous for walking and biking. For example, street widening is sometimes considered a "safety improvement" even though it will end up producing more injuries, particularly to vulnerable road users.
3. The real danger to safety comes from speeding vehicles. Again, "intersection improvements" says nothing about this, and the metric is completely mum about speeding elsewhere on the street.
4. The metric encourages spending money on intersections and interchanges, but it does not actually measure whether that money was spent in a worthwhile way.
5. It assumes that spending a half-million dollars is all that you need to do to "fix" an intersection.
6. Severity of crashes is not considered at all. If you had to pick a design based on safety consequences, which would you select? (a) One person will die each year in a severe crash, or (b) ten people will be slightly injured a year in minor crashes. Obviously, zero injuries or deaths is best, but if you had to choose, then I think that (b) is a much better outcome. The metric considers none of that.

If you are going to adopt performance metrics, then you should actually go out and measure something in the real world. This so-called "safety metric" does not do that: instead it simply divides the amount of money available by the number of intersections, and calls it a day.

Bicycling

Behold, the only bicycle-focused metric in the entire report. The main mobility metric only focuses on cars, so this one should partially make up for that, right? Right? Yeah right. This metric is the first one in the report where you start to think that this whole document is really just a long, boring joke. If you try to consider it, there are a variety of ways you might go about quantifying bicycle performance (not endorsing any particular one): bicycle-miles traveled, % mode share bicycling, % population that owns a bicycle, % population that considers bicycling, number of comments on the Internet complaining about bicyclists, etc. What all these metrics have in common is that they involve making measurements in the real world: something must actually be counted, a number that cannot simply be conjured out of thin air.

This is in stark contrast to the proposed MassDOT metric: completion % of the Bay State Greenway, a network of off-street cycle paths. The DOT's metric is computed by looking at the budget documents and comparing whether or not the Bay State Greenway is funded under both budget scenarios. Since it is, the metric is set at 0% from now until the end of time. The genius who put this one together probably thought he or she was pretty clever to insert a "performance metric" which required a grand total of 30 seconds of work behind a desk, and will never need to be revisited again.

Complete and utter failure. Secretary Davey, this is a disgrace.

Walking

Just kidding. There are no walking metrics in the report. Well, I suppose you could pray that the "Safety" metric will bring "intersection improvements" that actually benefit pedestrians, and that the "Mobility" metric does not end up destroying the walkability of streets which used to be safe before the traffic engineers got their hands on them.

Transit

The tone of the report changes once it reaches the transit metrics. Prior to this, the metrics were somewhat broad or simplistic, but the transit metrics are very precisely chosen. Unfortunately, they are chosen rather poorly for planning purposes. I was told by some MPO staff that the MBTA was responsible for their metrics, and that does seem to be the case because these metrics are largely only of interest to someone working in the T's maintenance department. Since there are so many, I will group a bunch together and address them at once:

• MBTA Bridge condition: the % of bridges that are in state of good repair.
• MBTA Elevator/Escalator condition: the % of elevators/escalators that are in state of good repair.
• MBTA Station Accessibility: the % of stations that comply with the Americans with Disabilities Act and the MA Architectural Access Board standards.
• MBTA/RTA Bus and Train performance: the % of vehicles in state of good repair.
• MBTA Signal performance: the number of annual signal failures.

All of these metrics are important -- to the people in charge of maintaining the system. However, these metrics are fairly irrelevant to the average customer. Reliability is important to the customer, certainly, but we don't go through the trouble of providing public transportation because we really like to do bridge or signal maintenance: the purpose, the reason for existence of the system, is making transportation available to people. People want to know that when they show up at a train or bus station, a vehicle will arrive in a reasonable amount of time. They want a system which gets them where they need to go, which gives them access to the city region and all of the opportunities contained within. Specific "state of good repair" statistics are part of the internal implementation of reliability. I suppose it's fair enough that they are being open about it and including it in the report. Fine. But then, where are the metrics which actually measure what people want out of a public transportation system? Not present in this report. Maybe the closest is another maintenance statistic:

Again, we have a performance metric based on delay, like what cars get. Good? Not quite. One improvement is, at least in this case, positive values don't imply a need for highly destructive road widening, since that is not on the table. So, improvements to delay are probably all upside, no downside other than cost. What else could be wrong? Well, it's another case of the same disease which seems to periodically infect planning at the Boston MPO and the DOT: vehicle-itis, the curious syndrome in which transportation agencies forget that their job is to help human beings, and instead focus entirely on pieces of steel and fiberglass.

This metric, like the mobility metric before, only focuses on the delay to vehicles, rather than people! In the other case, you might shrug it off because the average car only carries a few people at a time (although that attitude still screws over carpools). But it is huge loss when you neglect to account for the fact that a train could easily be carrying over five hundred people who are all delayed when the vehicle is delayed. The next time the Red Line breaks down, think about this: the MBTA, MassDOT, and the Boston MPO have all committed to the belief that your time, as a transit rider, is only worth 1/500th the value of anyone who happens to drive alone in their car.

A lot of perversity about our transportation system can likely be explained by this attitude:

• The ever-increasing insertion of delays into the routine of T travel. From completely pointless, customer-loathing tactics like "Front Door Only alighting," to random "safety stops in the middle of tunnels" that are never fixed, to schedule management that hasn't changed since the early 20th century, to deferred maintenance up the wazoo, it's all part of a culture of not caring about the riders on-board the vehicles.
• The continuing refusal to use signal priority for transit vehicles on the surface. Both Beacon St and Huntington Ave have the technology installed to do it. But the MBTA can't be bothered to implement this modern technique which is used all over the country and the world. In their eyes, a train with 300 people on board is not worthy of causing ten seconds of delay to a precious private automobile.
• An almost complete lack of bus lanes outside of the gimmick lanes on Washington St -- which don't extend into the part of the city where bus lanes are desperately needed. This one's on BTD as well, but the MBTA should be pushing hard to improve the ridiculously slow speed of its buses.
• The achingly slow turnaround time on the barest of improvements. The key bus route improvement project took about three years. Not because it was hard (it's not). Not because it was expensive (it wasn't). But because nothing happened for two years and then finally bus stops started getting moved, and painted in the street, within a few months at the end of last year.
• The truly pathetic state of some of the stations, particularly on the surface, where there's nothing more than a few feet of asphalt with cars zooming by, inches from your body. And that's on a section of roadway that was rebuilt relatively recently!
• The fingernail-pulling it takes to get sidewalks, crosswalks and bus stops shoveled clear of snow around here.

Choosing good metrics for transit is not easy, as it depends on the context and what you are trying to accomplish. But you cannot reasonably plan a transit system without using metrics which count people. Some examples of useful mobility-style metrics might include, but are not limited to: boardings per mile, net cost per passenger, average passenger load, or number of people unable to board vehicle due to crowding. Some examples of useful accessibility-style metrics are: number of jobs available within a 30 minute transit trip, number of people within a 5 minute walk of a frequent transit stop, number of people with disabilities who successfully navigate your fixed-route system, or percentage of riders who claim [dis]satisfaction with the transit system.

Conclusion

I know that I have been a bit crass about the problem here, but it is only because a strong tone is necessary. The "weMove Massachusetts" plan is terrifyingly terrible. It represents a large step backwards, encoding 1950s-style thinking about highways into a document which pays lip service to "smart growth" and "green travel" but does nothing to promote either. It is so bad, that I can only recommend one course of action: rip up the "weMove Massachusetts" document and start over from scratch. This time, choose empirical metrics which require real-world measurements, categorize them under either "mobility" or "accessibility," and justify why the people of Massachusetts should care about the metrics. I propose that the guiding principle be: "transportation investments that improve the quality of life of people in the Commonwealth" but I am open to suggestions.

I also want to add a note about GreenDOT: supposedly, it is a goal of MassDOT to triple the mode share of walking, bicycling, and taking public transportation. This report talked about that goal but, like with many other things, did not really go into any significant detail about it. However, I did hear from MPO staffers that the metric used will be "person-miles" for each of the "green modes." In other words, the Commonwealth of Massachusetts will either seek to encourage three times as many people to walk, bike, and ride transit; or, the Commonwealth will seek to force all current walkers, bikers and transit-riders to travel three times as far as they do today. According to the metric, either one works. Okay, that sounds a bit ridiculous, but that's because "person-miles" is not a good metric for walking, biking or public transportation in cities. If you want more details, read those links, but suffice to say: there's no public good served by forcing people to walk, bike or ride buses further and further away. Distance is not a virtue in itself.

I just wish that the people who put together metrics at our departments of transportation, and our organizations of planning, would put in even an ounce of thought into their work.

For more reading on metrics, and mobility vs accessibility, I recommend this paper by Todd Litman.