The street in question is a 4 lane shoulder-less road with an occasional extra parking lane. One side abuts a busy commuter railroad right-of-way, fenced off. The other side has a skinny sidewalk strip and typical suburban households. The road is not technically a highway, but it feeds one, and is treated like one by drivers.
There is no question that this is an unpleasant place to walk, day or night. I personally would go out of my way to avoid it, especially since the sidewalk that does exist is overgrown. Walking it at night set off many of my internal alarms about dangerous areas. It is dark, there are lots of gray areas, and nobody is around save for cars speeding by in the night. Addressing the darkness issue is one step, but I am worried this will create a false sense of security. The reason is simple: extra lighting does nothing if nobody else is around to see. The fundamental issue is the lack of pedestrian traffic and the lack of eyes on the street. This is a self reinforcing problem: nobody goes there because nobody goes there!
Admittedly, this is a difficult problem to resolve within the parameters that currently exists. In a better world, the street would be reduced to two lanes of traffic and the sidewalk upgraded. An 8-lane thoroughfare already exists just a couple blocks away as it is, and a highway not too far either. Then there would be space for a shoulder and bike lanes. I would also like to see zoning relaxed and commercial activity permitted on this street. As it is, it is very pedestrian-unfriendly, and anyone wishing to do shopping must currently head north past Middlefield, or go around to cross the tracks at one of the few possible places. The Caltrain/CAHSR grade separation project should offer the opportunity to redo this road, and also to introduce additional crossings for pedestrians and bicycles (and even cars). With a little bit of forward thinking, this street could be transformed from highway-like gray area into a pleasant pedestrian passageway.
It is an opportunity Palo Alto residents should be welcoming. Unfortunately, that will probably not be the case, as resistance to any kind of change -- even good change -- is a neurosis that infects the whole community, it seems.