There is a difference between "HAVING" the right of way and "YIELDING" the right of way.
Pedestrians in a crosswalk with a "do not walk" sign do not HAVE the right of way.
However, the right of way must be yielded to them. Obviously, you can't run them down, but likewise, a pedestrian can't just amble across a busy street whenever he wants....