Matter of fact that just happened to me about 1 week ago.
Not as drastic as your's, but just as scary.
I parked on a moderate incline and set the parking brake. I went inside and when I came out later and my car had rolled back 1 car length. (I know, I know leave it in gear) My heart about stopped coming out the door and seeing the car had moved! Good thing it was an empty lot behind me.
I chalked it up to not pulling the handle hard enough since it operated just fine afterwards???
I would say misadjusted rear brakes, but they are disc not drum...
Perhaps the cables are stretching/stretched.
I have noticed the brake does not work as well for backwards movement so that may have come into play as well.