Make sure the Ebrake is released and that the adjuster is released via the hole in the back plate. Squirting some WD40 around where the hub & studs poke through drum and then tapping with a hammer all around like ExDelayed mentioned should help loosen up the rust.
