The engine does not need to be running, in fact the common common compression test with a pressure gauge is done with the engine off. You do, however, need to be able to crank the engine in the normal manner with the starter. I would think this would usually be very difficult if the engine is out of the car.
