You mention "farm." Do you mean he actually farms, as in grows crops?
Don't know VA laws, but in WV we can get "farm permits", basically open season on deer on the property, if they have been destroying crops.
Still, I would never leave game to rot. Most hunters are ethical and would agree.
Everyone who said "game warden", whatever you call them in VA, is correct. But, I agree, this could lead to a very nasty dispute. I know some backwards unstable SOBs that would not hesitate to physically harm you over something like this.
Be cautious.