While I carry two sets and always use the one with only the remote fob (to protect the ignition lock), I also do not believe it is reasonable to expect omniscience.
I do not think someone taking your key, making a copy, and stalking you is a reasonably foreseeable outcome of handing a valet your regular key ring.
