If your wife isn't sending clear signals that she's taken, then you need to address it with her. Comments are one thing, gifts are another. Also, how long did it take you to find out and who told you. If it was by the end of the day and your wife, I think your fine.
-Andy