Try replacing the cap anyway. If there are no obvious drips anywhere, it could be that the cap isn't holding pressure. Since raising the pressure raises the boiling point, a faulty cap could also cause boilovers. Also, check your oil for coolant contamination (it'll be milky). If it is, then your solution will likely be more expensive.
