If the cam will turn freely by hand without finger followers or timing chains installed then you should be good. If there is any binding, then what ever is binding will have little to no oil film while running and will eventually score and seize.
BTW, swapping over cam caps from another head is tricky at best. The cam bore centerline in the head is not necessarily level with the top surface of the head (parting line). This means that the cam cap does not necessarily contain exactly one half of the bore. If it's even off by 0.05mm then will pretty much get binding. You might be able to get away by trying several cam caps and using plastigage to check clearance. It's subjective but it's the best for someone without access to bore gages.
For troubleshooting, you can also try moving cams from int. to exh. or vice-versa temporarily just as a process of elimination.
BTW, swapping over cam caps from another head is tricky at best. The cam bore centerline in the head is not necessarily level with the top surface of the head (parting line). This means that the cam cap does not necessarily contain exactly one half of the bore. If it's even off by 0.05mm then will pretty much get binding. You might be able to get away by trying several cam caps and using plastigage to check clearance. It's subjective but it's the best for someone without access to bore gages.
For troubleshooting, you can also try moving cams from int. to exh. or vice-versa temporarily just as a process of elimination.