The U.S. Secret Service is set to announce agent resignations following a prostitution scandal in Colombia, according to a source with knowledge of the investigation.

The resignations could occur as early as Wednesday evening, the source said, adding that not all 11 agents involved in the scandal are expected to resign.