Groups of teenagers as young as 10 are loitering on bridges over the River Cam and stealing the poles used to steer the boats as they pass by.
They then demand a ransom to return the sticks, and sometimes even capsize the vessels by diving into the water beside them.
Paul Holland, 36, said he saw more than a dozen teenagers “running riot” by the side of the river.
“They were divebombing people, splashing them, and nicking their poles and holding them to ransom,” he said.
“Something must be done to stop it.”
One victim, Lara Mackenzie, 25, landlady of the Green Man pub in Granchester, a village just outside Cambridge, went out on a punt on Monday with her husband Barry and six-month-old baby Sadie.
She said: “There was a really nasty-looking group of yobs on the bank who were shouting and swearing at us.
“They looked like they were about to attack and were talking about hitting and stabbing people as we moved past them.
“I actually hid my six-month-old baby because I felt at any moment something was going to go wrong.
“From the way they were behaving and glowering at us I thought we were going to get stoned.
“These people are not innocent children having fun. They are a gang of youths ready to wreak chaos.”
Husband Barry, 36, added: “There was a real element of intimidation. These idiots are up and down the river everywhere. It’s such a shame because it’s spoiling such a beautiful place.”
Rod Ingersent, the manager of punting firm Scudamores, said: “Many customers were coming back quite shaken up, and some of them cut their trip short because they said it had been spoilt. I had to deal with a number of complaints.”
Police said they had received complaints and were increasing riverside patrols.