United were closing in on the summit of the Blue Square Premier, courtesy of a Scott Rendell goal, when David Winfield levelled two minutes from time.
Then Quinn saw Marvin Robinson, the man he brought in as striking cover two days ago, come on as substitute and get sent off - all within 11 minutes. He now faces a one-match ban.
The result maintained United's record-breaking 12-game undefeated start to the season and extended their unbeaten streak to 14 games going back to the end of last term.
Quinn said: "We hung on in there but with Stevenage losing, it's not a bad result.
"The lads are all disappointed in there, though, which shows you far we've come.
"We put a lot of energy into the game, but we didn't kill them off. We had three opportunities in the game after we scored the goal.
"Aldershot had won all five of their games away on the trot and they've got a lot of pace in their team and a lot of young lads who are always playing on the front foot.
"When you look at it, they're top of the table and they've come here and done a job on us, but we're still unbeaten."
New loan striker Robinson picked up two yellow cards, both for fouls, and was sent off in stoppage time.
Quinn was disappointed Robinson, who only arrived from Oxford on a three-month temporary deal on Monday, would be suspended for Saturday's difficult trip to Forest Green.
He said: "I thought Marvin did really well because he has got that extra bit of pace.I think he was just a bit over-enthusiastic in trying to impress me and it's the first sending-off he has had."
Source: http://www.cambridge-news.co.uk/