A magnificent free-kick from Ryan Williams sealed the win for the visitors but it was Aldershot's opening strike, cancelled out by Michael Kightly, that sparked Gray's anger.
David Winfield's stab towards goal looked to have been smothered on the line by Blues stopper Ashley Bayes but, despite referee Steve Cook looking non-committal, his linesman signalled for a goal.
Despite the huge question mark hanging over the first goal, Grays did not really deserve a share of the spoils and Gray will be just as concerned his side never really got going until they fell a goal behind.
Granted they started like the proverbial steam train with Kightly and Glenn Poole combining to set up Lee Boylan for a header that drifted over and Dennis Oli firing just over from 25 yards after dumping Ryan Scott on the seat of his pants in the middle of the park.
But the Shots recovered to dominate most of the first half with nippy striker Joel Grant only denied a five-minute hat-trick by a fantastic double save from Bayes and a lick of paint from the bar.
Blues threatened sporadically on the counter-attack with their best chance of the half falling to Kightly who ghosted in unmarked at the back post to meet Poole's centre only to be denied by Nikki Bull.
The second half began in much the same fashion with the two Grants up front for Aldershot, Joel and John, causing the Blues back line more problems than they had experienced all season.
The pair combined five minutes after the interval when Joel found room on the left and crossed for his namesake who would have given his side the lead had it not been for his own team-mate Ryan Scott who blocked his header on the line.
The controversial breakthrough came just past the hour mark when Williams' over-hit corner was eventually turned back into the box by Louis Soares for Winfield who, amid a melee of players, prodded the ball goalwards before Bayes smothered the ball. Referee Steve Cook was unsure but his linesman was anything but and the goal was given.
Inspired by Kightly, Blues stormed back into the game after the goal. Boylan went close with a couple of half-chances before the ex-Southend man grabbed the leveller.
Minutes earlier he had gone close with a cross-shot after going on the outside of his full-back but this time Kightly cut inside and, albeit with the aid of a deflection, struck a wonderful effort into the far corner.
Kightly went close with another weaving run shortly after and Bull could only watch as a magnificent half-volley from Boylan flew wide but it was Aldershot who came back to nick the winner.
John Grant was felled by Djouman Sangare on the edge of the box and Williams, who had earlier gone agonisingly close with a set-piece from a similar distance, curled a stunning free-kick over the wall, beyond Bayes and into the top corner.
There was still time, however, for Aaron McLean to have a header headed off the line by Dean Smith after a pin-point Poole corner and then Sangare hit the post with a towering header at the back post.
But it wasn't to be for Grays who will be seething at the linesman's decision but can have no complaints about the result.
Barring a short spell at the beginning of the game and a similarly brief period after Shots' first goal, Blues were second best and Terry Brown's men fully deserved their win.