The Essex side, encouraged perhaps by the heroics of Barrow against Brentford the previous evening, warmed to the task of frustrating the League One promotion hopefuls.
Goalmouth action was at a premium, however, with the verbal sparring between two lively sets of supporters providing an entertaining sideshow.
Waddock's troops broke sporadically but never threatened to penetrate a Millwall defence which had conceded just once in its previous six games in all competitions.
Tame headers from Marc Laird and Alexander were the best the home side could muster early on before the constant ammunition from the flanks finally reaped dividends.
David Martin was allowed too much space to fashion a cross from the left and Alexander rose unmarked to plant a header firmly beyond Nikki Bull.
Martin came close to doubling the lead three minutes before the break with a fierce shot which would have found the top corner but for a slight deflection off the boot laces of Anthony Charles.
Alexander came within inches of a second goal just a minute into the second half when he got on the end of Lewis Grabban's cross, but he could only divert the ball wide of the far post when it looked simpler to score.
It was far from one-way traffic, however, and Aldershot would have equalised had Chris Hackett not taken the sting out of Danny Hylton's shot after a kind ricochet looked to have presented the midfielder with a simple tap in.
And the League Two side came even closer just shy of the hour-mark when Tony Craig performed a miraculous clearance from underneath his own crossbar following David Forde's failure to deal with a corner.
Just as the underdogs looked to be on the verge of forcing a replay, Millwall broke away and ended the contest with 13 minutes remaining.
Hackett's deep cross from the right located Alexander at the far post and he made no mistake with a thumping header which left Bull with no chance.
And substitute Grimes rounded off an ultimately convincing with an identical goal nodding home Hackett's cross with two minutes left.