The visitors struck early on a Kyle Woolven slid home a counter attack at the far post to give Newhaven the advantage, but United would fight back and drew level in the 37th minute as Lucas Mann stooped low to head home a Kane Penn corner.
There was little to separate the two sides in the second half as the game went into extra time. United's hope of progress to the Second Qualifying Round suffered a blow in the first period of extra time as Ryan Paul was sent off for a second bookable offence.
Updated 20:30 - 12 Sep 2017 by Dean Allchin
Worse was to follow for United, as one time United striker Ryan Walton saw a deflected header squirm past Phil Hawkins to give Newhaven the lead for the second time in the contest as the heavens opened.
United were handed a late lifeline thought as Mann was bundled over inside the area with the final whistle fast approaching and United were awarded a penalty.
Up stepped Dean Stevens, but his penalty went high of the Newhaven goal and United's adventure in the FA Vase was brought to an abrupt stop at the first hurdle.
To compound the misery, Peter Featherstone was also dismissed for a second bookable offence on the final whistle