The assailant “died on the spot” after the soldier opened fire, police spokesman Harald Soeros told AFP, adding that the attack took place just before midnight local time.
The motive for the attack is not yet known, police said.
The soldier, who was standing guard outside the building, suffered a wound to his upper arm in the attack.
“He initially used pepper spray” against the assailant, Soeros said, before opening fire. He was believed to have fired his weapon at least four times.
According to early indications he “acted within the rules”, said a spokesman from the ministry of defence quoted by the APA agency.
The official added that the soldier, who is from the Tyrol region of Austria, has been treated in hospital. Police have ordered reinforcements around all diplomatic missions in the city.
The incident comes just days after two knife attacks in Vienna, including one that left a couple and their 17-year-old daughter with severe injuries. A 23-year-old Afghan national confessed to those assaults but denied any political motive.