Over 100 soldiers will be stationed at the 24 embassies,the UN, and other international institutions located in the Austrian capital.
The soldiers received ten days training prior to taking up the positions, including how to deal with stressful situations with aggressive people as well as dealing with everyday situations such as administering first aid.
Like the police, the soldiers will be armed with a service pistol and pepper spray but they will not be allowed to make arrests and must call the police in such cases.
“So that they can guard property, they can move people away and they are allowed to use force,” said army spokesman Kurt Wagner.
The extra police freed up by the army will be used to tackle crime prevention, patrol neighbourhoods, and help to tackle the ongoing refugee situation.
"As long as the police request it, well trained soldiers are operation-ready to guard these properties," said Austria's Defence Minister Hans Peter Doskozil.