Hannes Führinger from Burgenland was flown to Vienna airport on Tuesday morning.
He had served only two thirds of his sentence but was unexpectedly freed by Egyptian authorities, according to Foreign Ministry spokesman Thomas Schnöll.
His lawyers and his wife had made frequent attempts to secure his release and get him extradited to Austria. Schnöll said that diplomatic negotiations had been tough but had finally achieved the desired result.
He said it was one of the most difficult cases the Foreign Ministry has ever had to handle, with more than 50 prison visits, a letter from Foreign Minister Sebastian Kurz (ÖVP) to his Egyptian counterpart, and a determined effort from the embassy.
During his time in prison Führinger became very ill, but Schnöll said that his health is currently stable although his time in an Egyptian prison had had an enormous psychological impact.
His mother told Austrian media that he was beaten in prison and went on hunger strike, and that conditions in the prison were unbearable.
Führinger worked for a private security firm, and was detained in November 2011 after he travelled to Egypt with four guns and 200 rounds of ammunition - which he declared on his departure, saying he needed them for a job guarding an Italian cargo ship.
He was charged with "illegally importing unlicensed weapons, ammunition and telescopes into the country and hiding the goods in order to avoid customs duties" and sentenced to seven years in jail.