A lower Vienna court had ruled in May that Aslan Gagiev, arrested in the Austrian capital in January, could be extradited to Russia where he is accused of personally carrying out at least six murders.
The gang's alleged victims include North Ossetia's deputy vice-president, the deputy public prosecutor, a mayor and the chief of police, Austrian federal police said at the time of his arrest.
But the appeals court said that it could not countenance his extradition because of "repeated violations of articles of the European convention on human rights... in custody" in Russia.
This came after the three judges examined reports from the Austrian embassy in Moscow and rights group Amnesty International and a 2012 ruling by the European Court of Human Rights.
The judges called on the lower court to seek assurances from Russia that Gagiev, who is of Georgian origin, can turn to the Austrian embassy at any time and for embassy staff to be able to visit him in jail.