The ruling ends the long-running legal saga of Aslan Gagiev, who was immediately taken into custody, police in Vienna said.
The 46-year-old was first detained at a Vienna train station in January 2015 on an international arrest warrant, but was freed on bail in February last year.
Russian authorities accuse Gagiev's gang of at least six murders.
Those killed include the deputy vice-president of the Russian republic of North Ossetia, a deputy public prosecutor, a mayor and a chief of police.
Gagiev denied wrongdoing and said Russia's attempt to have him extradited was politically motivated.
After several different rulings, Austria's highest court put his extradition to Russia on ice in 2015 because of concerns about how he would be treated there.
This was in part because Russia's constitutional court had ruled that Russian laws take precedence over judgements by the European Court of Human Rights.
However, Vienna's high court said Tuesday that Gagiev's extradition was legal after all.
The decision cannot be appealed.