The lyrics in the book produced in 1997 by the Germania zu Wiener Neustadt organisation included "Step on the gas, old Germanics, we can make it to seven million", according to media reports.
Six million Jews were killed in the Holocaust during World War II, many of them in gas chambers. Other songs in the book praised the Waffen SS and Nazi paratroopers behind war crimes committed in Greece.
The scandal took on a political dimension because until recently the vice-chairman of the fraternity was Udo Landbauer, a candidate for the far-right Freedom Party (FPÖ) in local elections in Lower Austria state last Sunday.
It also put Chancellor Sebastian Kurz under pressure since he had formed a coalition at the federal level in December with the anti-immigration, Islamophobic FPÖ, giving the party the interior, foreign and defence ministries.
Many members of the FPÖ, a party created by former Nazis in the 1950s, are members of student fraternities, some of which are pan-German and engage in duels with swords and torch-lit processions.
The FPÖ says the fraternities as harmless, with its leader Heinz-Christian Strache saying on Friday that "anti-Semitism, totalitarianism (and) racism are the opposite of fraternity thinking".
The affair has also caused embarrassment for the centre-left Social Democrats (SPÖ) after it emerged that a member of the party - one of four people under investigation by prosecutors - had illustrated the song book.
The party expelled him on Tuesday. The FPÖ has not ejected Landbauer, however. The 31-year-old has said he was 11 years old when the book was printed and only found out about it last week.