Some 17,000 people took to the streets of the capital despite the bad weather, according to police estimates, while organisers put turnout at 50,000.
Many were waving banners reading "Fascists out of the government", "Racism gets on my nerves", and "More love, courage, togetherness", as they made their way through Vienna before gathering outside the Hofburg imperial palace.
The rally was held on the eve of the first anniversary since Chancellor Sebastian Kurz's People's Party (OeVP) and its far-right Freedom Party (FPOe) partner took office on December 18, 2017.
Under their leadership, Austria -- which also holds the EU presidency until the end of December -- has introduced a series of tough anti-immigration measures over the past year.
In the latest move, the government last month finalised the details of welfare cuts aimed at immigrants, in a move criticised by anti-poverty campaigners and church groups.
"We want a different Austria," said MP Andreas Schieder of the opposition Social Democrats (SPOe) told AFP.
Schieder is the SPOe's lead candidate in the European Parliament election in May 2019, when traditional parties face major challenges from far-right and eurosceptic populists.
With growth of three percent expected for this year and one of the eurozone's lowest unemployment rates at 5.6 percent, Austria's coalition is still riding high in opinion polls.
Kurz's People's Party won 31.5 percent of the vote in last year's elections, but now scores around 35 percent in the opinion polls.
The Freedom Party, led by Vice-Chancellor Heinz-Christian Strache, has slipped only slightly from the 26 percent of the vote it won last year to around 24 percent.