A number of the Pegida supporters at the Vienna demonstration were skinheads, and some of them performed Nazi salutes, an AFP reporter at the scene said.
Police spokesman Roman Hahslinger told AFP that some 400 Pegida supporters showed up but that 5,000 people joined the counter-protest, shouting "Nieder, nieder, nieder mit Pegida!" ("Down, down, down with Pegida!") and waving rainbow flags.
The two groups exchanged verbal insults and chants on a chilly evening in central Vienna but were kept apart by around 1,200 police, and there were no incidents of violence reported.
Pegida (Patriotic Europeans Against the Islamization of the West) spokesman Georg Immanuel Nagel told AFP that more of the group's supporters had turned up than expected.
"There will be further demonstrations," he said.
"We need to do something to stop these vermin coming in from abroad," one Pegida supporter, an 84-year-old man who declined to give his name, told AFP.
"If we don't do something now, in 20 years we will all be wearing the veil," said a woman in her 30s who also preferred to stay anonymous.
Pegida marches in the German city of Dresden began with several hundred supporters and snowballed to reach 25,000 people on January 12 after deadly Islamist attacks in Paris.
But numbers have slid since the resignation of the movement's founder on January 21 and the subsequent departure of other senior figures.
Small offshoots have also sprung up in other German cities, some of them more radical, and in other European countries including Denmark, Switzerland and Spain.
'The real Pegida'
Unlike in Germany, Austria has a strong far-right party in the Freedom Party (FPÖ), the third-largest in parliament with around 20-percent support and which has long campaigned against immigration and "Islamization".
"In Austria the FPÖ has always been the real Pegida. We have always taken seriously the problems to do with Islamism," the party's leader Heinz-Christian Strache told News magazine in a recent interview.
Ahead of Monday's demonstration a number of anti-Muslim and anti-Semitic incidents were reported, including swastikas daubed on a Vienna mosque and etched on walls at the former Nazi concentration camp Mauthausen.
At the weekend two men were assaulted in central Vienna by four others shouting anti-Semitic slogans such as "Scheissjuden" ("shitty Jews"), media reports said. One of the victims was hospitalized and police were investigating.
That followed clashes between police and demonstrators protesting against a traditional Viennese ball organized by the Freedom Party when 54 people were arrested on Friday night.
In December unknown culprits left a pig's head and intestines in front of the door of another mosque in the capital. A street sign was changed to read "Sharia Street" in September.
Twitter users mocked Pegida's use of the flag of the failed Third Reich. Photo: Twitter