The Muslim groups who organised the rally said that over 3,000 people attended, and that the march was not just about the veil and the headscarf but more generally about human rights and rights for minority groups.
Muslim women at the demonstration held up signs saying "No to the headscarf ban" or "I love my hijab".
The Austrian cabinet unveiled tough new immigration, integration and security policies last week that include a ban on wearing full-face veils in public, as well as a de-facto ban on headscarves for justice and police officials.
The government programme also includes beefing up surveillance and security measures, obliging migrants granted the right to stay to sign an "integration contract" and a "statement of values".
"Every woman must be able to move freely in public without harassment and discrimination - no matter what she does or does not wear," a Muslim youth organisation that took part in the protest said in a statement on Facebook.
There was some criticism of the fact that young girls were seen wearing headscarves at the march, and the former Greens MP Efgani Dönmez tweeted that he found it "sad that left-wing groups had joined the march and are now against the secular state".
Dönmez, who has Turkish heritage, has said that the burqa and niqab have "nothing to do with religion... but come from "archaic tribal cultures that want to make demean women and make them invisible".
Trauriges Schauspieler, dass es nun "links" ist gegen den säkularen Staat zu sein. Shame on you Linkswende! pic.twitter.com/IKYyNLlnZ8— Efgani Dönmez (@efganidoenmez) February 4, 2017
The demonstrators also spoke out against increased surveillance powers for police and against restrictions of demonstration rights that have been proposed by Interior Minister Wolfgang Sobotka of the centre-right People‘s Party.
The coalition government of Social Democrats and People's Party is facing a strong challenge from the Freedom Party, which like similar groups across Europe has stoked concerns about immigration, security and the EU to top opinion polls.