"Nazis out!" chant the demonstrators, many clad in black hoodies and sporting sunglasses despite the rain, their shouts ricocheting off the three-storey residence.
This is no ordinary town and no ordinary house: behind the fading facade of number 15 Salzburger Vorstadt Street hides the birthplace of Adolf Hitler.
And almost exactly 70 years since the Nazi dictator's suicide in the ruins of Berlin, it continues to stir up controversy.
The building has been empty since 2011 as its owner and the government remains embroiled in a bitter legal battle over its future.
Only a memorial stone gives a clue to the house's significance, reading: "For Peace, Freedom and Democracy. Never Again Fascism, Millions of Dead Warn."
Hitler may have just spent the first few weeks of his life here, but the property has been a thorn in Braunau's side for decades.
It belongs to local resident Gerlinde Pommer whose family has owned the building for more than a century, except for a brief period during World War II.
In 1972, the Austrian government signed a lease with Pommer to prevent the premises from becoming "a pilgrimage site" for neo-Nazis, interior ministry spokesman Karl-Heinz Grundböck told AFP.
Under the deal, the 800-square-metre (8,600-feet) structure can only be used for socio-educational or bureaucratic purposes.
At first the agreement - which earns Pommer €4,800 ($5,150) per month in rent - appeared to settle the moral dilemma: the Hitler house, as it is generally known, became a centre for people with disabilities.
In many ways, this represented an "ideal" solution, according to local newspaper editor Monika Raschhofer, because it offered a home "specifically to those who were eradicated under the Hitler regime".
But in 2011, the arrangement came to an abrupt end when Pommer unexpectedly refused to grant permission for much-needed renovation works.
Now, the state has decided to take more radical steps.
"We have made a purchase offer but at the same time we are also looking into the legal possibility of expropriation," said Grundböck.
'Banality of evil'
The issue has generated strong discussion among the 17,000 residents of Braunau, near the German border, and beyond. Some want it to become a refugee centre, others a museum dedicated to Austria's liberation.
There have even been calls for its demolition - but the house is part of the city's historic centre and therefore under heritage protection.
For rally organiser Astrid Hainz, the memorial stone installed in 1989 is not enough.
Authorities ought to push harder to solve the issue even if it means "forcing the owner to sell".
"The house is here and we have a duty to deal with it," she says.
One person who remains noticeably absent from the public debate is the notoriously media-shy Pommer.
"She has inherited the house and pockets the money without fulfilling the terms of the agreement," local historian Florian Kotanko told AFP. "She could make history by giving the house to the republic."
Like the anti-fascist protesters, he believes that Braunau has a historical responsibility to deal with its legacy.
"It's short-sighted to say, let it fall apart or blow it up, because the fascination won't go away," Kotanko says.
In his eyes, this obsession is linked to what philosopher Hannah Arendt famously called the "banality of evil" - the idea that Hitler was born behind such ordinary walls into an ordinary, middle-class family.
"People want to see the birthplace of someone who was capable of exterminating large parts of the population in Germany, Austria and beyond," he said.
Locals, meanwhile, resent the attention, deploring that their pastel-coloured town is known for being the cradle of one of the world's most reviled political figures, rather than for its Gothic architecture or pretty river.
"The people here don't deserve this stigma," says district commissioner Georg Wojak. "Braunau's only crime was that Hitler was born here."
Try as it might, the town is unlikely to break the chains that tie it to Adolf Hitler.
He immortalised Braunau in the opening lines of Mein Kampf, describing as "providential" the fact that he was born in "this little town on the boundary between two German states".
And residents are learning to confront the past more openly. The days where they would point tourists looking for the building in the wrong direction are over, insists Raschhofer: "When I have guests, I always show them the Hitler house and the memorial stone. It's part of Braunau."
By Nina Lamparski, AFP