But with a far-right party sharing power and anti-immigration sentiment generally on the rise in Austria, Yilmaz -- along with potentially thousands of other Austrians with Turkish roots -- is worried he could be stripped of his citizenship.
Except in very special cases, Austria does not allow its citizens to hold dual nationality.
But the far-right and anti-Islam Freedom Party (FPÖ) -- junior partner in a ruling coalition with the centre-right Austrian People's Party (ÖVP) -- last year claimed to have received a list of Turkish voters which it said could contain tens of thousands of illegal dual nationals.
The affair drew comparisons to Britain's "Windrush" scandal, in which scores of British citizens of Caribbean origin were deported or detained because they had not collected the necessary paperwork proving their right be there.
Now many of the Austrians of Turkish origin whose names appear on the list could face a similar administrative nightmare.
Duygu Ozkan, a journalist for the Die Presse newspaper, said the dual nationality issue had become "virtually the only topic of conversation" for Austria's Turkish community.
Austria, like neighbouring Germany, invited thousands of Turkish citizens to come and work in the 1960s and 1970s, with many staying and putting down roots.
Turkish immigrants and their descendants now number around 270,000 out of the population of 8.7 million.
One of them is Cigdem Schiller, born in Austria 31 years ago to Turkish parents.
Schiller -- who handed in her Turkish passport to become an Austrian citizen when she was a teenager in 2003 -- said that because of her presence on the list, she received a letter from the authorities in February asking her to prove she doesn't illegally hold dual nationality.
After many visits to the Turkish consulate in the city of Salzburg where she lives, she says she eventually managed to do so. But the process was not easy and she argues that her case proves the list is unreliable.
Alper Yilmaz (L) with his lawyer Bernhard Hofer. Photo: Joe Klamar/AFP
Others have found themselves trapped in a similarly "Kafkaesque" situation.
Earlier this year, the Salzburg authorities stripped a Turkish man on the list of his Austrian citizenship, a decision subsequently upheld by a court, which argued that the list -- whose mysterious origin the FPÖ has never revealed -- could only have been produced by a bona fide Turkish authority.
The man's lawyer, Peter Weidisch, said his client found himself in the "extremely difficult situation of... being asked to prove that he hadn't done something".
To make matters worse, the Turkish authorities also washed their hands of him.
Weidisch said his client was told "you are an Austrian citizen, we can't help you".
Yilmaz, who runs a cafe in Vienna, has no idea how his name ended up on the list.
Born in Turkey, the 53-year-old came to Austria in the early 1980s and adopted Austrian citizenship in 1988. He says he has had no official contact with Turkey since.
Yilmaz said the Turkish consulate in Vienna suggested he go back to Turkey to pursue his case.
But the cafe owner is reluctant to go as he believes his status as an Alevi Kurd, and his opposition to the government of President Recep Tayyip Erdogan, could put him in danger.
Nationwide, 85 people have already had their citizenship revoked and the Vienna city authorities say they have hired an 26 extra staff to investigate 18,000 potential cases.
Yilmaz feels he has been caught up in political manoeuvrings outside his control.
On the one hand, Turkish President Erdogan is keen to keep Turkish communities in Europe closely tied to the motherland, not least because they count among his most loyal supporters.
According to Austrian media reports, some Turkish consular officials are reluctant to help Turks give up their nationality.
On the other hand is the anti-Turkish and outright Islamophobic rhetoric spouted by the far-right FPÖ.
Just last week the party posted a video about health insurance fraud that featured two fez-wearing cartoon criminals called Mustafa and Ali. It deleted the video after a storm of criticism.
Yilmaz says he has no political axe to grind with any party, but simply wants the government to come up with a fair solution.
"I am afraid, I have sleepless nights thinking: What happens now?" he says.
By Jastinder Khera/AFP