Police are investigating the assault which reportedly took place at a swimming pool in the town of Mistelbach, around 60 miles north of Vienna.
According to the Heute newspaper, the girl and witnesses told police that the man had a dark complexion and was a foreigner.
Following an emergency meeting after the incident, local authorities decided to post a sign in English at the swimming pool that read “today no entry for refugees”.
“This decision was not taken lightly, but I take responsibility,” said local mayor Alfred Pohl, who described news of the incident as a “shock”.
Pohl also confirmed that they are hiring two security guards to patrol the area.
“The ban on asylum seekers from the swimming pool is temporary, until the security is in place,” he confirmed.
Six year sentence for pool rape
It also follows a separate incident at a pool in Vienna late last year, where an asylum seeker from Iraq raped a 10-year-old boy in toilet.
Twenty-year-old Amir A., who is married with a child of his own - told police at the time that it had been a "sexual emergency" when he attacked the schoolboy.
He pleaded guilty in court in Austria in April and was sentenced to six years this month for the serious sexual abuse of a minor and rape.