Police detained the two men, aged 16 and 18, in the Leibnitz district in Styria state last Thursday after a tip-off from German authorities, the prosecutor's office in the city of Graz said.
While the younger brother is accused of belonging to the Islamic State (ISIS) group, his older sibling is thought to have fought for the Ahrar al-Sham militia, the office added.
The pair allegedly fought on opposite sides during the Syrian civil war, which has so far claimed more 250,000 lives and forced millions to flee their homes since it broke out in March 2011.
They face charges of committing terrorist acts, including attempted murder and torture, Austrian prosecutors said, adding that a third brother had also been arrested in Germany.
The arrests are likely to add fuel to an already heated debate in Austria over how to handle Europe's worst migrant crisis since 1945.
The Alpine nation of 8.5 million people expects to see a record 95,000 asylum claims by the end of this year, compared to 28,000 for the whole of 2014.
Many of those arriving are unaccompanied boys and young men, which has prompted far-right and conservative politicians to warn that the refugee flow may pose a threat to national security.