On a stage that entertained the 12,000 people with a choir singing songs of peace and hope, speakers told of their desire to bring peace to the world, and to call for unity and resistance against terrorism.
The rally, under the banner "Together against terrorism" was called by the Austrian government and religious communities.
It was held simultaneously with a massive march in the French capital Paris that drew more than one million people.
A government statement said that Austria would not be cowed by the attacks. “No one can extinguish our democracy and our freedom,” said the statement, read by two well-known Austrian actors. Others read texts by famed German writers focusing on equality and brotherhood.
Rainbow flags promoting tolerance, as well as Kurdish flags reminding viewers of the people who are bearing much of the brunt of the Islamic extremism in Syria and Iraq, waved defiantly in the cold, damp air.
The Vienna State Opera Choir sang works by Mozart and Verdi.
The EU and the Austrian national flags flew at half-mast on the Hofburg palaces overlooking the square, affirming Austria's sympathy for those who died in the barbaric acts of terrorism in France.
Half of Austrians believe that Islam is a threat to the stability of the country. Meanwhile in France, millions of people took to the streets to show their support for resistance against extremism and violence.