President Heinz Fischer was present and called for a rethinking of Europe’s migration policy. "The current policy in this area cannot continue as it is," he said. He described the disaster as a "monstrous catastrophe".
In the wake of the disaster EU countries have approved boosting maritime and border patrols in the Mediterranean, targeting people smugglers and working with migrants' home countries and the nations they travel through to try to dissuade people from making the perilous journey across the sea.
People who attended the vigil at Minoritenplatz lit candles in honour of the dead and carried placards with slogans such as "Stop the mass deaths in the Mediterranean".
Chancellor Werner Faymann attended, as well as ministers from the SPÖ, Greens and Neos.
Michael Landau, president of the Catholic charity Caritas, described the refugee crisis as a "disgrace for Europe".
"We are faced with the question of whether we want to live in a Europe that has a cemetery at its gates, and whether, if we swim in the Mediterranean, we are prepared to swim over the dead," he said.
He called for more funding for programs such as Mare Nostrum, which operated search and rescue teams off Lampedusa, but was terminated in October 2014.
The vigil was organised by Caritas, the Austrian Red Cross, SOS Mitmensch, Amnesty International and Diakonie, among others.
EU president Donald Tusk is to host an emergency summit on the migration crisis on Thursday.
President Fischer spoke to the crowd.