Tensions were high as hundreds of police, some in riot gear, began moving people out of Keleti station.
Although they offered no resistance, several hundred migrants staged an angry demonstration outside the station, an AFP correspondent said.
"Germany! Germany! We want to leave!" chanted the crowd, with some holding their babies up in the air.
The move came just 24 hours after Budapest police allowed people stuck for days in makeshift refugee camps to leave the Hungarian capital on trains bound for Germany and Austria, despite many not having EU visas.
The decision led the highest number of migrants entering Austria in a single day this year, police confirmed, with 3,650 arriving in Vienna by train
"We are still in the process of verifying how many of them are actually asylum-seekers," Austrian police spokesman Patrick Maierhofer told AFP.
Unrest among migrants on the streets of Budapest.
Many of the migrants slept at Vienna's Westbahnhof station, hoping to continue on their journey to Germany, which last week eased asylum
restrictions for Syrian refugees.
Europe is struggling to cope with its biggest movement of people since World War II, with more than 300,000 arriving this year, many fleeing war,
persecution and poverty in the Middle East and Africa.
The plight of the refugees was brought sharply into focus last week after 71 people, including four children, were found dead in an abandoned truck on an Austrian motorway near the Hungarian border.
So far, police in Hungary and Bulgaria and have arrested seven people in connection with the truck tragedy, among them four Bulgarians and one Afghan.
The nationalities of the other two were not known.
The grim discovery led to a security crackdown in Austria with massive tailbacks forming along the border on Monday, as officers inspected vehicles
in search of people-smugglers and migrants.
'We want to leave!'
The escalating situation has divided the 28-member bloc ahead of emergency talks on September 14, with western European leaders calling for greater efforts to help the new arrivals as countries on its eastern borders warned they were struggling to cope.
German Chancellor Angela Merkel warned Monday that the refugee crisis was testing the core ideals of universal rights at the heart of the EU, urging all
states to take their fair share of asylum seekers.
She also made veiled criticism of countries like Slovakia which has said it would reject migrants from majority Muslim countries. "If we start saying 'I do not want Muslims' ... that cannot be good".
At heart of the crisis lies the question over how to distribute the migrants across the EU and help relieve pressure on so-called "frontline" nations where migrants arrive by sea or land.
Much-flouted EU rules stipulate that refugees should be processed in the first country they reach.
But EU member Hungary -- like Austria, a country of transit for migrants heading to northern Europe -- says it cannot host record numbers of newcomers,
as 50,000 migrants arrived in August.
Arrivals in Germany
German police registered a record 2,200 migrant arrivals in a key southern border region with Austria in the 24 hours to Tuesday morning, federal police said.
The arrivals spiked as passenger trains, each packed with hundreds of refugees, left Hungary and crossed through Austria to head into Germany's
southern Bavaria state.
It was the highest number recorded by any of Germany's 10 federal police districts since the start of the migrant crisis that is expected to bring 800,000 newcomers to Europe's top economy this year.
The numbers were registered by the Munich police district, which stretches from the Bavarian state capital down to the Austrian border, between 0600 GMT Monday and 0600 GMT Tuesday, a police spokesman told AFP.
Fake Syrian passports
Budapest's right-wing government on Monday criticised Berlin's easing of asylum rules as "(building) up the hopes of illegal immigrants".
Hungary's own response has been to build a razor-wire fence along its border with Serbia to try to keep migrants out.
As vast numbers surge towards Europe with the hope of claiming asylum, a brisk trade in false Syrian passports has emerged, predominantly in Turkey,
the EU's Frontex border agency said on Tuesday.
"There are people who are in Turkey now who buy fake Syrian passports because they know Syrians get the right to asylum in all the member states of
the European Union," Fabrice Leggeri told French radio station Europe 1.
"People who use fake Syrian passports often speak Arabic. They may come from North Africa or the Middle East but they have the profile of economic
migrants," he said.
Meanwhile, 20,000 people took to the streets of Vienna late Monday in a show of support for migrants, while senior government officials attended a
church service for the 71 refugees found dead last week.