The crowd includes people in wheelchairs and on crutches, as well as parents carrying children on their shoulders, all prepared to march 175 km to the border.
Some flashed victory signs while others waved images of German Chancellor Angela Merkel, who recently announced that Berlin was easing asylum restrictions for Syrians.
"We are very happy that something is happening at last, The next stop is Austria. The children are very tired, Hungary is very bad, we have to go somehow," 23-year-old Osama from Syria told AFP.
Police watched the silent migrants walk through the Hungarian capital but did not intervene, an AFP correspondent said, adding there was no sign of conflict at the moment.
The march was causing traffic jams on the main route into the city from the western Buda area.
An estimated 2,000 refugees have been stuck in makeshift camps at Budapest's Keleti railway station hoping for trains to Austria and Germany, but international services have been suspended from the station.
Railway authorities have blocked refugees from boarding trains to Austria and Germany because they lack EU visas. Currently passengers wishing to travel to Austria have to change trains at Hegyeshalom.
Austrian Railways (ÖBB) head Christian Kern has said that trains are on stand-by at the Hungarian border, equipped with beds, and will be prepared to take refugees on to Vienna. He said that the ÖBB had been "deliberately misinformed" by the Hungarian authorities on Thursday, about a train which left Budapest and was supposed to be headed towards Austria.
BBC news producer Ron Brown has spoken to refugees at the railway station who say they plan to walk to Vienna.
Mahmoud from Syria has decided to join many others in attempting to walk to Vienna from Budapest. pic.twitter.com/TTN46lNyaM— Ron Brown (@ronbrown01) September 4, 2015
Meanwhile, a group of activists in Vienna has set up a Facebook group appealing to people to join a convoy of private cars and buses which will travel to Budapest on Sunday and bring back refugees across the Austrian border.
On Friday four activists from Vienna were stopped in Budapest on suspicion of people smuggling, and taken to a police station. They had planned to bring refugees by car across the border.
Austrian Foreign Minister Sebastian Kurz has been speaking to the Hungarian Ambassador and told the Austria Press Agency that the four will be allowed to return to Austria later on Friday. They face up to five years imprisonment in Hungary if found guilty of people smuggling.
"There is a difference between organized smugglers who are prepared to risk people dying to make a profit and those people who just want to help," Kurz said, adding that there is an urgent need for a European response to the refugee crisis.
A convoy of activists from Vienna plans to leave the car park at Vienna's Prater stadium at 9am on Sunday with the aim of bringing back as many refugees as possible, and helping them to continue their journey from Vienna to Germany.
The group has appealed to charities, NGOs, bus and taxi companies to support the convoy and has asked the Austrian government to persuade Hungary to let refugees leave on trains from Keleti station, to prevent them falling into the hands of unscrupulous people smugglers.
In a statement, the group said that it has taken legal advice and will not allow any volunteers or refugees to take unnecessary risks.
However, Vienna's Vice Mayor Maria Vassilakou from the Greens party has warned people against joining the convoy. "As much as I understand the need for such action, and admire the dedication and courage behind it, Hungary is not a safe country," she said on Friday.
Hungarian MPs are debating tough new anti-immigration measures on Friday afternoon, including criminalising illegal border crossing and vandalism to the new anti-immigrant razor-wire fence erected along the border with Serbia.
Hungary has in recent months joined Italy and Greece as a "front line" state in Europe's migrant crisis, with 50,000 people trekking up the western Balkans and entering the country in August alone.
A record 3,300 migrants crossed into Hungary on Thursday, according to the latest figures from the UN refugee agency.
The right-wing government headed by Prime Minister Viktor Orban has responded to the influx by erecting a controversial razor-wire barrier along its 175 km border with Serbia.
On Thursday, Orban defended his handling of the crisis, blaming Germany's lifting of asylum rules for the thousands of migrants travelling through his nation.
"Nobody wants to stay in Hungary, neither in Slovakia, nor Poland, nor Estonia. All want to go to Germany. Our job is just to register them," he said while in Brussels for talks on the crisis with European Commission chief Jean-Claude Juncker and EU president Donald Tusk.