"Because of the massive overcrowding of trains coming from Hungary, ÖBB has to suspend train services to Hungary temporarily," the company said in a statement.
The suspension covers both Railjet intercity trains between Vienna and Budapest and regional cross-border services, it said.
It also called on volunteers and bus companies to stop bringing people to train stations.
"The flow (of people)... and the large numbers of people waiting at stations to travel onwards has outstripped available capacities since this morning," ÖBB said.
The announcement came after 3,700 people crossed the border on foot from Hungary overnight, many of whom were then taken to stations in Vienna to board trains to Germany.
Several hundred more arrived by train from Hungary, while a further 1,000 crossed the border on foot on Thursday morning, according to police.
An AFP reporter at Budapest's Keleti main station said that at the time of the Austrian announcement, around 500 migrants were held back by police from entering the platform where Railjet trains to Vienna had been departing all morning.
Earlier, police reported around 2,000 migrants were in Keleti - the scene of high tensions last week with thousands stuck in a makeshift refugee camp unable to board trains - trying to board trains to the Austrian capital.
Police said that some 2,500 migrants arrived at Vienna's Westbahnhof train station on Thursday. Another 3,000 were still at the Nickelsdorf border crossing point.
"We are doing everything possible, but this is no longer enough," ÖBB spokesman Michael Braun told the Austria Press Agency.
He also said that for now - unlike last weekend when at least 15,000 passed through bound for Germany - no special trains for Germany were planned, and that only a limited number of migrants were allowed to board the regular trains to Germany.
Interior ministry spokesman Karl-Heinz Grundböck said that the situation "is like last weekend, with the difference that this time we have limited rail capacities."