The Vienna meeting took place three weeks after Moscow thrust itself into the heart of the crisis by launching a bombing campaign in support of the Syrian president, that has drawn sharp condemnation from the west.
And Russia and Jordan said they had agreed to "coordinate" their military operations in Syria.
The crunch talks at a Vienna hotel brought together Russian Foreign Minister Sergei Lavrov and US Secretary of State John Kerry with their Saudi counterpart Adel al-Jubeir and Turkey's Feridun Sinirlioglu.
Washington, Riyadh and Ankara -- which all back groups battling against Assad -- were sounding out Lavrov after the embattled Syrian strongman made a surprise visit to Moscow to meet President Vladimir Putin this week.
But the atmosphere appeared chilly, and there was scant hope for progress on resolving almost five years of war with the sides at loggerheads over the future of Assad.
"The fate of the president ... must be decided by the Syrian people"
At an impromptu press conference with Jordanian Foreign Minister Nasser Judeh before the four-way talks began, Lavrov reiterated the need to ramp up the fight against "terrorism" in Syria and the need for a political solution.
He made no mention of his bilateral talks with Kerry earlier in the morning.
Lavrov insisted it was for the Syrian people to decide on the fate of Bashar al-Assad, following key talks with his US, Saudi and Turkish counterparts.
"The fate of the president of Syria must be decided by the Syrian people," Lavrov told journalists in Vienna, reiterating Moscow's opposition to Assad's removal by any outside forces.
"Our common position is that we need to boost efforts for the political process in the Syrian settlement," Lavrov said.
"This foresees the start of full-scale talks between representatives of the Syrian government and the full spectrum of the Syrian opposition, both domestic and external -- with the support of outside players."
On September 30, Russia launched a bombing campaign in Syria, which has shifted the dynamics of the brutal four-and-a-half year war -- allowing Assad's battle-weary forces to go on the offensive and overshadowing a US-led coalition bombing the Islamic State (IS) group.
The US and its regional allies have decried Russia's strikes, insisting Moscow is not focusing on IS as it claims, but on other groups fighting the regime in Damascus, and that the Kremlin's intervention will only prolong the bloodshed.
Russia has been frantically trying to get the US and its coalition partners to cooperate with its bombing campaign in Syria, and Putin on Thursday stressed the need for "joint work" to defeat "terrorism" in Syria.
In a potential step forward, Lavrov announced that Russia and Jordan -- another member of the US-led coalition -- had agreed to "coordinate" their military actions in Syria and set up a "mechanism" to facilitate that end.
While the scope of the coordination was not clear, it appears to outstrip the limited understandings Russia has with Israel and the US to avoid accidental air collisions over Syria.
"I hope this mechanism will be effective in fighting all terrorism in Syria and beyond," Jordanian Foreign Minister Nasser Judeh said.
Assad's fate remains a major stumbling block for ending the conflict in Syria.
His surprise visit to Moscow on Tuesday -- his first known trip abroad since the start of the conflict in 2011 -- has now made the Kremlin a key link to the strongman and placed it at the centre of the equation after 15 months of Western isolation over Ukraine.
Washington and its regional allies have long insisted Assad has to go for there to be any chance of a political solution to fighting that has cost more than 250,000 lives, but Moscow says it must first help him defeat IS and other "terrorists" before talks can start on any reforms.
"The aim of the US is to get rid of Assad, probably that is so, our aim is to defeat terrorism, to battle terror, and to help President Assad claim victory over terror," Putin said Thursday in the Black Sea resort of Sochi.
Speaking the same day, Kerry said that while all sides agreed on the need to find a political solution and battle IS, only "one thing stands in the way... Bashar al-Assad".
While Syria is set to dominate the Vienna talks, Lavrov and Kerry will also discuss the latest Israeli-Palestinian violence with representatives from the European Union and United Nations.
The talks are part of a flurry of diplomatic activity to end more than three weeks of bloodshed which has raised fears of a new Palestinian intifada, or uprising.