The two climbers left the Solvay Hut, perched on a ridge of the mountain at 4,003 metres, to continue their descent at around 8am, cantonal police said.
About 100 metres below the hut the victim fell 600 metres down the Matterhorn’s east face, police said. It’s not clear what caused his fall.
The two men were not roped together, the force said. The dead man was reportedly from Styria.
The accident is a grim reminder of the risks undertaken by alpinists climbing the Matterhorn.
This year marks the 150th anniversary of the first ascent of the mountain overlooking Zermatt by a team led by English explorer and climber Edward Whymper in 1865.
On that climb four members of the party were killed during the descent.
Over 500 people have died climbing the mountain straddling the Swiss-Italian border since the first ascent and it claims an average of 12 victims a year.