Observers say Reinhold Mitterlehner's announcement move will pave the way for Foreign Minister Sebastian Kurz to take over leadership of the party.
Kurz, the 30-year-old shooting star of Austria's political scene, has been hailed as the ideal candidate to boost the party's popularity and fend off the far-right ahead of parliamentary elections scheduled for late 2018.
Recent polls suggest the OeVP is lagging behind its coalition partner the Social Democrats (SPOe) and the anti-immigration Freedom Party (FPOe).
"I've had enough... It's impossible to carry out government work while dealing with internal opposition," Mitterlehner, 61, told a press conference in Vienna.
"I'm not a placeholder who fills in while someone else decides when the timing, structure and conditions are right," he added in an apparent reference to the public speculation about Kurz's party ambitions.
Mitterlehner said he would step down as party leader on May 14th. The OeVP is due to hold a meeting this weekend to pick a successor.
Chancellor Christian Kern appeared to indicate that Kurz's appointment was a foregone conclusion.
In a statement, Kern said he was offering the "OeVP and Foreign Minister Sebastian Kurz a partnership" to push through reforms in Austria.
Like other traditional centrist parties in Europe, the SPOe and OeVP have suffered the wrath of disillusioned voters over rising unemployment and a huge influx of migrants.
The parties, which have dominated Austrian politics since 1945, suffered a disastrous defeat in the 2016 presidential ballot when both their candidates were thrown out in the first round.
READ MORE: Political news and views from Austria