It’s not clear yet how many jobs will be affected in Austria, where Siemens currently employs 10,400 people.
In a statement, the company said it was hoping to "streamline administrative and overhead functions".
"With our business concept Vision 2020, we want to bring the company back to sustainable growth and close the profitability gap between us and our competitors," said Siemens CEO Joe Kaeser.
The move is part of the company's ongoing restructuring plan to find €1 billion of savings, which started in May 2014. The company hopes to achieve that through innovation and smart investment.
Janina Kugel, a board member and labour director, said the company wanted to start talks with employee representatives about the cuts in Germany as soon as possible and "search constructively for socially responsible solutions".
Last month, Siemens posted a 25 percent drop in its net profit in the first quarter of the fiscal year, which it blamed on interest rates, but falling oil prices also meant struggles in its important energy sector.
It was previously announced that 1,200 jobs would be cut from the Siemens Energy business.
In 2013, 15,000 people lost their jobs at Siemens under Kaeser. His predecessor, Peter Löscher had cut 17,000 workers during his time at the helm.
In all, some 343,000 people work for Siemens globally. Around a third (115,000) of those employed with the company are in Germany.
IG Metall, the union that represents many Siemens workers, did not have immediate reaction, though chairwoman Birgit Steinborn said "I'm sick of companies presenting job cuts as though there were no alternative solutions."
In unrelated news, global IT solutions powerhouse IBM is laying off 26 percent of its workforce around the world, as part of Operation Chrome, according to an unconfirmed report from Forbes Magazine. IBM analysts were expecting the layoffs, although surprised by their scope, as the company has experienced 11 straight quarters of declining revenues.