The drawing "Zwei Liegende" ("Two Reclining Figures") is one of four works by Klimt and fellow pioneer of Viennese Modernism Egon Schiele that have been at the centre of a drawn-out legal battle between the city of Linz and the heirs of artist and collector Olga Jäger.
Jäger had given the works to a museum in Linz on long-term loan in 1951.
But when her descendants asked for them back in 2006, they were nowhere to be found.
They took the city to court and after a long legal battle were awarded damages of 8.21 million euros ($10.1m).
Now city authorities say that the Klimt drawing has been tracked down.
A secretary took the work and hid it for years in a cupboard, before leaving instructions that it be returned to the city after her death.
From next month until May the drawing will be included in of one of the dozens of exhibitions being organised in Austria to commemorate the centenary of the deaths of Klimt and Schiele.
It will then be returned to Jäger's descendants as long as they forfeit the compensation awarded for the work.