The proposal comes two years after the European Court of Human Rights accused Austria of violating the European Convention on Human Rights by not clearing the criminal records of those convicted under former anti-gay laws.
In 1971, Austria legalized homosexuality, but the age of protection for homosexual relationships was set at 18 years, whereas it was 14 for heterosexual couples. It was lowered to 14 for everyone in 2002.
The 1971 legislation also made it illegal to be a gay male prostitute and to endorse homosexuality, or be part of an LGBT group or association. Those laws were scrapped in 1989 and 1997.
However, the proposed draft law doesn't mean everyone whose criminal register carries a sentencing under one of these laws will automatically be pardoned.
The current draft requires anyone carrying one of those sentences to apply for a pardon - and a relative or the district attorney's office can also apply for someone to be pardoned.
Their case will then be taken to a court, in a non-public hearing, and it will be decided if their conviction will be deleted.
According to LGBT group Rechtskomittee LAMBDA, a decision would be based on whether their actions would be legal today.
The proposed new legislation is open for appraisal until September 18th, meaning relevant groups can give statements on the proposals, which will then be considered by the Ministry of Justice before the draft law goes before parliament.