The move comes two months after the party, launched by former Nazis, joined the government and follows a series of scrapes involving its members.
In late January a regional official caught up in a scandal over a 1997 student fraternity songbook with lyrics glorifying Nazis was forced to resign.
"There's been a lot of criticism of the FPÖ, much of it unjustified, some, we have to say, justified," parliamentary group leader Walter Rosenkranz told a press conference.
"We face the latent criticism that within the FPÖ Nazi and neo-Nazi ideas are tolerated.
"No, they are not tolerated and those who think they can impose such ideas on us have nothing to do in the party," Rosenkranz said.
Former FPÖ MP and retired university professor Wilhelm Brauneder will chair the committee, which will invite contributions from researchers and representatives of independent bodies.
These will include the DOW resistance archive centre, which specialises in Nazism and neo-Nazism and has been a powerful critic of the FPÖ.
The party issued a statement Tuesday saying it "recognises without reserve the Republic of Austria, democracy, parliamentarianism and the rule of law."
The FPÖ's position on the Austrian state has long been perceived as ambiguous, with one faction still considering the country annexed by Adolf Hitler in 1938 as a German province.
"As members of the government, we have a special responsibility," said FPÖ general secretary Harald Vilimsky, adding that the party "clearly rejects Nazism, racism and anti-semitism".
Rhetoric toned down
The new committee of historians, who are to produce a first report in the autumn, was promised last month by deputy chancellor and FPÖ chief Heinz-Christian Strache.
Strache, who flirted with neo-Nazism in his youth, has toned down the hardline rhetoric and expelled party members for overstepping the mark since taking over from Jörg Haider in 2005.
The FPÖ, which has 51 MPs and ministers, is still viewed with suspicion by Austria's main Jewish organisation as well as Israel which boycotts official contact with the party.
Some have reacted with scepticism to the setting up of the commission.
The human rights group SOS Mitmensch pointed to the fact that Brauneder has had work published in a far-right Austrian magazine, along with another member of the commission, Andreas Mölzer.
Mölzer had to resign as an FPÖ candidate for the European Parliament in 2014 after he reportedly told an event that the EU was in danger of becoming a "conglomerate of niggers, where everything is chaos".
The DOW has said it will collaborate with the commission if it undertook "serious work", in particular concerning the Burschenschaften student fraternities, some of which have ultra-nationalist leanings and which count many FPÖ activists among their former members.
Rosenkranz said on Tuesday the fraternities were private organisations and couldn't be forced to co-operate but DOW head Gerhard Baumgartner warned that the commission risked being reduced to an "attempt to whitewash the party's image" if they weren't included.