The moderate Democratic Left party in Greece says it will not join pro-bailout parties in a coalition without the more radical far left Syriza.
The Greek president has called the four main parties, including the centre-right New Democracy and the Socialist Pasok, to try to form an emergency government to avoid new elections.
But Syriza said it would not attend because it could not back any coalition which supported austerity.
A majority voted against last week.
“No unity government can emerge,” Fotis Kouvelis, head of the Democratic Left party, told Greek television.
“A government without Syriza would not have the necessary popular and parliamentary backing,” said Mr Kouvelis.
EU finance ministers are due to meet in Brussels to discuss the Greek crisis later on Monday.
The fear over holding new elections is that parties that oppose austerity measures that are a condition of Greece’s bailout deal might do well again in new polls, says the BBC’s Gavin Hewitt in Brussels.
And with no sign Europe’s leaders are prepared to renegotiate the deal, Greece could end up leaving the euro zone.
For the first time, some central bankers have spoken openly about the consequences of a Greek exit from the single currency, our correspondent adds.
President Papoulias had invited four parties, including Syriza, to further talks.
Both New Democracy and Pasok have so far been unable to form a new coalition.
They both agreed to swinging cuts in return for the last EU/IMF bailout, but suffered at last week’s polls.
Syriza, which came second, insists any new government must cancel austerity measures agreed in return for EU-IMF loans worth 130bn Euros ($170bn; £105bn).
Leading European figures, including European Commission head Jose Manuel Barroso, have warned that Greece must respect the terms of the bailout deal if it wants to remain in the euro.
Officials are weighing up the fallout of a potential Greek withdrawal from the euro and how that would be managed, say analysts.