Because there was no outright winner, a second round of voting will now be held, with the date depending on any appeals, the head of the ISIE elections body Chafik Sarsar told reporters.
Marzouki secured 33.43 percent of votes cast in an election European Union observers on Tuesday called "pluralist and transparent".
Leftwing figurehead Hamma Hammami came third with 7.82 percent, followed by London-based Islamist entrepreneur Hechmi Hamdi (5.75 percent) and wealthy businessman and football club president Slim Riahi (5.55 percent).
Sunday's election was the first time Tunisians had voted freely for their head of state since independence from France in 1956.
The North African nation that sparked the Arab Spring had known just two presidents -- "father of independence" Habib Bourguiba and Zine El Abidine Ben Ali, who was forced to flee on January 14, 2011 by a popular uprising.
Marzouki was elected president at the end of 2011 by the National Constituent Assembly under a coalition deal with the then ruling Islamist Ennahda party, which came second in a parliamentary election last month behind Nidaa Tounes.
Runoff battle begins
Annemie Neyts-Uytterbroeck, who headed the EU observer mission in Sunday's presidential election, described any irregularities as "minor".
"The exercise of freedom of expression and assembly was guaranteed," she said.
The contest for the expected runoff had begun even before the first round result became official on Tuesday.
Wasting no time in relaunching the battle after Sunday's vote, Essebsi called Marzouki the candidate of "jihadist Salafists", to which his rival countered by calling for "a debate on policies... not (a campaign of) insults".
The election is a milestone for Tunisia, where the ouster of long-time strongman Ben Ali set off a chain of revolts that saw several Arab dictators toppled by citizens demanding democratic reform.
Signup to our newsletter and follow us on Facebook and Twitter!
US Secretary of State John Kerry hailed Sunday's vote as an "historic moment" in Tunisia's transition to democracy, and pledged Washington's support.
His French counterpart, Laurent Fabius, called for Tunisia's transition towards building new institutions to remain "inclusive and democratic".
Spared the chaos
Tunisian leaders pride themselves on the fact the country has been spared the bloodshed that has ravaged other Arab Spring states such as Libya and Yemen.
Despite fears of disruption by Islamist militants, polling on Sunday passed off peacefully.
But the runoff is set to be polarising, with Marzouki's camp portraying him as the last line of defence against a return to the autocratic ways of the old regime, and Essebsi deriding him as an Islamist pawn.
Marzouki in a speech urged "all democratic forces" to back him against Essebsi, who served under both Ben Ali and Bourguiba.
"I am now calling on all democratic forces... alongside whom I have campaigned for the past 30 years for a real democracy, for a break with the past, for a genuine civil society and for a separation of powers," he said.
Marzouki argues that only he can preserve the gains of the uprising, while his critics say he hijacked the spirit of the revolution by allying himself with moderate Islamists Ennahda in 2011.
Ennahda rule was marred by a surge of radical Islamism and the assassination of two leftist politicians by jihadist suspects.
Essebsi insisted on Monday that only he could defend Tunisia against the threat of Islamist extremism.
"The people who voted for Marzouki were the Islamists... that is to say Ennahda members... but also the jihadist Salafists," he told French radio station RMC.
If Essebsi wins he will still have to form a coalition government, even with Ennahda, because Nidaa Tounes fell short of securing an absolute majority in October.