The stunning 15-minute call, starting at 2:30 pm (1830 GMT), was the fruit of a diplomatic opening forged by Rouhani's election in June on a mandate to ease confrontation with the West and lift nuclear sanctions that have pulverized the Iranian economy.
"Just now, I spoke on the phone with President Rouhani of the Islamic Republic of Iran," Obama said in a televised statement, revealing the most intriguing turn yet in relations between the Islamic Republic and a superpower it branded the "Great Satan."
"The two of us discussed our ongoing efforts to reach an agreement over Iran's nuclear program."
The impetus for the call came from Iranian officials, who US officials said told them hours earlier in New York that Rouhani wanted to speak to Obama before leaving the United Nations General Assembly.
The White House had indicated to Tehran earlier this week that it was open to an informal encounter between the leaders at the United Nations.
But the Iranians at the time said such a meeting was too complicated, raising questions as to whether Rouhani was wary of angering hardliners in Iran's clerical hierarchy.
The leaders' momentous conversation took place when Rouhani was on his way to the airport in his official limousine, the Iranian side said.
Obama spoke in English and Rouhani spoke Farsi as they chatted through interpreters, according to US officials.
But before hanging up, in an exchange that would have been thought impossible only days ago, Obama bade Rouhani "khodahafez" -- farsi for "goodbye."
Rouhani replied "have a good day, Mr president" in English, according to tweets by the Iranian leader's office and a US official.
The two countries severed diplomatic relations in 1980, a year after the Islamic Revolution in Iran toppled the US-backed shah.
The aftermath of the revolution also triggered a 444-day standoff at the US Embassy in Tehran in which a total of 52 American hostages were held by militant students until January 1981.
Obama said that, despite the unprecedented nature of the call with Rouhani and hopes for a lasting diplomatic breakthrough, he was mindful of obstacles ahead.
"The very fact that this was the first communication between an American and Iranian president since 1979 underscores the deep mistrust between our countries, but it also indicates the prospect of moving beyond that difficult history."
Obama said he told Rouhani that he believed a "resolution" was possible to the dispute over Iran's uranium enrichment program, which the West believes is a covert effort to produce nuclear weapons -- a charge Tehran denies.
Signup to our newsletter and follow us on Facebook and Twitter!
Obama noted that Rouhani had said that Iran would never develop nuclear arms and that Washington respected the right of Iranians to access peaceful nuclear energy.
"So the test will be meaningful, transparent, and verifiable actions, which can also bring relief from the comprehensive international sanctions that are currently in place," the US president added.
Washington and Israel have both warned of possible military action if diplomacy fails to assuage their concerns over the nuclear program.
The Iranian presidency confirmed the telephone call between Obama and Rouhani.
"The two insisted on political will for quick resolution to the nuclear issue, as well as paving the ground for resolving other issues and cooperation in regional issues," it said on its website.
A Twitter account run by Rouhani's office also gave details of the call.
"In phone convo, President Rouhani and President @BarackObama expressed their mutual political will to rapidly solve the nuclear issue," one tweet said.
The call took place after pathbreaking bilateral talks Thursday between Secretary of State John Kerry and Iranian Foreign Minister Mohammad Javad Zarif at the UN, on the sidelines of wider discussions on the nuclear program between Iran and major world powers.
A senior US official said that Washington had informed Israel of the call, as well as leaders in Congress.
Israeli Prime Minister Benjamin Netanyahu meets Obama at the White House on Monday, and is likely to raise Israeli doubts that Rouhani's charm offensive is genuine or will lead to a dismantling of the Iranian nuclear program.
The Israeli leader has branded Rouhani a "wolf in sheep's clothing" but a senior US official said Washington understood his concerns.
"It is entirely understandable for the Israeli government to be deeply skeptical," the official said, referring to past threats to wipe the Jewish state off the map made by the previous Iranian president Mahmoud Ahmadinejad.
Rouhani earlier continued his charm offensive in New York during a trip which set the next high level talks on the country's nuclear program in Geneva on October 15 and 16.
He said at a news conference that Iran would present a plan at the talks to resolve concerns over its atomic activities and vowed never to deviate from promises to the West.
"Iran will prepare that plan and will present it in Geneva. We hope it will serve as an even more effective step to settle the nuclear issue," Rouhani said.
Rouhani said he hoped to resolve the row in a "short period of time," a day after his foreign minister said that major powers had set a goal of a deal within one year.
The Rouhani call was the latest staggering turn for Obama's foreign policy after the United States secured agreement with Russia in New York on a deal to eradicate Syria's chemical weapons arsenal and on a UN resolution that later cleared the Security Council.
It came at a moment in which his presidency is under siege in a battle with Republicans over a budget and the national borrowing limit, which threatens to shutter the US government and could force the country to default on its debts for the first time.