Surveillance video seemingly from the moment of the attack that spread online showed the two men, dressed in black suits and ties, calmly walking into a cafe before pulling out guns and opening fire on its terrace.
Most patrons fled in panic, though some fought back at the cafe at Sarona Market in Israel's commercial capital.
At one point, one of the attackers violently threw a handgun to the ground in frustration as two of the victims lay motionless on the terrace.
Five people were wounded in addition to the four killed.
With the attack causing shock among Israelis, officials said they were suspending entry permits for 83,000 Palestinians during the holy Muslim month of Ramadan in a move that was likely to further stoke tensions.
Those killed were all Israelis, identified as Ido Ben Aryeh, 42; Ilana Nave, 39; Michael Feige, 58; and Mila Mishayev, 32, police said.
Police said one of the attackers was arrested, while the other was wounded by gunfire and had undergone surgery.
They were identified as Khaled Mohammad Makhamrah, 22, and his cousin Mohamad Ahmad Makhamrah, 21, both from the Hebron area in the occupied West Bank.
The market and complex of bars and restaurants is located across the street from Israel's defence ministry and main army headquarters.
Prime Minister Benjamin Netanyahu visited the scene of what he called the "cold-blooded terrorist murder" after returning from a trip to Moscow and conferred with senior colleagues, including newly installed hardline defence minister Avigdor Lieberman.
"We discussed a range of offensive and defensive steps which we shall take in order to act against this phenomenon," Netanyahu's office quoted the premier as saying.
"There will be intensive action by the police, the army and other security services, not just to catch every accomplice to this murder but also to prevent further incidents."
The shooting will serve as a first major test for Lieberman, sworn in on May 30 and who has in the past threatened severe action against Palestinian "terrorists."
A spokesman for Hamas, the Islamist movement that runs the Gaza Strip and which is also present in the West Bank, called the attack a "heroic operation."
It was not yet clear if the attackers were acting alone of as part of a larger plot.
One of Israel's first responses was to revoke tens of thousands of entry permits.
"All permits for Ramadan, especially permits for family visits from Judea and Samaria to Israel, are frozen," said a statement from COGAT, the defence ministry unit which manages civilian affairs in the occupied West Bank.
Signup to our newsletter and follow us on Facebook and Twitter!
Israelis refer to the West Bank by its biblical names, Judea and Samaria.
It said that 83,000 Palestinians would be affected, adding that hundreds of residents of the Gaza Strip who had received permits to visit relatives and holy sites during Ramadan would also have access frozen.
COGAT also froze permits for 204 relatives of one of the alleged attackers.
Stalled peace efforts
Israel's army locked down the Palestinian town of Yatta, where the attackers were from, with soldiers patrolling and stopping cars as they entered and exited.
The shooting drew international condemnation, including from the United States and European Union.
UN Special Coordinator for the Middle East peace process, Nickolay Mladenov, said "all must reject violence and say no to terror".
"I am also shocked to see Hamas welcome the terror attack. Leaders must stand against violence and the incitement that fuels it, not condone it," he said in a statement.
Violence since October has killed at least 207 Palestinians, 32 Israelis, two Americans, an Eritrean and a Sudanese.
Most of the Palestinians killed were carrying out knife, gun or car-ramming attacks, according to Israeli authorities.
Others were killed in clashes or by Israeli air strikes in the Gaza Strip.
The violence has steadily declined in recent weeks, though attacks have continued to occur.
Shooting attacks have been rare, though Tel Aviv has seen two other major incidents in recent months.
In March, as US Vice President Joe Biden visited, a Palestinian went on a stabbing spree along the Tel Aviv waterfront, killing an American tourist and wounding 12 people.
On January 1, an Arab Israeli killed three people in a rampage in Tel Aviv.
Many analysts say Palestinian frustration with Israeli occupation and settlement-building in the West Bank, the complete lack of progress in peace efforts and their own fractured leadership have fed the unrest.
Israel says incitement by Palestinian leaders and media is a leading cause of the violence.
Last week in Paris, representatives from 28 countries, the Arab League, European Union and United Nations met to discuss ways of restarting Israeli-Palestinian peace efforts.
Negotiations have been at a complete standstill since a US-led initiative collapsed in April 2014.
France hopes to hold an international peace conference before the end of the year.
Israel strongly opposes the French plan, calling instead for direct negotiations, while the Palestinians support it.