When one youth started to taunt Israeli police warily watching the gathering crowds early on Thursday, others angrily remonstrated with him.
This was a night for celebrating what the Palestinians saw as a rare victory.
For Israelis, the situation grew out of a horrible attack on July 14 that killed two policemen.
But many of them also viewed Israeli Prime Minister Benjamin Netanyahu's handling of the crisis as inadequate.
While Palestinians' joy at the removal of the security measures was somewhat tarnished by clashes inside the mosque compound later Thursday, Palestinians that night chanted and hugged each other, as car horns sounded incessantly.
A huge Palestinian flag was carried by young men onto one of the Old City's walls -- an extremely rare act in a city that Israel considers its undivided capital.
"We feel joyous. I live quite far away but I walked here for Al-Aqsa," said Nisreen, a young woman in the crowd.
"The Israelis think this is it. God willing this is just the beginning."
- Two weeks of tension -
The celebrations came nearly two weeks after the attack near the Haram al-Sharif compound, known to Jews as the Temple Mount.
Two Israeli police were killed, while the three Arab Israeli attackers were shot dead.
The site, which includes the revered Al-Aqsa mosque and the golden-topped Dome of the Rock, serves as a key unification point for Palestinians.
It is located in east Jerusalem, seized by Israel in the Six-Day War of 1967 and later annexed in a move never recognised by the international community.
It is the third-holiest site in Islam and the most sacred for Jews as the site of the first and second ancient temples.
Israel responded to the attack by installing new security measures, particularly metal detectors and cameras.
They argued such measures were standard at major religious sites but Palestinians saw it as Israel trying to take further control of the compound.
The Waqf, the Islamic endowments authority that runs Al-Aqsa, refused to enter until the measures were removed.
Days of street protests followed, with thousands praying outside the compound as part of a boycott.
Signup to our newsletter and follow us on Facebook and Twitter!
The situation came to the boil around the main weekly prayers on July 21.
Clashes between Israeli security forces and Palestinians erupted in Jerusalem and the occupied West Bank, leaving three Palestinians dead.
Later that evening, a Palestinian broke into a home in a Jewish settlement in the West Bank and stabbed four Israelis, killing three of them.
Warnings grew that the unrest could spiral out of control.
A top aide to US President Donald Trump flew in for crisis talks, while King Abdullah II of Jordan, the official custodian of Muslim holy sites in Jerusalem, urged Netanyahu to remove the security measures.
Early on Tuesday, the metal detectors were removed but barriers and other new structures remained in place. The boycott continued.
Two days later, police returned around 1:00 am to remove the rest, sparking the joyous scenes.
A poll of Israeli Jews found 77 percent thought the move constituted "capitulation", while even the normally pro-Netanyahu newspaper Israel Hayom attacked his handling of the crisis.
- 'Cut across all lines' -
Netanyahu, who heads what is seen as the most right-wing government in Israel's history, has since called for the death penalty for the Palestinian who stabbed the Israelis in what some analysts saw as a move to please his right-wing base.
"There is a strong sense of humiliation, especially among the right wing," Ofer Zalzberg from the International Crisis Group think tank told AFP.
"They are pushing the government to reverse this humiliation by giving them something else."
The Palestinian movement was called to the streets by the Waqf but quickly took on a life of its own.
"This cut across all lines -- religious, not so religious, Muslim, Christian, rich or poor," Diana Buttu, a former Palestinian official turned analyst, said.
According to Buttu and others, the Palestinian political leadership of all factions, including the internationally recognised leadership of the Palestinian Authority, had been mostly irrelevant, with the movement led largely by protesters.
"Palestinians have been very encouraged by what for them is one success within a sea of defeats," Zalzberg said.
He put the victory down to an "Israeli inability to stop the movement because of the sheer size and because it was around Al-Aqsa."
Al-Aqsa is a rare unifying symbol for all Palestinians and there is a risk they could fall back into political infighting now that the immediate threat has been defeated, he added.
But Zalzberg said the mostly young people who had taken part in the two weeks of protests will be keen to push their leaders.
"The next time there is a major issue, will they not go back to the same religious authorities and tell them: 'You were successful with the metal detectors. Why don't we do something?'"