But the war in neighbouring Syria has transformed parts of the southern camp into a safe haven for jihadists travelling to fight there, creating districts where even Palestinian security forces fear to tread and raising tensions among residents.
In Taware district, the black flag of the Islamic State extremist group flutters in the wind.
A photograph of IS leader Abu Bakr al-Baghdadi, who has proclaimed a "caliphate" in parts of Iraq and Syria, is plastered on the walls of a small kiosk, alongside a picture of slain Al-Qaeda chief Osama bin Laden.
Abu Hajer, a Palestinian from Ain al-Helweh who fights with Al-Qaeda affiliate Al-Nusra Front in Syria, uses the camp as a rear-base in between fighting.
"The first time, I spent three months in Syria, then I went back to the camp after I was wounded," he told AFP by Skype from Syria.
"This is how it is every time, I go and I come back," the man in his twenties said.
Wanted by Lebanese security forces, he crosses government checkpoints at Ain al-Helweh's entrances with a fake ID and "small adjustments to my appearance," like shaving his long beard.
"It's jihad in the path of Allah, who I hope will end my fighting with the martyrdom that I love," he said.
- 'Mosaic' of jihadists -
Lebanon's security forces say at least 46 men have left Ain al-Helweh to fight in Syria's four-year war, in addition to a number who "come and go."
Major General Mounir al-Maqdah, head of Palestinian security forces in Lebanon, said many of the fighters were no older than 17.
"They're from a new generation that is very reckless," he said.
Lebanon's Palestinian refugee camps offer few employment opportunities. Lebanese law bars their residents from several professions.
Under a decades-old agreement, state security forces do not enter the camps, leaving them under the control of Palestinian militant groups.
But even they are afraid of entering the three neighbourhoods of Ain al-Helweh that have come under the control of extremist groups.
Signup to our newsletter and follow us on Facebook and Twitter!
One of these groups, "Shabab al-Muslim" (the Muslim Youth), was established in Ain al-Helweh as hardliners rose to prominence in Syria. It includes militants accused of bombings and assassinations in Lebanon.
Jamal Hamad, a cleric and leading official in Shabab al-Muslim, called the group a "mosaic" of jihadists.
"There's empathy in the camp with jihadist groups... There's empathy with Al-Nusra Front, and there's some empathy with the Islamic State group," Hamad told AFP.
He said neither group had an "official" presence in the camp, but supporters were galvanised by sectarian rhetoric.
"It's alright for the Shiite Muslims to come from Lebanon to fight in Syria, and it's not alright for Sunnis to empathise with the Sunni people being killed," complained Hamad.
Syria's conflict began in March 2011 with anti-government protests but spiralled into an armed conflict that has killed more than 215,000 people.
The Sunni-led opposition has significant support among Sunnis in Lebanon, while President Bashar al-Assad's government is backed by Lebanon's Shiite Hezbollah movement, which has dispatched forces to Syria to bolster his regime.
More than 61,000 Palestinian refugees live in Ain al-Helweh, including 6,000 who recently fled the war in Syria.
The rundown alleyways are filled with potholes, and electricity wires dangle perilously above residents' heads.
Posters of Palestinian leaders adorn the walls, and armed men patrol the streets.
At the camp's entrance, Lebanese soldiers search all those entering and leaving, with female officers searching women wearing the veil.
The rise of jihadists in Ain al-Helweh has worried camp authorities, who are stepping up cooperation with the army.
They want to prevent a repeat of the 2007 battles between Lebanon's army and Palestinian Islamists in the northern Nahr al-Bared refugee camp that left dozens dead.
"There's an understanding with the army that the issue of wanted persons will be resolved calmly," said Maqdah.
Palestinian security official General Sobhi Abu Arab played down the concerns, insisting only "a minority" of residents were involved in the conflict in Syria.
He said many of those had returned because "they weren't convinced by what they were doing."