The overcrowded and impoverished Ain al-Hilweh camp near the southern coastal city of Sidon has gained notoriety in recent years as a refuge for Muslim extremists and fugitives.
It also saw deadly fighting last year between the Jund al-Sham Islamist group and members of Palestinian president Mahmud Abbas's Fatah movement.
And in September the army said security forces had arrested a Palestinian refugee suspected of links to the Islamic State group who was in the camp.
"The construction of the wall began some time ago and the aim is to stop the infiltration of terrorists inside Ain al-Hilweh from nearby orchards," the military source told AFP.
"It's a security measure" that was taken after the arrest of "fugitive terrorists" who had taken shelter in the camp, he said.
Pictures were posted online showing cranes lifting huge concrete blocks on the western side of Ain al-Hilweh then setting them side by side, as well as watchtower.
Social media users compared the wall to a controversial separation barrier which Israel has been building in the occupied West Bank since 2002.
Signup to our newsletter and follow us on Facebook and Twitter!
"Soon, the children of Ain al-Hilweh will draw pictures depicting Palestine and freedom on the wall of shame," one person said online.
A camp official, Fuad Othman, called the wall a "provocation".
Major General Mounir al-Maqdah, the head of the Palestinian security forces in Lebanon, criticised the construction of the wall.
"The wall, parts of which have been erected, is causing psychological pressure for the Palestinian refugees," said Maqdah.
"We wouldn't have needed a separation barrier and watchtowers if the Lebanese authorities had, years ago, found a solution to the Palestinian presence in Lebanon," he added.
The military source said Lebanon "is not building a prison or a separation wall, but a wall for protection", adding residents would be able to go in and out from the camp, except from the western side.
By long-standing convention, the army does not enter Palestinian refugee camps in Lebanon but holds positions outside of it, leaving the factions to handle security inside.
More than 61,000 Palestinian refugees live in Ain al-Hilweh, including 6,000 who recently fled the war in Syria, according to the UN's agency for Palestinian refugees, UNRWA.