The symbolic move by the young Shiite cleric comes after thousands of his supporters have been camping outside the restricted area's entrances for more than a week.
Sadr wants Prime Minister Haider al-Abadi to enact reforms tackling corruption and announce a technocratic government to replace the current cabinet of party-affiliated ministers.
In his most recent address, Sadr had said Abadi should announce new measures by Saturday or face further pressure from the street.
Late in the afternoon on Sunday, Sadr walked into the so-called Green Zone, not only unopposed but even apparently assisted by members of the security forces.
Signup to our newsletter and follow us on Facebook and Twitter!
"I will enter the Green Zone alone... and will sit in the Green Zone while you sit at the gates," he said in an impromptu press conference moments earlier.
The Green Zone covers a sprawling area by the Tigris in central Baghdad. It houses the prime minister's office, parliament as well as the US embassy, which is the world's largest.
He asked his supporters to be steadfast in their peaceful protest to push for reforms and warned against any violent behaviour.
"Whoever harms the security forces harms me personally," he said.
Sadr's entourage set up two foam mattresses and a small marquee to shelter him from the rain as the cleric prepared to begin his sit-in protest.
Sadr had previously given Abadi a 45-day ultimatum that expires on Tuesday but the premier has to grapple with leaders from his own political bloc who are both reluctant to relinquish their positions and rattled by Sadr's return to the spotlight.