Hundreds of faithful gathered inside the town's burnt out Immaculate Conception church for mass before starting the traditional Palm Sunday march, a procession during which palms are carried to commemorate Jesus's entry to Jerusalem.
"Thank God, we are returning to our towns and churches after two years," Abu Naimat Anay, an Iraqi priest, said inside the church, which is Iraq's biggest and where jihadist inscriptions were still visible on the walls.
Qaraqosh, with an overwhelmingly Christian population of around 50,000 before the jihadists took over the area in August 2014, was the largest Christian town in Iraq.
It was retaken by Iraqi forces late last year as part of a massive offensive to wrest back the nearby city of Mosul from IS but it remains almost completely deserted.
The area is now considered safe and the Palm Sunday mass and march were secured by the Nineveh Plain Protection Units, an Assyrian militia.
Speakers playing hymns were set up near a large cross recently erected at the entrance of the town.
The archbishop of Mosul, Yohanna Petros Mouche, moved back to the town last week but it needs to be extensively rebuilt and basic services restored before displaced Christians can return en masse.
Signup to our newsletter and follow us on Facebook and Twitter!
- 'We will stay' -
After the Palm Sunday service, he praised the high turnout and told AFP: "This is a message of trust and hope to our neighbours."
When the Islamic State jihadists took over Mosul, the country's second city, the Christian minority there was told to convert, pay tax or face death.
"Honestly, this makes the heart happy and sad at the same time, because we were torn away from our birthplace and this kind of devastation we didn't even see during the wars of the 80s and 90s," Aby Naymat Anay said.
Many of the more than 120,000 Christians believed to have fled their homes when IS swept across the region less than three years ago moved in with relatives or into camps in the nearby autonomous region of Kurdistan.
"There is a mixed feeling but sadness dominates. We fled to Arbil and we are not back yet," 62-year-old Yusef Nisan Hadaya said, referring to the Iraqi Kurdish capital.
The celebration in Qaraqosh already had a sombre mood when news broke among the faithful that IS had attacked two churches in Egypt, killing at least 38 people.
"The Christians are persecuted, but no matter how much they target us, our belief in God is great and we will stay here because we are not outsiders, we are the owners of the land," the archbishop told AFP.