An intelligence officer and a local official said that 162 out of 170 men were released after being taken by IS from the villages of Al-Shajara and Gharib in Kirkuk province, where two of the jihadist group's flags were torched.
A 39-year-old from Al-Shajara who was among those detained said that they were taken by pickup truck to an open area where they were bound and questioned about who burned the flag.
He and other detainees were kept overnight in houses with five to 10 people per room, after which all but eight were released, the man said.
A resident of Al-Shajara confirmed that dozens of people had returned to their homes in the area.
Signup to our newsletter and follow us on Facebook and Twitter!
IS has previously turned to mass detentions as it seeks to quell resistance in the swathes of territory in Iraq that it has overrun since June.
It seized 50 people in Kirkuk province after residents burned one of their positions and flag in September, and 20 more the following week for allegedly forming a resistance group.
Some of those seized were subsequently released.
But the group has also executed thousands of people in areas it controls in Iraq and Syria, sometimes in grisly beheadings it videotapes and posts online.