The watchdog documented cases of arbitrary detentions, beatings, forced marriages and rape by the jihadists on women who have fled the town of Hawijah, which is still under IS control.
HRW recounted the story of Hanan, a 26-year-old whose husband had already fled Hawijah, who was captured by IS fighters along with other women when they also attempted to escape the town.
IS fighters told her that her husband's flight made her an apostate and that she should marry the local jihadist leader.
When she refused, she was blindfolded, beaten with plastic cables, suspended by her arms for some time and then raped.
"The same guy raped me every day for the next month without a blindfold, always in front of my children," Hanan told HRW.
Signup to our newsletter and follow us on Facebook and Twitter!
The group said the issue was not given enough attention and that too little was being done to tackle the stigma that prevents many more victims from coming forward.
"Little is known about sexual abuse against Sunni Arab women living under ISIS rule," said Lama Fakih, deputy Middle East director at HRW.
"We hope that the international community and local authorities will do all they can to give this group of victims the support they need," she said.
The Islamic State group, which proclaimed a "caliphate" straddling Iraq and Syria in June 2014, claims to follow a pure brand of Sunni Islam as it was practised in the time of Prophet Mohammed.
But its fighters have justified the mass rape and sexual enslavement of women from the Yazidi minority -- who are neither Arab nor Muslim -- on the grounds that they are polytheists.
The accounts collected by HRW often refer to IS fighters accusing the women of apostasy, or abandoning their faith, before abusing them.