Special forces, soldiers, police, border guards and pro-government paramilitaries are involved in the operation to retake the Anbar province town, Iraq's Joint Operations Command said in a statement.
Tanks and artillery are taking part in the operation, which is also backed by air support from Iraqi forces and the US-led coalition against IS, it said.
Rutba, located in western Anbar province along the main road to Jordan, has been held by the jihadist group since 2014.
"Rutba's important to the enemy because it's another support zone for them," said Colonel Steve Warren, the spokesman for the US-led operation against IS.
IS uses it "to stage and prepare forces for operations in... the main battle area," Warren told journalists in Baghdad last week.
Signup to our newsletter and follow us on Facebook and Twitter!
"It's not heavily defended as is Fallujah or as was Ramadi," he said, referring to the capital of Anbar, which has been retaken, and its second city, which IS still holds.
Warren said the number of IS fighters in Rutba varies from around 100 up to several hundred, and that once the Iraqis "decide they want to liberate Rutba, they'll be able to."
IS overran large areas north and west of Baghdad in June of that year, and later made further advances in Anbar, seizing Ramadi in 2015.
Iraqi forces have since regained significant ground from the jihadists, securing the Ramadi area earlier this year and retaking the town of Heet last month.
But parts of Anbar -- including Fallujah -- are still under IS control, as is most of Nineveh province, to its north.
And the jihadists are still able to carry out bombings in government-held areas -- something they did more frequently prior to the June 2014 offensive.
As IS continues to lose ground, it has in recent weeks stepped up its campaign of bombings, including three in Baghdad last Wednesday that killed nearly 100 people.