The kingdom, which also borders Iraq, has for years struggled with homegrown Islamists and is part of a US-led coalition carrying out air strikes against jihadists.
The operation, which began late Tuesday in the northern city of Irbid, 80 kilometres (50 miles) north of Amman, was the most significant of its kind since Jordan joined the coalition in 2014.
One member of Jordan's security forces was killed and three wounded in clashes that raged for several hours until dawn, the authorities said. Two civilians were also hurt.
Jordan's intelligence services said IS had been plotting "attacks against civilian and military sites in order to destabilise national security".
"The terrorists refused to surrender and put up strong resistance using automatic weapons," it said, adding that the dead jihadists were wearing suicide vests.
Twenty-two suspects were arrested during the operation and automatic weapons and explosives were seized, according to security officials.
They said that investigations before the raid had led to the arrest of 13 others linked to the jihadists.
Jordan has reinforced security along its frontiers since the conflict erupted in Syria five years ago.
Signup to our newsletter and follow us on Facebook and Twitter!
The US ally stepped up its air strikes on IS in February 2015 after the Sunni extremist group burned alive one of its air force pilots who crashed in Syria while on a mission.
The kingdom also faces a danger from within -- nearly 4,000 Jordanians belong to jihadists groups, mostly IS, Islamist sources estimate.
More than 400 Jordanians have been killed fighting in jihadist ranks in Syria and Iraq since 2011, according to the sources.
The Irbid operation marks a turning point in the fight against jihadists, said Mohammad Abu Rumman, a researcher at the University of Jordan's Centre for Strategic Studies.
Initially the fear was about "lone wolf" sympathisers travelling to Syria and Iraq to join IS.
But Wednesday's clashes illustrate "an evolution in the relationship between the Salafist jihadist movement in Jordan and IS."
The resistance put up by the jihadists and the amount of weapons seized are all clues to the evolution of this movement which now favours violence and closer ties with IS, he warned.
Irbid is just a few kilometres from the Syrian border where Jordanian security forces regularly detain drug traffickers and jihadists attempting to join extremist groups in Syria.
Jordan has tried and imprisoned dozens of jihadist sympathisers since toughening its anti-terrorism law in 2014.
The resource-poor country hosts more than 630,000 of the roughly 4.6 million Syrian refugees overseas, according to the UN refugee agency.
The Jordanian government gives a much higher estimate of 1.4 million, saying many of them are unregistered.
It has been braced for a new wave of refugees since Russia began intense air raids in September in support of Syrian President Bashar al-Assad.