Iraqi Prime Minister Haider al-Abadi received Jubeir and his accompanying delegation, a statement from his office said, a key step in efforts to normalise frosty ties.
Both sides "discussed cooperation in various fields, including the fight against the Daesh gangs," it said, referring to the Islamic State group Iraqi forces are currently battling in the northern city of Mosul.
The Saudi minister also met his counterpart Ibrahim al-Jaafari, who said in a statement the visit was "the first by a Saudi foreign minister since 2003."
"This visit is to reestablish relations in a more stable way than previously," a senior government official said told AFP on condition of anonymity. "It's the first visit of its kind."
Abadi, who has been at the helm since 2014, has supported efforts to improve strained ties but the road to normalisation has been rocky.
Thamer al-Sabhan, whose credentials were received in January 2016, became the first Saudi ambassador to Iraq in a quarter century, after relations were cut following ex-president Saddam Hussein's 1990 invasion of Kuwait.
He left the same year after Baghdad demanded he be removed following remarks he made to the press about an alleged plot to assassinate him and criticism he voiced of the Hashed al-Shaabi.
Hashed al-Shaabi (Popular Mobilisation) forces, which have played a key role in the fight against the Islamic State (IS) group, are a paramilitary umbrella dominated by Shiite militia and seen by Riyadh as a proxy for arch-rival Iran.
Jaafari was one of the most vocal critics of Saudi Arabia at the time and issued several strongly-worded statements against the kingdom and Jubeir himself.
Signup to our newsletter and follow us on Facebook and Twitter!
He told him directly on the sidelines of a global conference on the anti-IS war last year and in a statement expressing Iraq's "annoyance" over what he called "unacceptable interference."
Saudi Arabia is very unpopular among Iraq's Shiite majority and often accused of direct support to the IS jihadists that took over a third of the country in 2014.
Saudi Arabia nominally supports the fight against IS but Iraq and other partners have argued it needs to do more to help durably defeat IS and its ideology.
As evidenced in recent attacks, IS has retained its ability to sow chaos and undermine the Iraqi government's authority even after it lost fixed positions in various regions.
Several of them have occurred in Iraq's vast Anbar province, which has long and porous borders with Saudi Arabia, Jordan and Syria.
"Jubeir congratulated Iraq on the victories achieved against Daesh and pledged Saudi Arabia's support to Iraq in fighting terrorism," the statement from Abadi's office said.
It said the minister had also "expressed Saudi Arabia's willingness to back the stability of liberated areas."
While Iraq has often suffered from being turned into a battlefield where the rivalry of its neighbours Iran and Saudi Arabia -- the region's Shiite and Sunni Muslim powerhouses -- played out, the Iraqi government official said there was an opportunity for Baghdad.
"The whole region is heading towards compromise and Saudi Arabia sees Iraq as an important player to have on its side," he said. "Iraq's neutrality could make it a ground for Iranian-Saudi rapprochement."
He stressed that despite intense pressure from the street to take a strong stand against Saudi Arabia, Abadi -- who belongs to Iraq's largest Shiite political bloc -- had "never indulged in aggressive rhetoric against Saudi Arabia."