Yemen's foreign ministry accused George Abu al-Zulof of lacking impartiality in his assessments of the human rights situation in the country and declared him persona non grata.
Ban said in a statement that he has "full confidence" in Zulof and urged the government "to reconsider its position on his expulsion."
The decision came just days after the United Nations raised alarm over the use of cluster bombs by the Saudi-led coalition backing Yemen's government in its war against the Huthi rebels.
The office of the UN high commissioner for human rights said Tuesday that its staff in Yemen had found remnants of 29 cluster bombs during a field visit in Haradh district in the northwest.
Ban said the Yemeni government was failing to uphold its obligations by "impeding the United Nations human rights work."
Signup to our newsletter and follow us on Facebook and Twitter!
The people of Yemen have suffered "grave human rights violations" and the UN rights office is "actively and effectively helping to document these violations," Ban said.
Yemen's foreign ministry indicated in its statement that Zulof may have been misinformed by his local staff in rebel-held Sanaa.
UN spokesman Stephane Dujarric earlier described the decision to expel Zulof as "an extremely regrettable development" and praised the official for doing an "excellent job."
Ban added that he was extremely concerned about the safety of the remaining national and international staff.
The United Nations has been increasingly alarmed by the mounting civilian toll in Yemen and the dire humanitarian crisis.
Yemen descended into chaos when the coalition began air strikes in March to push back Huthi rebels who had seized Sanaa.
More than 5,800 people have been killed and 27,000 wounded since then, according to UN figures.