Ayatollah Mohammad Yazdi, 83, was a deputy speaker of parliament after the 1979 Islamic revolution and headed the judiciary for a decade until 1999.
He gained 47 of the 73 votes cast at a closed-door meeting in Tehran, according to the website of state television, citing officials.
Yazdi was among five contenders whose names had been linked to the post by Iranian media in recent weeks but he was not the most talked about.
His election represents a heavy defeat for former president Akbar Hashemi Rafsanjani, a relative moderate who previously held the position between 2007 and 2011, and who received 24 votes.
Yazdi, described in Iran's official Who's Who as rightwing, takes up a position vacant since October 2014, when Ayatollah Mohammad Reza Mahdavi Kani died following a heart attack.
Signup to our newsletter and follow us on Facebook and Twitter!
Officially comprised of 86 religious figures elected by the people, the Assembly of Experts chooses the supreme leader and monitors his actions.
The clerical body grants the leader, currently Ayatollah Ali Khamenei, an indefinite term but it retains the power to sack him, if it sees fit.
Yazdi, initially at least, will hold the post for just one year as elections for the Assembly of Experts are scheduled alongside parliamentary polls next year, with a new vote for chairman to follow.
Although Rafsanjani put his name forward, he had appeared reluctant to resume the post, insisting his membership of Iran's top political arbitration body, the Expediency Council, already kept him busy.
"Psychologically, I am not at all prepared to become the chairman of the Assembly of Experts. I work enough at the Expediency Council and why would it be necessary to work more than this?" he said in an interview published by the reformist Shargh newspaper on Tuesday.
"We'll see on the day. My criterion is that it should be someone who befits the stature of the Assembly of Experts."