The rocket hit a courtyard where the children were playing, said Abdel Ghani Shaalan, Marib's deputy regional director of security.
Two other rockets hit a house and a shop front, wounding 25 civilians including women and children, he told AFP.
The toll was confirmed by Saleh al-Shaddadi, the director of Marib's main hospital, where casualties were admitted.
The Huthis launched the Katyusha rockets from Mount Haylan, 15 kilometres (10 miles) west of Marib, Shaalan said.
Marib city and the majority of the surrounding province are held by government forces who are fighting the Shiite rebels who control areas to the north and west of the oil-rich region.
Signup to our newsletter and follow us on Facebook and Twitter!
The plight of children in Yemen prompted a dispute between Saudi Arabia and the United Nations in June when the UN accused a Saudi-led military coalition in Yemen of being responsible for the deaths of hundreds of them.
On the basis of that report, the UN put the coalition on a blacklist of child rights violators.
Riyadh pressured UN Secretary General Ban Ki-moon to remove the coalition from the list by threatening to withdraw Saudi funding for UN aid programmes.
The attack in Marib came as Saudi-backed government and Huthi negotiators take a break from peace talks after two months of UN-backed negotiations that have made little progress.
The talks are due to resume on July 15 in Kuwait.
The conflict in Yemen has killed more than 6,400 people dead and wounded 30,000 since the Saudi-led coalition intervened in March 2015.
Saudi Arabia aims to restore exiled President Abedrabbo Mansour Hadi to power and assert government control over large parts of the country that had been taken over by the Huthis in 2014.