Speaking to reporters in rebel-held Sanaa as he concluded a two-day visit, O'Brien said he had visited a hospital in the Red Sea city of Hodeida where he met "very small children affected by malnutrition".
"It is of course absolutely devastating when you see such terrible malnutrition," he said, hailing the efforts of relief agencies trying to meet the "very severe needs" for food in the country.
"We need to do more. We need to do everything we can to meet the very large scale of needs which are here in Yemen," he said, in his first visit since August last year.
The UN children's agency UNICEF says nearly three million people in Yemen are in need of immediate food supplies, while 1.5 million children suffer malnutrition, including 370,000 enduring very severe malnutrition that weakens their immune system.
Signup to our newsletter and follow us on Facebook and Twitter!
The conflict between Yemen's government and Iran-backed Huthi rebels escalated last year with the intervention of a Saudi-led Arab coalition in support of the president.
The coalition that supports President Abedrabbo Mansour Hadi has been trying to push them back.
It has imposed a sea blockade on Yemen and conducts daily air strikes on areas under the rebels' control.
O'Brien said the "very best humanitarian solution is that we see a political solution to this crisis".
But in the absence of such a solution "it is our job to work with everybody to meet the humanitarian needs and to have the chance and the space to do that wherever those needs arise and to meet those needs impartially," he said.