Speaking at counter-terrorism forum in the Saudi holy city of Mecca, Al-Azhar grand imam Ahmed al-Tayib linked extremism to "bad interpretations of the Koran and the sunna", the teachings of the Prophet Mohammed.
"There has been a historical accumulation of excessive trends" that have led some people to embrace a misguided form of Islam, he told the gathering.
"The only hope for the Muslim nation to recover unity is to tackle in our schools and universities this tendency to accuse Muslims of being unbelievers," he said.
Tayib's comments come days after he expressed outrage at the Islamic State group for burning to death a captured Jordanian pilot who took part in US-led air strikes against the jihadists in Syria.
On February 4, after IS released a video showing Maaz al-Kassasbeh dying in a cage engulfed in flames, Tayib said the jihadists deserved to be killed or crucified.
On Sunday in Mecca, home to Islam's holiest sites, he made no mention of IS but denounced "terrorist groups... who have opted for savage and barbaric practices".
Signup to our newsletter and follow us on Facebook and Twitter!
He blamed unrest in the region on a conspiracy by what he called "new global colonialism allied to world Zionism".
Tayib said that this plot has exploited "confessional tension" in conflict-hit Iraq, Syria, Yemen and Libya.
The opening day of the conference also heard a speech from Saudi King Salman who called for "an efficient strategy to combat terrorism".
"Terrorism is a scourge which is the product of extremist ideology," the monarch's speech, read by the governor of Mecca, said.
"It is a threat to our Muslim nation and to the entire world."
The three-day conference, organised by the Muslim World League group of non-government organisations, is being attended by senior clerics from across the Muslim world to discuss how Islam can combat extremism.