The Saudi-led alliance does not include the kingdom's Shiite regional rival Iran, or Syria and Iraq.
It will be based in Riyadh "to coordinate and support military operations to fight terrorism", with participation from Middle Eastern, African and Asian states, the SPA state news agency said.
The coalition will tackle "the Islamic world's problem with terrorism and will be a partner in the worldwide fight against this scourge" said Saudi Defence Minister and Deputy Crown Prince Mohammed bin Salman at a press conference in Riyadh.
Arrangements would be made for "coordination with friendly peace-loving nations and international bodies for the sake of supporting international efforts to combat terrorism and to save international peace and security", SPA added.
The 34 members all belong to the Jeddah-based Organisation of Islamic Cooperation.
More than 10 other "Islamic countries" had expressed support for the coalition, including Indonesia, it said.
"These countries have procedures to go through before joining the coalition, but out of keenness to achieve this coalition as soon as possible, (this alliance of) 34 countries has been announced," said the Saudi minister.
The United States has called for broader international participation in the fight against the Islamic State group of jihadists in Iraq and Syria, saying Turkey needed to do more to control its borders with Syria, and Saudi Arabia and the Gulf states had been distracted by the conflict in Yemen.
The coalition will fight "any terrorist organisation that appears," said Mohammed when asked if the alliance would concentrate on fighting IS only.
The announcement is the latest in a more assertive Saudi foreign policy since King Salman ascended the throne in January and named Mohammed, who is his son, as defence minister.
In March the kingdom formed an Arab coalition of about a dozen countries to support the government of Yemen against Iran-backed Huthi rebels and their allies, who seized much of the country.
Saudi Arabia and the United Arab Emirates have, however, assumed the main roles in that coalition.
Signup to our newsletter and follow us on Facebook and Twitter!
- Prominent Saudi role -
Rights groups have repeatedly criticised both the coalition and the rebels over civilian casualties in Yemen.
Saudi Arabia has also played a prominent role in efforts to find a political solution to the war in Syria.
Last week, the kingdom hosted unprecedented talks between opposition Syrian political and military factions, which agreed to negotiate with President Bashar al-Assad's regime but insisted he step down at the start of any political transition.
The meeting follows an agreement last month in Vienna between diplomats from 17 countries, including Saudi Arabia, for a political transition and end to the Syrian war.
The Vienna plan set a January 1 target for peace talks and would see a transitional government set up in six months and elections in 18 months.
Saudi Arabia supports some rebels in Syria, and last year joined a United States-led coalition bombing IS in Syria.
Other Gulf states are also in that coalition but the US Air Forces Central Command chief, Lieutenant General Charles Q. Brown, said last month that Gulf strikes in Syria have diminished since they began action in Yemen.
The kingdom has said military operations in Yemen were nearing an end, although fighting continued ahead of a ceasefire to begin at 0900 GMT Tuesday in conjunction with peace talks in Switzerland.
The United Arab Emirates said two weeks ago that it is ready to commit ground troops against jihadists in Syria and described Russian air strikes in the country as attacks on a "common enemy".
Quoted by the official WAM news agency, Emirati State Minister for Foreign Affairs Anwar Gargash said the UAE would "participate in any international effort demanding a ground intervention to fight terrorism".
"Regional countries must bear part of the burden" of such an intervention, he said during a discussion on Syria.