The Turkish presidency said in a statement that the plane was a Russian Su-24 fighter jet, while Turkish media said one pilot had been captured by rebel forces in Syria.
Moscow confirmed that one of its planes had been shot down, but said the pilots' fate remained unclear.
The Turkish army said that the plane had violated Turkish airspace 10 times within a five minute period and was shot down by two Turkish F-16s. However Russia insisted that the plane was inside Syrian airspace.
"A Russian Su-24 plane was downed under the rules of engagement because it violated the Turkish airspace despite the warnings," the Turkish presidency said.
Reports said two pilots had ejected from the plane and Turkish television pictures showed two white parachutes descending to the ground.
The CNN-Turk channel said Syrian Turkmen forces fighting the Russian-backed regime of President Bashar al-Assad captured one pilot.
Russia meanwhile confirmed that one of its planes had been shot down.
"Presumably as a result of firing, an Su-24 plane of the Russian forces crashed in the Syrian Arab Republic," Russian news agencies quoted the defence ministry as saying.
The Russian ministry said the fate of the pilots was not yet clear.
Turkey's Dogan news agency broadcast footage of what it said was Russian helicopters flying over Syrian territory in an apparent search for the lost pilots.
TURKEY TO SEIZE NATO, UN
Signup to our newsletter and follow us on Facebook and Twitter!
The fighter jet exploded in the air and the fireball fell on a mountain on the Syrian side of the border, television pictures showed.
Footage posted by the state-run Anatolia news agency showed plumes of smoke rising behind a mountain a few kilometres from the Turkish border.
Turkish reports said the incident happened in the border area between Turkey's southern Hatay province and an area in northwest Syria populated by the Turkic-speaking Turkmen minority.
The incident came as Russian and Syrian jets are carrying out a heavy bombing campaign against targets in northern Syria.
The Turkish government has expressed anger at the bombing campaign, saying it is aimed at buttressing the Syria regime and has displaced thousands of Turkmen Syrians.
Russia however insists that the air campaign is aimed against Islamic State (IS) jihadists.
Amid fears of a diplomatic crisis, key NATO member Turkey said it would take the issue to the United Nations and NATO.
"Necessary initiatives will be taken at NATO, UN and at the level of countries concerned by the foreign ministry upon instructions from Mr Prime Minister," a statement from Prime Minister Ahmet Davutoglu's office said.
Russian fighter jets entered Turkish airspace in two separate incidents in October, prompting Ankara to summon the Russian ambassador twice to protest both violations.
Turkey and Russia have long been at loggerheads over the Syrian conflict, with Ankara seeking Assad's overthrow while Moscow does everything to keep him in power.
The Turkish military in October also shot down a Russian-made drone that had entered its airspace. But Moscow denied the drone belonged to its forces.
Russian Foreign Minister Sergei Lavrov is due to visit Turkey on Wednesday in a bid to smooth ties and find a joint approach to finding peace in Syria.
Along with Saudi Arabia and the United States, Turkey and Russia are taking part in talks in Vienna that aim to narrow differences on the Syria conflict and have taken on an extra importance after the Paris attacks.
A Turkish foreign ministry official told AFP Lavrov's visit would go ahead as planned. "There is no change in the programme," said the official.