Kalyani and Israel's Rafael Advanced Defence Systems will make advanced missile systems and other technologies at a facility near the the southern Indian city of Hyderabad.
Kalyani managing director B.N. Kalyani said his company would hold 51 percent of the venture, in one of the first such tieups since the cap was lifted on foreign investment in the sector last year.
"The joint venture will ... manufacture (remote) weapons systems and advanced armour solutions for Indian and subsequently global customers of Rafael," Kalyani told reporters in Bangalore.
Kalyani did not disclose the size of the venture, but said it would be "substantial".
The deal was announced at an aviation industry conference in the southern city of Bangalore where the bosses of hundreds of defence firms have gathered seeking lucrative contracts with India.
Signup to our newsletter and follow us on Facebook and Twitter!
India, which has long been the world's largest buyer of defence equipment, is in the midst of a multi-billion-dollar upgrade of its ageing military hardware.
At the opening of the air show on Wednesday, Modi vowed to end India's status as the world's number one defence importer, saying he wanted 70 percent of hardware to be manufactured domestically by the turn of the decade.
Israeli Defence Minister Moshe Yaalon met with Modi in New Delhi on Thursday after attending the show's opening, where he said Israel was willing to work in joint production and development with India.
"This visit gives us an opportunity to enhance and improve our relationship," Yaalon said at a public lecture after the meeting.
"We cooperate openly in all the areas but we have found a way to cooperate with each other behind the scenes in security," he said without elaborating.
The US has the largest contingent at this year's five-day show, held at an air base on Bangalore's northern outskirts, with 64 companies including Boeing, followed by France, Britain, Russia and Israel.