Modi also paid floral tributes to Kalam at his 'samadhi' inside the memorial, built by the Defence Research and Development Organisation (DRDO), where the "People's President" worked for decades, and other central government departments.
Evincing keen interest in the features of the memorial, which has on display replicas of rockets and missiles on which the late scientist had worked, the Prime Minister examined it for about 15 minutes.
It has been built at a cost of Rs 15 crore on land alloted by the Tamil Nadu government at Kalam's village Peikarambu.
On the occasion, the Prime Minister also unveiled a wooden statue of Kalam playing the 'veena'. The former president was adept at playing the musical instrument.
The memorial also has about 900 paintings and 200 rare photographs of the late scientist, who held the office of the president from 2002 to 2007.
A bronze statue of Kalam has also been erected at the memorial, the entrance of which has been designed in the style of the India Gate in New Delhi.
The rear portion of the memorial has been modelled after the Rashtrapati Bhavan.
Modi was accompanied by Tamil Nadu Governor Ch Vidyasagar Rao, Chief Minister K Palaniswami, Union ministers Pon Radhakrishnan, Nirmala Sitharaman and NDA's vice presidential nominee M Venakaiah Naidu.
The prime minister earlier unfurled the Indian national flag at the entrance of the memorial. Later, he interacted with family members of Kalam.
Modi was seen affectionately holding the hands of Kalam's elder brother A P J Mohamed Muthumeeran Maraikayar.
With 'unity in diversity' as its theme, the memorial features quotes of Kalam and pictures of him as a scientist and the president of India.
A large number of children were present at the venue carrying placards with slogans on the achievements of Kalam.
Earlier, Tamil Nadu Governor Ch Vidyasagar Rao and TN Chief Minister K Palaniswami, received Modi at the Madurai airport from where he arrived here by an helicopter.
A three-tier security ring was put in place in and around Rameswaram for the Prime Minister's visit, during which he would also dedicate to the nation a 9.5 km-long Link Road to NH 87, which runs between Mukundarayar Chathiram and Arichalmunai.
A new express train from Ayodhya to Rameswaram would also be flagged off by the Prime Minister through video conferencing from nearby Mandapam.