During the era of Pallibana Perumal, it was decided to transform the Neelamperoor Shiva Temple into a Buddhist monastery. Ten Brahmin families, later known as Pattiyillam Pottimar, moved to Vazhappally, taking the Shiva Linga from Neelamperoor. They first placed the Shiva Linga in the northern part of Vazhappally gramam, known as Devalokam. When attempts to shake the Shiva Linga failed, the grieving Brahmin family was visited by Parasurama, who offered them the Shiva Linga he had worshipped. He advised them to build a temple on the Ardhanariswara concept, which they did, constructing a large shrine for the Shiva Linga and more an idol of Parvati. The sanctum sanctorum was built within three walls of a rounded stone granite shrine. The Shiva Linga was placed in the east, and the Parvati idol in the west, with Dakshinamoorthy and Ganapathi situated inside. A large Tidapalli was built in the southeastern corner of the Nambalam, and Raja constructed a sub-shrine for Sastha at Kannimoola outside the Nambalam temple. The Taranalloor series and the Tulu Brahmin family of Kasaragod were appointed for tantric rites and daily pooja respectively, with the main priest residing in the Kuthasanthi Math. The temple gained prominence during the second Chera dynasty. Despite the fall of the Cheras and subsequent rule by Thekkumkur and Travancore kings, the temple survived numerous wars and upheavals, safeguarded by divine will and the prayers of devotees. According to legend, after Pallibana Perumal was dethroned by Bhattas, he travelled to Neelamperoor. Hearing that he intended to install his idol there, the Potti Leaders of the Pathu Illams quickly moved the Neelamperoor Shiva idol to Vazhappally, ensuring its safety and continued worship.