Most of them are destroyed over a period of time.. Enemies do not permit them to become a place of worship to avoid maintaining hatred & vengence.. hence they remove the evidence that it was pallippadai... the word is removed in Panchavanmadevi's pallippadai!
No Hindu King has ever destroyed any temple - including Hindu temple! They would have removed evidences only.
There are inscriptions found on the walls inside the temple tank of Thiruvalanchuzhi - where Rajendra performed the First Srardham for his father Rajaraja. In the Kshethrabala (Bairavar - which is now kept in the basement of Rajraja Manimandapam (near New Bustand - Thnjavur) temple we can find so many inscriptions including a grant by Madevadigal - daughter of Rajaraja. The Swetha Vinayaka temple architecture is with Chalukya influence. May be of Vimalithan (son-in-law of Rajaraja). Swetha Vinayaka itself is a unique form of identification. The peetam is buried inside and cemented! We have to escavate to read the complete inscriptions. Georgrphically it is only two KMs north of Pazhayarai palace.. At the entrace of the temple, above top level - Sundaramurthy - Seramanperumal (elephant - horse) reaching Kailasam (with mortal bodies) are depicted.
Considering all the important factors, efforts are on to undertake this work by some of the scholars for detailed research in co-ordination with State Archeology Deptt.