Your next pocket travel guide
score:9
There are no scholarly literature (like publications in reputed academic peer-reviewed papers etc) supporting the theory that it was a temple earlier. Till such materials are available, it would be wise to consider such stories as myths.