One of the oldest temples in Bangalore, located very near to the Bull Temple this temple is a cave temple (Gavi - cave in Kannada), is well known for its remarkable architecture. It is said that lord Shiva's idol (Linga) originated naturally in the cave and was discovered by a sage in the dates back to the Tretha Yuga. The temple built during the Ganga dynasty rule of Karnataka.

The temple was renovated by Kempe Gowda I, this temple with four remarkable monolithic pillars is dedicated to Lord Shiva. The temple also has a rare, beautiful idol of Agni, the God of fire. Devotees gather here on Makara Sankaranti day when it is believed that at a particular moment in the evening the sun's rays fall directly on the linga inside the cave, passing between Nandi's horns.

There is a small but deep cave inside the temple, after a short distance the cave forks out into two branches. It is said that the one of the passages lead to Varanasi the holy city in UP via Shivaganga hills near Tumkur. The other is a cave passage through one can go around the main shrine of the Lord Shiva.