One of the oldest temples in Bangalore, located very near to the Bull Temple this temple is a cave temple (Gavi - cave in Kannada). It is said that lord Shiva's idol (Linga) originated naturally in the cave and was discovered by a sage in the early 15th Century. The temple built during the Ganga dynasty rule of Karnataka.

The temple was renovated by Kempe Gowda I, this temple with four remarkable monolithic pillars is dedicated to Lord Shiva. The temple also has a rare, beautiful idol of Agni, the God of fire. The annual festival is on Makara Sankaranti day when thousands of devotees flock to see the sunrays fall on the Sivalinga between 17.00 hrs and 18.00 hrs.

There is a small but deep cave inside the temple, after a short distance the cave forks out into two branches. It is said that the one of the passages lead to Varanasi the holy city in UP via Shivaganga hills near Tumkur. The other is a cave passage through one can go around the main shrine of the Lord Shiva.