The tallest underground waterfall can be found inside a cave in Tennessee. My husband and I love to visit the mountains and caves of Tennessee. With nearly 10,000 known caves in the state, Tennessee is the cave capital of the United States. Recently my husband and I went to Chattanooga for the weekend and we had a blast. Rock City was absolutely incredible (even though Rock City is technically in Georgia), and then we went to a cave that led to a stunning waterfall.