"We were a bit hesitant at first, but apparently this was the normal standard for bringing babies into the cave," the mom told Newsweek.