Living in either sound both very weird to me, but I’d definitely would choose a cave house. Tree-houses to me seem more like a play area and not a place where you can live, it just doesn’t seem comfortable. A cave-house kind of seems fun to live in and reminds me of a rich person's house in a movie, it also seems safer to me seeing as it would most likely be a place where people wouldn’t mind and not approach, which means more peace. If I were to live in a tree-house though, I’d be anxious most of the time thinking it’ll fall down, especially if I hear a single creak.
Image from Google