Sometimes, it's a case of the horse not thinking about you at all (I mean, really, why would they? They don't think about the size of a cart when they pull one; it's why they often wind up in such terrible wrecks when horses in harness run amok). Sometimes, though, they are thinking about you, in a very calculated way - I have known horses that would deliberately go under low branches to try to get rid of a rider.
But yeah, something between you and the saddle is a must. Fabric that absorbs sweat and breathes is about the best you can do - and trying to schedule your rides for the cooler parts of the day. After all, that saddle doesn't breathe; the horse gets hot, too (our current temperature is in the upper 80's, and the humidity is up there. Latte is dark with sweat, and all she's doing is standing in the shade, swishing flies).