This has been said a hundred times already, but it really is true. Pick a horse not a breed, and pick a horse YOU feel comfortable with. Who cares what colour it is, a pretty horse is just an expense if you don't enjoy it.
I learnt this the hard way. We live on a ranch with 90+ horses, but there was only one that was suitable for my confidence level and he's in my FIL's string, so often being used. They mostly vary between halter broke, green broke, barn sour, broncs, and just too quick/athletic. All are foundation QH's. They're good at their jobs, but don't want anything to do with pleasure riding. Haul them in a trailer and work cattle all day and they're fine, but trying to go trail riding away from the house is pretty much impossible
That's just not in their mindset, they want to work or be left alone, they don't want to be pets.
So we went horse shopping. Horse #1 was a 7yr old Quarter Horse. We arrive after driving 200 miles to see him, and are told he is petrified of men and they think he was beaten. That might have been useful to know before we made the drive. The seller was riding him when we arrived, and he was already sweaty! I won't know to this day why I got on him. He had no brakes or steering, and would not back up. I rode him back to the seller and she offered to show me how she had taught him. Suddenly something spooked him, and her father made a grab for the reins. (Does that bother anyone else btw? Nothing freaks me out more than someone messing with the reins when i'm on the horse!). Being petrified of men he ran backwards and ran off. The fence at the other end of the pasture stopped him. I got off right there, and was done. If he acted like that at home, who knows what he'd have done when we got him back here. It took a good 2 hours for me to stop shaking!
Horse #2 was a 7 yr old Thoroughbred mare. She was a dork, but not mean. She had been a racehorse, but came last in the two races she had entered. She was described as 16hh. She was actually 17.1hh. I am 5'1. She came running behind the truck when we arrived, and other than the fidgety TB behaviour she was a nice horse. Being afraid of horses, the fidgety stuff got to me, but they convinced us to take her home for a 2 week trial. When we collected her, they had us sign a contract stating that if anything happened to her we would buy her. We weren't 5 minutes from their house when she broke the halter and we felt her pacing the trailer. (They forgot to tell us not to tie her in a trailer. It's amazing how forgetful horse sellers can be) That was the longest 2 hour drive of my life, knowing if she was injured we would have to buy her regardless of what she was like. She spent a two week vacation in our pens, as they had also forgotten to mention that she hadn't been worked in three years, and if she came up lame... you guessed it! We'd have to buy her.
Horse #3 was a 4yr old Quarter Horse, described as 'suitable for anyone'. I now know I should have kept looking, when I saw 'mare' and '4yr old' in the same sentence. I had my husband ride her first because by this point I wasn't going to trust anyone's horse. He liked her, but couldn't do a whole lot as the pen was very muddy and it was an extremely windy day. We led her to our trailer to swap out saddles, and she stepped on a rein and reared. That pretty much put me off. My husband didn't think it was a big deal, but my kind of horse would probably not notice or fall asleep. I got on her, and didn't like her. She kept trying to buffalo me, and my confidence isn't at a level to deal with that. I told my husband, but somehow he talked me into her because he thought she would be more challenging and teach me more than a push button horse. So we paid for her, it took four of us to load her in the trailer. Halfway home we had a flat. It was an omen! The first day my husband rode her, she colicked. She got over that, and a few days later I decided to ride her out from the house. She would not move unless I turned her in a circle back home then kept her going for a step or two. After 15 minutes we were about 100' from the house. She saw some calves and perked up, so I decided to trot after them. Soon after she started bucking. I rode it, but I wasn't going to push my luck with a strange horse and nobody else around, so we went back home. Probably not the best plan, but my self preservation instinct took over. I decided i'd stick to lunging her until my husband had ridden her a few more times. Problem is, she'll run at you bucking, or just refuse to move away and rear. She sidesteps when being saddled, dumping your saddle on the ground unless you push her up against a fence. She's hard to catch. She beats the crap out of the other horses at feeding time, even if they're the other side of the fence. Some days she won't turn left, and runs backwards instead flinging her head. Some days she can be perfect, but you never know what you're going to get. She's good looking and well-bred, but that's about all she has going for her.
A couple of months after we bought her, my FIL brought an old horse home for me. He's ugly, old, and wind-broke. The whites of his pig eyes always show, and his tail is broken and crooked. His coat is wooly and curly, and he has feet like saucepan lids. He's also trustworthy, honest, and kind. He knows when I am afraid, he takes the lead and doesn't take advantage. He'll work his great big butt off chasing cattle, but if I feel afraid I can pull him up and he will not move an inch. Finding a horse like him is like striking gold, and I don't care what he looks like. If only I had kept looking for something similar, instead of the mare. The mare is an expensive pasture ornament that my husband doesn't want me riding until he's put in a few months with her.
Beauty is as beauty does with any horse. Temperament is far more important than looks, kind of like driving a fancy looking car that always breaks down vs something rusty and reliable. The mare and gelding are both QH's, but couldn't be more different.
ETA Something else I found out from this whole experience is that it is important to ask what exactly a horse has been used for and what it is used to. Not all trail horses, roping horses etc are created equal. A ranch horse might make a horrible arena roping horse, we have several that get mad being ridden in an arena but are fine in the pasture. We sold one that was a horrible ranch horse, but he loves team roping. A trail horse that's used to being ridden daily may be a handful if only ridden on weekends. Some aren't suited to being ridden once a week. Some have to be turned out or they are rotten, and these are often the same ones that need riding frequently!