Well, I'm not a veteran turkey breeder, but I was in your shoes about five months ago. I started out with three Standard Bronze turkey poults because I knew that I wouldn't like the Broad Breasted variety due to their sluggishness and general ill-health. At that point, I already had loads of chickens and several ducks. After doing loads of research, I was terrified about Blackhead Disease and the chickens picking on the turkeys. And then I worried about other things, like them dying after I'd just paid $10 a poult and $20 for a bag of high protein turkey feed. It's a huge investment when you think about it.
There is loads of information here on BYC on good starter breeds. I, personally, love my standard bronze turkeys but I'm sure there are other opinions on "better" breeds. I've found that they are much more interesting and personable than my chickens. I've never had one of them act aggressive towards me, but I've always showed them who is boss first (with the toms, I recommend showing them early on that you are top dog by picking them up and hugging them a bit, just to prove that you are bigger and stronger than them). Jennies are nearly always docile and sweet, in my opinion.
They will do well so long as you put in the time and effort. One concern I would recommend having is being aware of the various diseases they are more susceptible to like Blackhead Disease. Chickens may carry it and never be affected, but turkeys have been known to die from it. Finding out if it is prevalent in your area is very important. I didn't, but I got lucky and we didn't have any problems. But because I didn't know anyone who would know, I made sure to feed them apple cider vinegar in their water and cayenne pepper to worm them. I think it helped.
I would be cautious when thinking about introducing new birds to your care, but I wouldn't be fearful. You have a wealth of knowledge here on BYC and I'm sure there are many breeders willing to answer any of your questions. Good luck!