I'm going to admit it.
I don't even know what 'diseases' people can catch from chickens. And I currently have 25 of 'em. I, and my kids, go into the coop everyday. My daughter is on a mission to make every bird her best friend. She goes out there several times a day and lets the birds climb all over her. I've yet to find myself concerned about her health due to this. I have her wash her hands every time she has handled the birds and that's it.
I don't scrub her down with disinfectant.
We keep a clean coop with DE and SweetPDZ mixed with the bedding, to help it stay dry, thus preventing much of the disease/bacteria/mites/whatever from being able to survive in it.
That said, maybe gathering info on what the possible health risks to humans are and ways to prevent transfer of illness to humans from chickens would be good info to arm yourself with prior to the 'chat'. I think that it may ease your neighbors mind to know how utterly easy it is to keep a flock healthy. Also, pointing out that much of what is in the media regards commercial chicken farms, not the typical backyard flock, could be helpful. I think the diseases and crap we hear about in the media are attributable to overcrowding and poor management practices, commercially. Aside from that- another hugely beneficial thing about chickens is the poo for the compost bin!