Keep in mind that fresh food will always taste much stronger and richer than anything you can buy in a store. And sometimes that takes a little getting used to, lol! Duck eggs are firmer than chicken eggs, and slightly richer, but other than that, I can't tell much difference between them and farm fresh chicken eggs. However, I do prefer them and won't ever go back to chicken again.
As for baking, my friend who is a personal chef and buys a lot of my eggs (about a dozen a week on average) LOVES them for her baking. The proteins are different and give a higher loft to the baked goods. She couldn't believe how her angel food cake came out the first time she used duck eggs.
For me, it's gotten to the point that I simply can't order eggs when I'm out at a restaurant because they are so icky--pale, watery, tasteless things that make me want to gag. I guess if I were growing all my own food, I probably wouldn't be able to stand to eat anything else, lol.
Anyway--I hope the flavor grows on you. If not, they can always be used in recipes, and your family will be healthier for it!