I personally can't tell a difference between the two when it comes to taste. My husband thinks they are richer and have more flavor than chicken eggs. 
They do look and feel different than chicken eggs. The shells are more opaque than chicken eggs, and are harder to crack. The eggs I get from my Cayuga feel like they have a waxy coating on them. When you crack them open, they smell a bit different than chicken eggs.
Duck eggs are the best for baking. They make things lighter and fluffier. Belgian waffles made with duck eggs are fabulous, as are chocolate chip cookies. 
Duck eggs have a bit more fat and cholesterol than chicken eggs.