I don't think it's unusual, at the core ducks are prey animals so that instinct to be wary of things wanting to pet/hold them is strong. Just do what you always do, hopefully they will come around.
I still have a few goofballs but they were bought as older birds and not handled, we've had them...