Hi.
Can you post pics of your flock? Maybe some videos of the ducks quacking?
The problem is is that the drakes won't have anything to mate and they will go after the chickens once they start submitting. The other posters who don't have problems most likely have enough duck hens for the drakes not to bother with the chickens.
In order to keep 2 drakes all as one happy flock you need at least 3 hens each. That is a minimum. So, that would be six extra ducks. Or the option is to separate and have a drake flock. But, even that can be difficult as they will fight, but most likely not as bad since there are no ducks to fight over.
Maybe post a pic of your set up and we can come up with ideas to separate within the limited space. Duck houses don't need to be as complicated as a chicken coop nor as big.