Unless your rooster is defective they all will be. With only 4 hens, I would try to find an older rooster look for someone who wants to retire one from their breeding flock. (Don't forget to quarantine!) Good luck.
There's no difference in fertile or non-fertile for eating except for an extra cell(sperm).
Some people prefer fertile but really there's no difference unless they've been stored above 75 degrees for any length of time.
I agree with emys an older rooster won't be so aggressive. Young ones will ride 4 girls to death (almost) and have them cowering in the corners.
The eggs will also be fertile a couple weeks after the roo is gone.
This thread has photos and good information on how to look for the bull's eye to tell if they are fertile. If you cannot find anything on top, the bulll's eye may be hiding on the bottom of the egg. Just turn it over gently with a spoon. If most of the eggs you open are fertile, most of the ones you don't open are also fertile.