Hi there. Yes, that are fleas for sure. It's hard to say what species that is, because you need to see it under microscope for precise identification. The most common fleas species is Ceratophyllus gallinae but it could be a different one depends on geography. The wormy objects are larvae and the dark are adult stages. Adults are feeding on birds and mammals. They are sucking their blood as that is the only meal for them.
Dark spots on eggs is blood that was regurgitated or excreted by adults. Blood in this form is also nourishment for larvae.
In bird boxes with nestlings fleas have very suitable conditions for development - from eggs through larvae (3 stages) up to adults. Larval part of the life cycle could last several weeks up to month. Adults can live up to one year if appropriate environment conditions (huminidy and temperature). Larvae are most fragile stages.
The best would be to clean all bird boxes, don't leave any old debrit there. After cleaning might also help to use an insecticide but I can't advice any brand as I don't know what you have in USA.
pyrethrin spray for bedding, nest boxes and/or birds and if you want, a nice dusting with sevin. No, DE is more of a preventative. Go for the big guns with these little buggers.