Maybe you can get a pic of your "tennants" this spring / summer, Gramma. And we can try to clinch the I.D. for you!
Lots of "swooping" might be Tree Swallows (they are a stunning blue and white). They will readily use bluebird boxes. We have a pair that fledged their young 2 years ago and then returned last summer. Hopefully, they'll be back. If you've got Bluebirds setting up in your boxes, lucky you. Mine never seem to stick around, though they are here now, singing their pretty song and eating bugs off the trees.
Some will stay around through the winter, just like robins.
Cass, this is a shame -- I'm sorry!!
TOB