Walk around and see if you see any other bees. Watch to see if the one on the feeder are flying back to a larger group. A swarm is pretty big and they are like a ball we had one that covered the siding on the corner of the garage and then it moved to a tree and it was bigger than a basketball...
Put some sugar water out for the bees away from the chicks. There probably isn't alot of food for bees right now and something in the feed is attracting them. They could be swarming or just the food is the cause.