I agree @BrentG . It is difficult to identify with a couple photos, but the closeup of the leaf certainly looks like buckthorn. We called them "poison berry bushes" when we were kids.It's difficult to tell without seeing the berry mature, but if I were to wager a guess -- or at least err on the cautious side -- I'd say it could be buckthorn, which would be toxic if eaten. I would not allow the little ones to consume. Again, it's hard to id any plant with just a couple pictures, but if it's purple or black and not red, then buckthorn is a great possibility.