Some horse feeds are OK, since horse feeds are more likely to have the correct 2:1 balance of Calcium and Phosphorus. That balance is something you'll want to really watch with the little male...too much phosphorus and not enough calcium leads to urinary calculi. Urinary calculi is usually a death sentence.
Watch about feeding him textured feed, too.. Some little buggers get in the textured feed and pick out what they want...like corn, which is reeeeeally bad for male goats. Pelleted feeds are sorta safer in that respect, since every pellet is mixed the same way. No cherry-picking.
Really, though, the primary diet should be good quality hay.. Bagged feed is supplemental feed...use it to supplement.
BTW...have you found a mineral mix to feed yet? They do much, much better with access to high-quality, highly-bioavailable mineral offered free choice in small quantities and refreshed often. Mineral deficiencies can lead to anything from lameness to chronic illness to infertility, and they're very hard to pin down.. You're way ahead of the game if you can start them right and keep it up than to have to battle a deficiency later.
And bheila's right -- that buck's fertile *right now*. If she were to have an early heat -- which does happen -- and he breeds her this young....not good at all. If you're looking to breed them one day, get them seperated ASAP. If you decide he needs to just be a companion to the doe, band him.
They're really cute.