Some Colostrum (if they didn't get it from the mama), Vitamin B Complex (if needed for energy), Scour Ease (just in case), and a bottle. We use a baby bottle, actually. They get 1, 9 ounce bottle 4 times a day about the first week and then 2, 9 ounce bottles 3 times a day until they are 8 weeks. Then, we trim them down to 2, 9 ounce bottles twice a day, then once a day. After they get colostrum, we give ours whole cow milk. It is so much better for their bellies than the powdered milk replacer. Around 6-8 weeks, they will start to eat grass, hay, and grain regularly. Before then they will nibble at it, but don't actually eat much of it. We start ours on goat pellet (with coccistat) then start mixing in sweet feed as a treat to get them started well on solid foods.
Also, at 2 weeks and 6 weeks of age you should give him a dose of CD/T vacc. You can find it at your local
TSC for about $5.
You are looking at bottle feeding several times a day for 8-12 weeks - or longer if you elect.
One more thing - if he is going to be a pet, I would suggest you look for somebody to wether (castrate) him after he's 8 weeks old. Adult bucks get really stinky and have annoying habits such as peeing on themselves to attract the ladies. A wether will not have all of those buck-y raging hormones and will be easier to handle.