This is what I have found for our goats, which were almost all Nigerian Dwarves until last year. We added a fainting goat, a pygmy and a mini nubian. The cattle panels are good, but little ones go right through them, which also means small dogs and other animals can also go through them. Hog panels are good for little ones, but they can jump them when they get older and dogs can also jump them. Combo panels are a good compromise. We use the 5' welded wire fencing with the 2" X 4" squares for a pasture and the combo panels inside the barn to make pens, etc. This keeps our guys contained fairly well.
HOWEVER, there are exceptions to every rule. We once had a full sized Nubian wether, he would walk the fence down to get the weeds on the other side, eventually climbing over. Not unusual for full sized goats. We also had a Nigerian wether that we got at 2 years old and he was an escape artist. He jumped the five foot fence and gates whenever he felt like it. It wasn't too bad until he started visiting the neighbors or following us out into the road when we tried to leave. We finally found him a new home that had higher fencing than us! We have one other wether that started following the trouble maker, but now that he is gone, he usually stays in pretty well. Remember the fencing must keep the goats in and also the predators out. Dogs are probably the most common threat and they will often kill the whole herd if given the time. This happen to friends of ours, two shepherds jumped their 5 foot fence and killed all 14 of their goats.
Flooring, definitely dirt or cement. If you use cement, you will want to put down shavings or straw to cover it. Wood will rot, stink and hold bacteria. Our original barn is 12 x 14 with the gambel roof and cement floor. When we added on when did not put a floor in. Goats are famous for picking through their hay for the good stuff and by the end of winter are laying on top of 2 feet of wasted hay. Ok, so they are fat and spoiled, what can I say!