If the horse is well trained, you can use hot wire. Just don't have it tall enough that your leg will touch it.
For training a horse, you really can not cobble together anything cheap. If your horse crashes into the fence, the fence must be able to hold the weight without any give. It must be tall enough that the horse will not consider jumping it. Anything else and you risk serious injury.
You might be able to use heavy posts and livestock panels. Although, I doubt that would be any cheaper than $500 by the time you got it big enough, high enough, and sturdy enough.
If you've got a trailer, maybe you could rent space in a round pen for an hour or so, a couple of times a week. For starting a colt, if you don't have safe equipment, it makes sense to board the horse in a stable that has excellent training equipment for at least a month. Then train like blazes for that first month, and maybe you can get away with continuing at home with less sturdy equipment.