I've used it by injection in horses *many* times, and it did not cause any issues with them.@Anabariful be careful of injection site soreness and watch for necrosis... I don't *think* penicillin causes injection site necrosis, but I would talk to Kathy @casportpony about it first... it can be drawn out with a needle and then given orally as well...
I once asked a vet about giving it orally and she said something like "no, injectable penicillin should not be given orally because it's poorly absorbed".