You might not want fresh manure water running into and splashing onto foods at ground level that you're going to be eating raw, like lettuce, radishes and bunching onions, as an example. It's better to feed them with composted manure. That way, any bad bacteria or other organisms that could be in the manure will get cooked in the compost pile and killed, before you use it.
So, maybe just think about how you have things placed, how far away the garden is and how much water will really be running that way. You may or may not have a problem.