Try walking through him when he stands up to face you that way.
Not answering his challenge, but just being a force of nature moving on through so that he has to get out of your way -- the same as if you were a horse or a cow in the same pasture.
Pick up a bucket or something, walk straight through where he's standing without diverting, and put it down. Don't challenge him or be aggressive, but do brush him aside if he doesn't move.
No guarantees, but one of the experienced people here told me to do that with my boys some time ago and I've had great success with it so far.