As far as grazing goes: just how big is that lot next door to you? If you say, "40 acres" then yeah, you could just graze the horse in there for a lot of the year, and be fine. However... a horse can dry-lot a piece of ground pretty quick. So if your piece of ground is more like, 1 acre, well, the horse will go through that, eat all the best parts, and not touch the weeds. Pretty soon, all you will have is weeds in there. Any horse pasture benefits from being mowed, because the weeds have a hard time competing with the grass and mowing favors grass, but if it's not big enough, you are going to run out of grass. There is no rule of thumb here, I have no idea what part of the country you are in or what the ground there is like, how much rainfall etc. Here, I have to irrigate.
Now, I DO have 40 acres. But I still have to pull the horses off every now and then and give the grass a chance to grow back (i.e. rotational grazing). We mow what we can (some parts are too rough) but guess what, I still have to fill my barn with hay every year. I feed hay every single day of the year to at least one group of horses or another (most of mine are minis). Don't rely too heavily on that grass. As for those cheap big bales? Again, I have no idea where you live or what is in them, but around here, those would be hay for cattle, and cattle can get by with stuff a horse just can not eat. Plus let me tell you, they are about impossible to use for feeding a small amount twice a day with. I've done it, I'm not just guessing. Do you have a tractor to pick them up and move them? Do you have a place to store big bales like that?
I know it sounds ridiculous but you do have to think about stuff like that. Small bales might cost more, but holy cow, they are SO much easier to move and store. I'm not guessing about that either; we are currently in the process of putting up 400 bales of hay in the barn. And... you DO have a barn for storing hay, right?