I would be on the alert to make sure the "town" isn't influenced to change your zoning. Probably when those people built houses in your area, one by one, they asked to have their property zoned residential, until yours is smack dab in the middle, but still agriculture. Just keep on top of it in case.
Hopefully it won't be a problem for much longer. My MIL has a friend who is a realtor and may have a lead on some empty acreage for us!!