Go for it! But, make sure it is the right property for you. Right now it might seem like a lot of land, but in a few years you might want more, or might want more privacy than 5 acres can offer.
You should have quite a bit of equity built up in your home. Have it appraised and see what you have to work with.
The sense of peace that comes from not having close neighbors and all of the drama is well worth the effort. When I first moved out here I couldn't believe how much stress just disappeared. I got the biggest kick out of walking to my mailbox or back to the barn to check on horses in my PJ's....lol.
My husband was happy where we were in the suburbs. We lived there for 18 years, the plan was to buy the house, improve it sell it and then buy our farm. But, life got in the way of our dreams...kids...jobs. The house had all of the improvements in and was easy to upkeep. So the hubby didn't care if we moved into the country or not.
After a very obnoxious family moved in and upset the entire neighborhood I had my fill of suburban living and started looking at farms. Hubby dragged his feet, but I told him that for my own peace of mind, I needed some space around me. I grew up on a farm and knew how peaceful the lifestyle was.
10 years later, and hubby still says that it was the best move we ever made. He will never go back to the city or suburbs. We met some really great friends out here and the few neighbors we do have on the road are like us, in that we help each other out when needed, but we don't hang out together like people do in the suburbs. None of us have the time for that...and like it this way!
You might have to take the lead, like I did. I found this farm and really pushed the move, but now that we are here we are both happy!