This was something we wanted to do, and have been slowly making progress. Now we make, grow, hunt, or produce nearly 75% of what we consume. My wife and I bought our farm 4 years ago in the fall and started right away, just don't get overwhelmed. I had more experience as I grew up on a farm. Where as she is from Chicago.
I told her we would allocate some money right off to establishing an orchard, as it takes the longest to produce. That first fall, we planted 2dz fruit trees as well as 100 yds of berry plants. We also started on establishing a garden spot. Over that first winter, I built our chicken coop & run, and rabbit cages. We started small with 24 chickens, and have grown our flock of layers up to 50 now. We do raise 150-200 meat birds annually. Rabbit and chicken manure is sold, and remaining is added to our garden annually.
Grow what you will eat! We consume a lot of cabbage, cucumbers, tomatoes, peppers, and my favorite...meat! That is what we focus on. We don't buy beef at the store! We hunt deer, raise our meat birds/chickens, and fish. My best friend raises cattle, and we trade for beef often. The only meat we currently buy is pork, and I'm working on getting set up for a hog.
Also, pound for pound, fresh organic herbs are more expensive that meat. We use a ton of rosemary, dill, oregano, basil, thyme, marjoram, sage, etc (I make a lot of homemade sausage using these). We have a garden bed just for herbs. We can grow and preserve a years supply in a small space, saving literally hundreds of dollars annually.
One final point, this is often a touchy subject. Money! It can be as expensive or cheap as you want it to be. I work full time, have the farm, and my wife works full time and runs our side business. Time is money and I work 100hrs a week between my job, and side business (not counting farm chores). Sometimes, its more cost effective for me to pay someone to do some work (that I could do, but don't have the time for). You don't have to live like a pauper, UNLESS YOU CHOOSE TO! We choose to do without some things. I do without TV but refuse to do without AC in my house (I prefer 65* year round). That is your choice what you want to cut back on. Debt is a big issue. We found out as soon as my wife and I got together that we could live on my check, and pay off debt and save with hers. In 3 years, we paid off over $112k of combined debt. Remember, cash flow is key. If you are constantly dumping money into your homestead, and getting nothing in return, something is wrong! Take a second to step back and reevaluate the situation.