I'll tell you how I got started. I bought a farm with a layer house. I moved in, put the layer house into production and started selling eggs. I followed the applicable laws of my state, which wasn't much. Wisconsin requires me to register my livestock with the state, so I did that. I wanted to do business under a business name. Wisconsin requires me to register my company name by filling a "doing business as" form with our county clerk. I did that. I started keeping the appropriate accounting books for my own info as well as for tax reporting purposes.
I financed any upgrades myself, or with the occasional use of a credit card. The rest of the facilities were included in the original farm mortgage.
Doing business as a Sole Proprietor means that legally the business and I are one and the same. Any profit the business makes is mine, any liabilities the business incurs are also mine. There are other forms of business structure that you can choose that separates the business from your personal finances, such as a Limited Liability Corporation, or a Corporation. You can read up on these and if one of those sounds like it's appropriate to what you would like to do, an attorney can help you file the appropriate paperwork with your state.
If I wanted to borrow money to finance some major construction or purchase more livestock, I would either save up the money, or get a loan from the bank.
A loan is much easier if you have some collateral to back up the loan though, such as your home (a home equity loan). Otherwise you are going to have to sell the loan officer on a business plan, which may be difficult given your lack of experience in farming and business.