I agree with the "no treats by hand" rule for most people, and all children.
I disagree with the statements that horses must continue the bite. I worked full-time with horses for many years, and raised many youngsters from birth to the showring, including some stallions. I've received my share of bites (actually, not that many), mostly from the young colts still learning boundaries. They most definitely CAN let go. Trust me. They can and do, for the lead mare, meaning me. The lead mare squeals very loudly and lashes out like lightening, hard and fast, usually a swift kick in the ribs. Then becomes calm the instant the boundaries are respected.
That being said, there is a safe way to open the horses' mouth, even the clever ponies' mouth. Look at the previous link to the diagram first. You will put your thumb into the mouth at the corner of the mouth and run it firmly along the lower toothless portion of the gums and into the tender area just inside, BELOW the tongue. Sort of where you brush your teeth on the inside, along the gums...feel for the roots of your own molars next to your tongue to get an idea of what you are feeling for.
Press HARD to keep control of where your hand is, also to cause sufficient discomfort to the horse to get him to open his mouth. This will take about a second or so if done correctly.
Ask the next professional (vet, farrier, experienced friend) to show you how. It is a skill you may need again....hope not, but you probably will, even if it is your sleeve or a button you are rescuing.
I met a woman at a boarding stable who offered someone else's horse a carrot and got the end of her index finger bitten OFF as a result. A year later, she is still having surgeries to repair the damage, which refuses to heal correctly. I also once had a mare given to me that I retrained....but on that first vet visit, she picked the poor guy up by one shoulder and carried him down the aisle before anyone could react. He was a good-size guy, too. Oops. I found out from his assistant what his favorite snacks were and always had refreshments waiting for them both from then on!