The grandfather thing doesn't hold much weight depending on how badly the powers that be want to eradicate some activity. There was a hog farm here for many decades. A city was created around it. Then, a subdivision was built around it. (Didn't the developer know there was a hog farm there already?)
I'm sure the developer figured he could use their financial power to put this guy out of business. It worked.
He was forced out.
Just 1 street over from me - at the top of their road - it has always allowed horses - There was a 1987 ruling that said llamas were beasts of burden and were equal to horses . There are several llama farms in the area. A different family bought a llama from a neighbor in 1994. Their neighbor didn't like it, so tried to get the city council to overrule that 1987 ruling and ban llamas from the "elite" equestrian only neighborhood. Tried to get the city to try to pull EVERYONE's ability to raise them. Thankfully, the city voted 3 - 0 to keep the ruling, and if horses are allowed, so are llamas. So sometimes it doesn't work.