Actually, a few years ago there was a people house up the road from me made of straw. I think they used it for a few years until their other house was finished, or as a demonstration. Don't know if it's still there.
Let the three little pig jokes begin.
Since you technically need a permit to even put a fence post in the ground, I can not imagine the hoops one would have to jump for that.