The ALBC definition of a heritage breed pretty much covers it and Fred's Hens answers are great too.
I'd just like to add that though you may hear many people "bash" hatchery birds, I raised them for over ten years before I became interested in heritage and rare breeds. There is NOTHING WRONG with acquiring and raising hatchery chickens if you just want to have a flock for table eggs and have no interest in breeding. Not everyone wants to hatch, keep roosters or find homes for/dispose of all the extras. The only thing I would caution is that there is a somewhat higher incidence of defects like crossed beaks, spraddle legs or as noted, non-Standard (APA Standard of Perfection, the guidelines for breeders) features like color, size, feathering, etc. Note that those things can also be found in "pure bred" or heritage breeds if the breeder is not culling for them.
If you want to preserve a living piece of history, wrap your brain around chicken genetics, study and understand the SOP and learn how to breed to it, spend a lot of time locating, bidding on and traveling to pick up starter stock, enjoy the addiction of hatching and raising your own chicks, the thrill of getting one that just might be "perfect" and finding homes for all the culls, then by all means, jump into the heritage or rare breed(s) that sparks your interest. Because basically that's what it all comes down to. Buy what you like, whatever breed, as long as they're healthy. I still have cross bred birds that I love for their personality, coloration, crests, and/or egg laying ability. I am also working with heritage and rare breeds primarily because I like those particular breeds, but also because I want to preserve and carry on the qualities that make them what they are as best I can. You may think some of the breeds I like are darn ugly or maybe not. That's what is so great about having so many kinds of chickens. They all lay eggs for the table while providing beauty to the eye of each beholder.
Since I am on the east coast anxiously hoping to avoid power outages from Sandy, I am reminded of an analogy that may make some sense to compare the "modern" breeds with the "heritage" breeds. Each of our family members have smart phones and for the house we just had to purchase a replacement (a roughly 6 yr occurrence) land line phone with all the latest bells and whistles. However, when the power goes out in the extensive areas predicted, also having my 1978 vintage "AT&T Slimline" basic phone THAT STILL WORKS, without electricity, is an advantage. Go with what works for you.
Edited by Laingcroft - 10/29/12 at 7:11pm