There are lots of perennial herbs, depending on your hardiness zone.
It gets really cold here in winter but greek oregano, rosemary, sage, thyme, tarragon, winter savory, chives and parsley overwinter.
The most tender in my opinion is basil. It is gone with a hint of frost.