It's better not to clean them at all! If they are covered in poop, you need to just toss them. If there is just a little, use some sandpaper to scrape it off. If you MUST wash your eggs with water, use water that's the same temp as the eggs. Dry them thoroughly and refrigerate.
An egg shell is porous, so I wouldn't recommend using any sort of chemical to clean them, even soap.
Another tip: if you don't wash your eggs, they will stay good for 2 (or more) weeks outside the fridge.
It's counter-intuitive but true: Your eggs will stay fresher if you don't wash them at all. When your hens lay eggs, there is a natural coating that is laid on top called the "bloom" that helps keep out bacteria. When you wash eggs, you drive some bacteria in through the pores of the shell, so it's a bad idea to do so as a general practice. If your nests are clean, your eggs should be clean. In fact, fresh eggs don't really even need to be refrigerated if they're going to be used soon. They can be kept at room temperature, although refrigeration will keep them fresher longer. (We always refrigerate our eggs--it just makes sense to keep them as fresh as possible!) Store them large end up.
A hen lays an egg a day, and after 10 or 12 days or so, she has gathered a clutch together and will begin sitting on them to hatch them. The first egg she laid has been sitting in the nest for two weeks or so, but is still good enough to turn into a baby chick!
If your eggs are very soiled for some reason, some people recommend you use sand, sanding sponges or sandpaper to carefully buff off the dirt, but it's really best not to wash them in water as that can get the bacteria inside the eggshell. Commercial eggs must be sanitized because they are often laid on top of feces or even worse.
If you do decide to wash your eggs, your solution should be slightly warmer than the egg you are washing to reduce the amount of bacteria you are driving in (as the egg cools, what is not the shell will be drawn inside, and if you have removed the bloom, the bacteria will enter). If you are consistently getting very soiled eggs, though, you might consider changing your flock management practices. Check to see if your hens are sleeping in the nest boxes (and pooping there!) rather than on their roosts. Maybe you will need to change the height of the nests or the roosts, or darken the nests so they will be less inclined to do anything in there other than lay eggs.