When the egg shell is deposited in the shell gland, it is white for brown eggs. Look on the inside when you crack a brown egg and remove that membrane, you will see the white color. The brown is pigment laid on top of that white eggshell in the last 30 minutes or so in the shell gland before it is laid. Those speckles are put there at the same time. As the egg comes out, the hen coats it with a liquid that quickly dries. That liquid is the bloom, put on after the brown and speckles are put on.
You can rub that brown off with your thumb if you try without washing it, that takes the bloom off too. When you rub it with the wash cloth you take the bloom off and part of the brown, that's why it turns a lighter tan.
That pigment does not come off with water but bloom can. If you rinse it off I'd refrigerate the egg, whether you rub or not.