Some metal jar lids make a fairly good click. Experiment.
In principle anything works, a whistle, a word, even a flash of light.
In practice, though, you ideally want something that is very distinctive and un-missable, AND very quick for you to 'deploy' with no delay at all.
A clicker in your hand tends to be faster-deployed even than a word like 'yay' or a whistle, hence it is become so commonly used.
With my dog I use a clicker when exact marking of a behavior is real important, basically when teaching something new or shaping an existing behavior; a lot of the rest of the time I just use a standardized-sounding very happy little 'yay!!' as the marker. Both have their pros and cons but they very clearly work somewhat *differently*.
If you are going to want to carry the clicker work over to ridden (or driven) work, then you will almost certainly want to use a verbal cue ('yay' or whatever) either alone or as an alternate to your click, since most people have neither enough fingers nor enough brain cells to accurately click a horse with a *clicker* during, like, canter departs
Pat