Sounds like our cats. They don't seem to let a cleaned out litter box stay that way more than a few minutes.
I clean our cat's box every day. And every day as I clean it out, there is a cat watching and waiting until I'm done so they can hop right in. It's like a game to see who can be the "first"