I have a 4'x8' brooder right now, sub-divided to smaller chicks (in less than 1/4 of the brooder) and bigger chicks (about 5 weeks old). I have about 3" of sand in the bottom and like you struggled with SLOW cleaning times. I think the trick is all about your scoop.
I started with a cat litter scoop, which just did not catch all the small bits from the chicks. Then I found a "scoop" that looked like it would get the small stuff but still sift my sand out -- which it got all the small stuff plus some of the larger pebbles in the sand. Since it was really a draining kitchen spoon, it sucked for cleaning, it just could not sift much at once. I got 1/4" hardware cloth and put it over the first scoop as a compromise to try and catch more with it -- works better but still not fine enough.
I think the best bet is to get a good sized scoop base with large gaps, be it a cat litter scoop, a barn fork, etc and then cover it with two offset layers of 1/4" hardware cloth. If there's 1/8" hardware cloth out there, that'd do the trick I think (I haven't checked though).
With the 'ol cat scoop I could get the whole brooder done in a few minutes -- but it didn't get it very clean. This worked better for the bigger chicks.
With the kitchen spoon/sift it'd take me 3 hours or so (no joke). This was pretty much the only way to clean the small chick section.
With the cat scoop + hardware cloth it takes me maybe 30 minutes but isn't as clean as I'd like. This works better overall for both chick sections.
I'm guessing with the second layer of hardware cloth, when I apply it, it should take about 30 minutes to scoop still but work a heck of a lot better. Really the speed seems tied to the relative size of your scoop to the task -- bigger scoops just plain sift more sand at a time.
Long term, since I want sand in the coop, I'm guessing I'll just take a plastic bucket of appropriate size, cut it to scoop shape, cut out all of the plastic along the side of the bucket and attach two offset layers of 1/4" hardware cloth, then put a handle on what used to be the bottom of the bucket. That'll give the scoop a lot of sifting power and the fine grain catching that it really needs.
Really though, I'm guessing this is more an issue for the babies - they have droppings to match their diminutive size.