Select, select, select.
However, smutty buffs is not something I have had to worry about. I have had to worry, to an extent, about black in the primaries & tails.
With my firsts, the biggest issue was the boys gnarly, too red combs and lack of type. I've had difficulty hanging on to good boys. Lost a very sweet, very typy cock while I was gone at Christmas. I did get a couple of chicks from him, who are maturing well. Without a buff cock, I am trying to experiment with a ginger red cockerel to see how that will work--unfortunately, the buff ladies are on strike. He has absolutely NO black, but is lacking in crest. Since they all have large crests, I hope they will pair okay, but without eggs, ...
My issue with blue is getting sufficiently light colour that it registers visually as blue. When crossed to a non-silkie, the blue is always the perfect shade, but on silkies it is dark, dark, dark. If you read some of the more advanced genetics site, blue is considered a difficult colour to get right--that is certainly my experience. Buff has been a walk in the park, comparatively.
I don't have off-colours in my blues except on one cross who is in my layer flock, not to mention also being my best broody (who else has a bird who will raise THREE clutches in a row, without a break, hatching only the first?, and will take any chick you give her and happily decide it's time to "mommy.")