I don't think an extra 10 or 15 minutes will make a huge difference in spoilage. Hunters will shoot a duck or a pheasant and then carry it around for hours while they hunt for the next one and it doesn't seem to effect the safety or quality of the meat.
However, I also don't think it hurts to do what you are doing. One thing I do when I'm processing my chickens (I hand pluck) is that I do as quick a pluck as possible right after the chicken has been scalded. There are still some random feathers and pin-feathers here and there as well as a little bit more on the neck and perhaps down below the drumstick when I bring it inside to remove the innards. Some of those feathers take care of themselves when the excess neck skin is discarded and when the lower part of the leg is removed. The rest I ignore until I've finished the butchering and doing the final rinsing out of the body cavity. When the chicken is rinsing, I clean up the outside too before placing it in the ice-bath in the cooler and go on to the next bird. The running water helps wash any remaining feathers off my fingers and off the chicken's skin once they have been plucked. I've never done more than 6-9 on any given day, so when they've been in the ice bath for a few hours, I usually take them out to drain in the dish drainier and usually spot a few more quills that need to be removed. Then they are tucked into empty grocery bags and placed into the refridgerator for a 2-3 day "rest" before they are either bagged whole or cut up into portions and shrink-wrapped. I often find a few more quills at that time too and pretty them up before they go into the freezer.
So, I guess I'm doing more or less what you are doing, the bulk of the plucking right at the begining, but not worrying about perfection. I want to be sure that I'm presenting an attractive meal for my family, but don't care if it takes a few tries to get it there.