they won't ''get soft'' if he is almost fully feathered he should be fine, if you think it is too cold for him then go ahead and bring him in
If they shiver or seem to cold bring them in, just use common sense with birds as some will just freeze and die standing on a roost.
Providing what a baby needs does not make it "soft".
Is the chick crying loudly? If so it's probably cold. How long has it been off the heat already?
Have you provided better predator protection so you don't lose the rest?