As you've seen from comments, we all do things differently. For me, 4 week olds are off heat (down to mid 40s) and integrated with the flock. Many people take things slower. Do what feels comfortable for you.
Can you turn off the heating in the garage? If so, I would consider turning off one heat source first (either lamp or garage heater) and then after a few days, turn off the other, and that should bring them fairly close in line with the outside temperatures.
What's your heat source? If it's a heat lamp, you can swap out for a lower wattage bulb, or turn it off for half the day, for example. If there's windows where the brooder is, you can prop them open to let in the cooler outside air. Or you can take them outside for increasing duration, i.e. 2...
They can be moved out now, WITHOUT heat, however if you haven't yet weaned them off heat and started acclimating them to outside temps, you should do that first for about a week before they move out permanently.
Might be tight trying to get them into a routine before you leave for vacation, but...