That would be my fear too. The glass aquarium will already hold in more heat, and with the sunlight coming in from the window that light will be amplified - as well as the heat. I have to be blunt and tell you that I'd never do it. I, too, have heard that aquariums work.....but to me they're more like a crock pot than a cozy place for chicks. Of course, if you still want to use it, then I won't say more to discourage you, and you'll probably find a way to make it work just fine.

If you want to use the glass tank so you can watch them, for 3 chicks, my recommendation would be a wire dog crate. You can wire a hardware cloth skirt or even tuck cardboard around the bottom to keep them from escaping. You still have good visibility and they can see you. And you can hose it off when you're done!
As for bedding, I use pine shavings if they are inside, but I don't keep chicks inside but for a day to make sure they are all eating and drinking and not suffering shipping stress. Chicks need exposure to cooler air - they can't be in constant heat surrounding them from all points and thrive. Survive, yes. Thrive, not in my opinion. So I brood mine outside and when I do I use straw. Mne are out there right now, and rather than suffering they are growing like crazy and feathering out beautifully!
Welcome to the Broody Brigade, and remember that your dues are photos of your babies and your setup!