Yeah, I'd go crazy trying to keep the entire brooder one specific temperature, especially if it is outside (which mine is). The way I get around that is to make a fairly good sized brooder with a whole lot of ventilation up top but with a draft guard where they are, heat one area to the right temperature, and let the rest cool off as it will. Often in the spring, that means the far corners are 20 to 30 degrees below the "recommended" temperatures. Mine roam all over the brooder, going back to the heat when they need to. You'd be surprised has seldom that really is. Let them find their own comfort zone.
They can definitely get too hot. They can handle being a bit too cool a lot better than being too warm.
At what temperatures is it necessary to have a heat lamp? You only have four chicks. They don't have the ability to huddle up and keep each other warm like 15 or 20 chicks would, so you probably have to watch the temperatures a bit more than I would, since I usually brood a lot more than four. I'll give you a very safe chart. If you follow this you will not have a problem. I'll tell you right now many of us violate this and do fine. They are a lot tougher than many people think. But for someone starting out and not having experience, it is usually good to have some guidelines.
1st week - 0 to 7 days old - 90 to 95 degrees
2nd week -8 to 14 days old - 85 to 90 degrees
3rd week - 15 to 21 days old - 80 to 85 degrees
4th week - 22 to 28 days old - 75 to 80 degrees
5th week - 29 to 35 days old - 70 to 75 degrees
Usually, by the time they are 4 to 5 weeks old they have feathered out enough that they really don't need any suplemental heat regardless of your climate, unless you are somewhere that is really extreme.
What do you feed them? That kind of depends on what is available at the feed store. The normal rotation is 22% to 24% Starter for the first 4 to 8 weeks, 16% Grower from whenever you stop the Starter, then 16% Layer when they start to lay or 20 weeks. You'll see a lot of different recommendations, but it doesn't matter that much about when you switch from Starter to Grower. Whenever the Starter runs out after four weeks is great. The percents show the percent of protein in the feed.
But not all of us can get Starter. My option is a combined 20% Starter/Grower. You can feed that from Day 1 until you switch to Layer.
Some people feed a 20% Flock Raiser from Day 1. This is sort of formulated for a mixed flock, where some are going to be layers and some will be processed for meat, but it works well for just a flock that will be a laying flock.
There are other options, but this is probably enough. The only real rule is to not feed Layer until they are ready for it. Layer has a high calcium content for the egg shells. High calcium can harm growing chicks, so you don't want to feed that too soon. Otherwise, almost anything goes.
There are some feeds of higher protein content meant for game birds, not chickens. Look on the bag to see what it is recommended for. You have a lot of room in how much protein they eat and are still OK, but I don't like giving them too high a protein content feed. Too much of a good thing is often not a good thing.
Then you have the question of medicated feed. The medicated feed is intended to reduce the possibility of them getting sick from Coccidiosis. For most of us, I think it is unnecessary in the brooder. They are not usually exposed to the protozoa that causes Coccidiosis while in the brooder. That first exposure comes when they first hit the ground. If you take them out of the brooder and let them play outside, then they are possibly exposed. But even then, you can usually manage Cocci by keeping the brooder dry. There are some people with a history of Cocci where medicated feed is important. There is nothing wrong with feeding medicated feed. If you want to, go ahead. But if you have trouble finding it or decide you don't need it, don't panic. Just watch your chicks for signs of Cocci and go from there.
I know this is long, but you asked some pretty good questions. Good luck and welcome to the adventure.