I’ll start with two of my standards responses.
1. You are dealing with living animals, no one can guarantee you of any behavior. It might work out great, it night not.
2. I always recommend you keep as few roosters as you can and still meet your goals. That’s not because you are guaranteed problems with more roosters, just that problems are more likely. I don’t know your goals but I’m pretty sure the right number for you is either zero or one. You are the one that has to decide that.
Your concern is adolescence, which is the right concern. Plenty of people have flocks of adult roosters and hens with ratios less than your 4 to 1 and it works out great. But you don’t have roosters and hens, you have a cockerel and some pullets. It doesn’t matter if you have a 4 to 1 ratio or a 15 to 1 ratio, adolescence can possibly be rough. It can also go quite smoothly. No one can predict with any certainty how adolescence will go with yours. I don’t have Silkies but I’ve read enough posts on here to know that you can have problems with any breed and you can have no problems with any breed. I fully agree with OHLD, the only way for you to find out is to try it.
Sometimes the problem is not with the flock itself but some people see certain behavior and just can’t stand to watch. While some of us consider that behavior normal chicken interaction some just have to put a stop to it. I don’t know what your expectations are or how you might react if it does get rough. I think it is always a good idea to have a plan B ready. With chickens it is always possible you will need to isolate an individual on pretty short notice, it doesn’t have to be the rooster. You may need a broody buster or may need to isolate an injured hen. Some separate predator proof enclosure is a good idea. A metal dog cage can work really well, but other things can work also. Even if you don’t have any males, have an idea in the back of your head what you might do in an emergency.
In some ways Silkies are special needs chickens because they can’t fly. They need to be able to get into the nest to lay. Roosting can be problematic. Ramps or maybe ladders can come in quite handy in providing access. Some Silkies never roost, even when they have access to some type of roost, some do roost. I don’t know how old yours are or how they spend the night. It sounds like the Silkies spend the night on the floor of the coop while the others are roosting, but I don’t know that for sure.
If you remove that cockerel there will be some changes to pecking order and flock dynamics. Since he has not exerted his dominance yet it probably won’t be a big change. When they are immature and before they start to roost chickens often like to huddle together. It has nothing to do with them being cold, they just like the comfort of another chicken close by. From your screen name I assume you can have pretty cold winters. As long as you have a coop with good ventilation and breeze protection that should not be a problem with your chickens. They can keep themselves warm without huddling together for warmth. If you remove that cockerel the Silkie pullet will have to adjust. Chickens don’t like change but they are pretty adaptable. Any social disruption should be over within a day or two. Your Silkie pullet will manage quite well without him, any stress should be very short-lived. Don’t let that short term influence your long term situation. If you are going to remove him, just do it. Things will quickly sort themselves out. The chickens will not suffer long term stress over this.
I cannot give any strong recommendations one way or the other. I don’t know your goals, set-up, or situation. I think your odds of success are pretty good either way but of course I can’t give you any guarantees.
Good luck!