I'm so sorry your little silkie isn't doing well. From what I can see he is displaying wry neck symptoms. I've seen this exact behavior in many of the silkies I have that get wry neck. It could be from a head injury or it could be a vitamin deficiency. Unfortunately silkies do seem to be prone to both vitamin absorption issues/deficiencies and also head injuries in my experience.
Putting their heads between their legs, walking backwards, and flipping over are all very common when they have wry neck.
I would go ahead and treat him as though it was a vitamin deficiency. Vitamin E and sometimes vitamin B deficiencies can cause wry neck symptoms. He'll also need some selenium to help him absorb the vitamin E. I usually treat wry neck by mixing vitamin E, selenium, and B complex and giving it via a 1 cc syringe. Him being only 3 weeks I'm not sure if he'll be quite big enough to take the syringe well as you'll need to stick it part way down his throat past the tracheal opening to avoid aspiration.
In your position I'd try getting some poultry cell if you can (it's a poultry vitamin you can put in the water, it shouldn't hurt your other chicks to have it too). I'd also get some selenium goat paste and try giving him a pea sized drop a couple times a day. Sunflower seeds are a good natural source of selenium and vitamin E so if you ground some up and sprinkled it on the food that might help as well.
Sometimes the condition is cured easily or goes away on its own. Other times it can be a long process for them to recover and in some cases they may not (I've personally never lost one to this condition though). In the more serious cases they can progress to where they can't eat or drink on their own and have to be tube fed to survive until the treatment takes effect. These serious cases it is not unusual to need to tube feed for a couple weeks and treat with vitamins for around a month or more before symptoms completely subside. Sometimes the ones with the serious cases are just fine afterwards and don't have a recurrence and other times they have recurring episodes.
With young silkies I always like to have something to tube feed on hand along with some kaytee baby bird formula to feed them until they can eat on their own again. I've found size 14 french to be a good feeding tube size for silkies around 5 weeks old to adults (I haven't had one younger than 4-5 weeks needing to be tube fed). For younger silkies I might try something around size 10 although I'm not sure how well the food would go through. Some breeders choose to put down a chick if it can no longer eat on its own and of course this is a valid choice as well. I do not put them down for wry neck (unless they never fully recover and suffer quality of life issues) but I will not use a chick that had wry neck for breeding.
If they are having bouts like your little guy it's a good idea to put them someplace where they don't have access to water to dunk their heads in (I've heard of cases of them drowning since they can't lift their heads up during an episode). If possible he should only have access to water when you can supervise until he has stopped having attacks. Stress will often aggravate the symptoms, expect symptoms may be worse when you have to give him vitamins/handle him.
Good luck, hopefully he has a mild case that he will recover from easily!