Please make sure they have plain water available too!
If they're already drinking some water, then they need water in addition to milk.
At 4 weeks, they will probably be fine. It's not rare for people raising meat rabbits to wean the babies right at 4 weeks. They usually do fine.
A rabbit's pregnancy is usually 31 days, which is a little over 4 weeks. I think she just got re-bred promptly after giving birth to that first litter.
Is there a buck who shares her cage? (Not a good idea, BTW)
Is there a buck she sometimes gets to visit/play with?
Is there a buck she could have possibly gotten together with for a single minute?
(Assume that a parent, sibling, child, friend, etc could have opened a cage briefly, even if you did not.)
I'm an expert too! (Or at least, I'll claim to be

) I say she must have just gotten re-bred, with no funny sperm-storing business. Chickens store sperm, rabbits do not.