Actually, it's a very common mistake, though it usually goes the other way. I have seen some very experienced rabbit breeders get embarrassed at a rabbit show when what turns out to be a young buck is entered in the young doe class. You have to get really personal with a young rabbit to know its gender; it's easy to make mistakes. We rabbit people often joke about the rabbits changing sexes - they don't, of course! - because the "parts" are small and the differences subtle. By the time an animal is of reproductive age, of course, it's hard to miss!
Congrats on the little ones!