My mom set up my dad's closet so that there was never a problem. And anyway, he only ever wore white, black, or blue dress shirts. And all of the suits could match any of the shirts. 
So it was just the ties... and my mom organized those.
He did come home one say with his new glasses.... pink. My mom marched him back to the glasses place, and fussed at the people working there when they didn't want to swap them out. She said "HOW could you not have thought to mention to him that the glasses he was picking where pink? HOW? Just LOOK at the man!" And there is my dad, nicely dressed, in a suit, ironed shirt.....
Anyway, they swapped out the glasses for free. My dad thought that they were gray.
As to you not being colorblind... if your dad wasn't colorblind, and you are female, you won't be either. He gave you his "able to see color" X.
But since your mom was colorblind, then you and your sisters (and your cousins... since their mom was also colorblind) are all carriers... any sons have a 50% chance of being colorblind.
But any of your brothers would have been colorblind.

So it was just the ties... and my mom organized those.
He did come home one say with his new glasses.... pink. My mom marched him back to the glasses place, and fussed at the people working there when they didn't want to swap them out. She said "HOW could you not have thought to mention to him that the glasses he was picking where pink? HOW? Just LOOK at the man!" And there is my dad, nicely dressed, in a suit, ironed shirt.....
Anyway, they swapped out the glasses for free. My dad thought that they were gray.
As to you not being colorblind... if your dad wasn't colorblind, and you are female, you won't be either. He gave you his "able to see color" X.
But since your mom was colorblind, then you and your sisters (and your cousins... since their mom was also colorblind) are all carriers... any sons have a 50% chance of being colorblind.
But any of your brothers would have been colorblind.