It is not silly it is painful. You can't change them but maybe it would help to talk with someone outside your family. Do you have a pastor (not the just suck it up preachy kind) or some kind of counselor that you could talk to. Time won't make it go away it will just grow with more hurts. My husband went through this in his relationship with his dad but got some good advice on how to handle the hurt.