When I was a child, I remember my dad taking his dad back to his childhood home. We only expected to see the outside, but the owners let us go inside. It meant a lot to my grandfather.
I don't think you need to worry about it not being as he remembered. It is your home now. If he wanted...