I have toy and mini rat terriers. I use early neurological stimulation with mine from the time they are a day or two old. And my pups seem a lot better for this, but your pup might not have had the same socialization opportunity.
One thing that might be helpful is what I do with my pups is to say, in a happy voice, "Puppy Up", before I pick them up. They are small and people look like giants to them. By saying puppy up they know why you are reaching for them. My grandkids and anyone that comes around the pups when they are young are asked to say puppy up when handling the pups. It gives them a heads up and makes them feel more secure.
He sounds like he is starting to settle in fine. I agree with not rewarding the nervousness with petting. When he over reacts, like with the rabbit, just tell him, silly pup, and ignore the behavior. If he gets curious instead of afraid, then praise that behavior.