It will be tough, but the best thing to do is to spend some time near him while completely ignoring him. Sit on the floor and read a book or watch tv or something. This will help him get used to and comfortable with your presence. Once he's good with that, you can start placing yummy greens near you and on your legs and stuff. The goal here is to get him comfortable with approaching you and to associate you with good things. Then you can work up to hand feeding, to get him comfortable around your hands. Once he's good with that, you can try to pet his head while he eats.
It will take a lot of time and self restraint, but this should help him get comfortable with you. If he is not neutered, you may also want to look into getting that done.
And a quick note on bunny etiquette that may be helpful: You should generally ask your bunny permission before you pet him (except for when you do the hand feeding thing), it is good manners. To do this, hold your hand out near his face. A bit to the side is usually best since they can't see too well directly in front of their faces. If he puts his head down for you, you have permission. If not, you should show that you respect his wishes and refrain from petting.