So, whats the next step?(Sorry, kinda long)

[kalija]

Many of you know (and have helped me with) the problems I've had with Zuki. 6 months ago I couldn't even take him for a walk for 10 minutes without him barking the whole time. He has come a very long way since then and I think its time to go to the next step but I want to make sure my goals are realistic. At this point this is where he is at: he can walk passed people without freaking out, even on the same sidewalk if its a woman. If its a man I have to cross to the other side but he can then move right by. He can go passed a dog tied in their front yard even if it barks, or a loose dog if it doesn't bark and/or run at us. A loose dog that charges the sidewalk still freaks him out.(When I say loose I mean on invisible fence - the dog can't really get to the sidewalk but very close) He is no longer afraid of inanimate objects such as garbage cans or lawn ornaments (believe me - a big step!). He has come far, but he is still afraid of people.
He will sniff the hand of a guest in our house as long as that person doesn't actively try to pet him - we have instructed all visitors not to look at, talk to, or try to touch him. Typically if we have guests he will observe from the other side of the room, if its a quiet person who follows the "rules" he will eventually go over and sniff. If they try to touch him he ducks away with a quiet growl. The growl is what worries me - is it necessarily a bad thing if its just a warning?
I have accepted that he probably will never like people outside of us. With that said - I would like him to be more comfortable with people - for instance, if my Mom comes over to babysit, Zuki goes into the other room. He doesn't bark, is not aggressive in ANY way - he just leaves the room. The problem here is if she puts him outside to potty, I worry he might not come in for her. Thus, we haven't had her babysit for months. She and my Dad come over at least once a week for dinner and have since we got him so I don't know why he won't accept them. I've tried having them offer him treats, he just won't take them. (At dinner, he lays under the table, right by thier feet, just like he does when they aren't there) I almost had him signed up for an obedience class back in February but at the last minute decided he just wasn't ready. At that point he couldn't do any of the things he can now. I am thinking I might be able to try it now?
I am looking for feedback - are my expectations reasonable or should I just be happy with what I've got? I've actually had people comment that they don't know why I am so obsessed with this since they say their dogs are WAY worse than him?!!(yes, that freaks ME out!)
Sorry this is sooooo very long. Can anyone comment on where I should go from here?