I would agree that your pup probably has some deeply ingrained habits...that said, with some consistent training, you will be able to make progress with him. Having him altered will definitely make a difference, the rest is up to you to teach him proper behavior for living in the house. You will definitely need to be vigilent about removing all trace of urine from any other dog from your home. Nature's Miracle and Simple Solutions are good products...make sure you follow instructions esp for the old spots.
Physical corrections will not stop your dog from marking. If you discipline the dog while he is marking, he will just learn to mark where you can't see him. The umbillical leash will be helpful though. Keep the dog near you and watch closely for the mark. You will know when he is getting ready to do it. Get him outside before he marks. This is going to take a lot of time and effort, but with consistent clear training, you will be able to improve the problem. If you are unable to monitor him, put him in hios crate...the crate is an invaluable tool. You need to make sure it is a positive place for him, so make sure you NEVER use it as a punishment. Give him a marrow bone or stuffed Kong to chew on while he is there...many people feed their dogs in their crates to build positive association. If a dog is comfortable and happy in his crate, it makes your life and the dogs a lot less stressful.
If you are commited to neutering and training, this dog will definitely improve the behavior. Also, good for you for taking this dog in. Neutering and getting him off the chain will be the best things for him. If he is used to being chained, there are going to be other behavior problems that arise when you bring him in the house. This is not his fault as he was never taught how to live in the house. There are lots of resources on the web, some good, some bad. If you use your judgement and find a source of positive training info, you should be OK. Thanks and good luck!!!
Here is an article to get you started:
http://www.veterinarypartner.com/Content.plx?P=A&A=2284&S=1