What to do if I sold a collector car with an oral agreement that I could finish the car but there was no agreed time limit for completion and now he wants me to give him his money back?

An oral agreement is enforceable, so if you fail to make the changes, you are in breach of the agreement. 
If there is no set time limit for completion, then if the buyer wants his money back but you refuse to do so, he could sue you for breach of contract. A court then would decide if, on all the facts of the case, you have taken an excessive amount of time and violated the agreement or not. There's no hard-and-fast answer for when you have taken too far--it depends on the facts. For example, say that realistically, the changes could be done in 4 - 8 weeks if parts were available and you worked steadily:
* If it's been 10 - 12 weeks, you're probably not in breach yet; that's 25% - 50% over, but only about a month (or less) over, when the time line was fuzzy. You should finish as soon as you can, but you're probably ok.
* If it's been 4 months, but the problem is that even though you placed a parts order right away, the parts have not come, due to factors beyond your control: a court would probably not hold you liable for a third-party's delays, IF there were no other reasonable vendors or alternatives to get the parts.
* It's been 4 months or more without some factor beyond your control causing the problem--you'd be in breach. 
The above is just an example, to illustrate that you need to look at the facts of the case.