Yikes! Yup, I'm going to double down on the recommendation to document his work hours and try to acquire receipts, or at least try to estimate what the materials you've seen would cost. And take plenty of pictures. That way you'll have the info available if you need to refuse further payment or (hopefully it doesn't come to it) do anything through small claims court....you ready? I agreed to pay $2300 for the 4 garden beds, this was to include everything, materials, labor, and apparently it also came with free migraines in the small print, lol. Well, he told me he messed up and didn't charge enough, I basically said I didn't have any more in my budget for the garden, because I don't, but I would work with him on the design so it didn't cost as much...so now I'm only getting the 3 beds shown for the original price. Original deposit was $500 and he said he was short on purchasing the materials and needed another $500...so I'm $1000 in and you see what I have to show for it.
I hate being pessimistic, but you're definitely not in a good spot. At least he seems to have picked up the pace a bit.