That is my exact plan
I just paid off the house and almost have enough for a 20% down payment
Friend of mine paints and does this and that in home maintenance. Not a contractor but is licensed. Around 2000 he bought a fixer upper house in east Oakland, CA and spent a few years fixing it up, doing most of the work himself. This was before the real estate bust of the early 2000's and real estate was surging. He'd refinance occasionally, pocketing a bunch of money each time as his property got more and more valuable. He acquired a run down fixer upper way out in the boonies of northern California, about 140 miles north of his east Oakland house. He'd spend time at both places, working on them. Then he sold his Oakland house just before the real estate bust. He was sick of east Oakland, had a run in with a neighbor, he wanted out. He's lived in his tiny boonies house ever since, I guess around 20 years now. He wants like crazy to move because the Clear Lake culture is lousy with meth freaks, rednecks and there's not much to do. He wants to trade his place for a fixer upper close to the coast, preferably in or around Bodega Bay. He's got a lot of stuff packed up in anticipation of this stored in a shed. Meantime he continues to work on his boonies house and gets odd jobs in his vicinity, painting and doing other building maintenance activities. This pattern has persisted for several years. He seems to be stuck, but I suppose he could make his escape as he intends at any time. He's broke now but figures to inherit some money because his mom has been in hospice since June, in New Jersey, a stones throw from his 2 brothers who he doesn't get along with one bit. No trust, so his mom's money will be tied up on probate for who knows how long.