Oh my god, that's so relatable! I often have a slightly different thing: I'm scared of criticizing myself while working (like, working on a task and seeing that I'm not doing very well and getting depressed over it) so I just avoid work altogether.
Acknowledging this chain of thought is the first step. Congratulations on figuring this out! It took me way too long personally. Next, I would recommend getting into... almost like a researcher's mind. Got a work task? Approach it like an experiment. "I'm going to sit down and do this, and I'll check if I really get exhausted and destroyed". It's just an experiment; any outcome is okay - you're just researching. Write down the results too, so you can gather statistics that you can check any time you're scared again.