Buyer says he is going to take me to court?!

I sold my old iPhone on eBay. Before I shipped it, I factory reset it through iTunes to wipe my pictures and contacts. The buyer received it locked because I completely forgot that could happen. The buyer threw a massive tantrum and threatened to get his lawyer involved, saying I did this on purpose. Normally, I could just remove the iPhone from my account online but I hadn't used the phone in so long I don't remember the login details.

I eventually cracked and sent him a full refund of $50. Now he saying that me sending him a refund is evidence and that I will be hearing from his lawyer?! What do I do?!

FYI: I've got a terrible anxiety disorder and am freaking out!!!