Am I in the wrong?

I was recently contacted about a program I wrote. Someone wanted to purchase the source code and so he paid and I gave him the source code (a hobby project I made that he obviously wanted to sell to other people). After I give him the code, he asks for a refund and says I will be in a lot of trouble if I don't refund the money. Am I in the wrong here to not want to refund?

I spent roughly 40 hours working on this and gave it to him for $300 if that helps at all.