frustrated scared...single mom

I have a 3 year old daughter...4 at the end of april. For the large majority of her life I have been taking care of her with no help. And when I say no help I do not mean purely no help from her father but also well no help at all. Eventually there was child support ordered...of which he avoided for 6 months. I don't trust his family, (for multiple quite understandable reasons not just because i do not like them) and I prefer her not being around my own family. I broke up with and kicked him out of the place I was living at when she was around 1 and a half but even before that it was nearly 90% 95% of the time me taking care of her, making sure she had what she actually needed while he would ***** and whine about me not letting him buy what he WANTED. Anyways after some bad stuff happend i entirely disallowed him to even be in my house but NEVER i repeat NEVER told him he couldent see his kid...infact for more then a year well pretty much since she was born actually I was BEGGING him to spend any time with her. Setting everything and all of that. Eventually it came to be too much and I just left it up entirely to him when he would come and see her. Which let to months between visits then at one point a full year. Magically his family decided to move back to the city I was in and suprise suprise sponging son moved back with them. Now he was asking to see her so i said fine. I asked him to set up a day each week where he would come visit her to start off with. He showed up two maybe weeks in a row on that day...then didnt show up for 3 weeks on that day. I then moved about 30mins away informed him which city I was in and that if he wanted to see her just call me set a day and i would meet him etc. He is now insisting on having my address which I refuse due to some issues he previously caused me. But this is not me refusing for him to see her. He threatened me with lawyers which makes no sense...im not keeping her from him. He also says the stupidest things like i should bring her to him,... even when we were in the same city. He never asks if we need anything etc. And when he comes to see her he always has someone else with since she was very young. I dunno all this is leading to how unbearable my life is when he just decides to randomly show up see her once or twice then dick off and slander me to his friends. He was the one who convinced me to keep my kid in the first place then he just ****ed off and acted like this. Im thinking of giving my near 4 year old up for adoption because im facing the fact that while her father isent father of the year material im certainly not trophy mom either. I started off with the best of intentions but I cant count the amount of times I think to myself, "should I have given her up for adoption?" There is serious questioning in my ability to be a parent. And while I do not want him to have her as its obivous he is quite incapeable...i cant take all the crap anymore. From him, from his family, from the drama, from the lies, from the everyday dealing with a young child. Its really grating on me and has been for a very long time. I hear about mothers who give their child up for adoption and regret that choice...but how often do you hear of someone like me who regrets the choice of keeping their own child? I was talked into in a way. Well im still thinking of it...and they way things are in my life with me with my kid, my ex butting in, causing issues, me not being stable, trying to go to school to....everything... shes nearly 4 and i dont want my own kid? i feel beyond disgusted at myself....