Open adoption struggles

I have been struggling with open adoption for a long time. My son was taken from me at the age of 18, my then ex gf was using drugs and couldnt keep her self out of trouble and I had an unstable living envrioment due to my adopted parents and I always fighting. I refused to sign over my rights, but I was eventually told I had to, or they would be taken from me. After I lost my son in court I thought that was it. Somehow though my adopted parents still kept on seeing my son. It pissed me off to say the least. They told me that they had obtained sometype of rights through the open adoption to see him. So for years they went and saw him once a month , talked to him on the phone, put his pictures everywhere ....it killed me. It wasnt until my son was about 7 that my now wife contacted Amom to try and establish some type of contact. She was prepared for the answer of no, ,but to our surprise she started to write to us about how my son was doing, she sent us pictures and two years later after consistant contact we set up a time for me , my wife , my mom and my son to spend the weekend alone together. It was awesome. It truly was and I am eternally greatful. After the visit though, my mom got kinda jealous that we were sharing contact with my son and told the Amom some not so flattering things about me that landed us with no emails , no contact period for about two years. My mom had lied to me and told me that my son had stopped calling her all together as well, but i found out she had been lying. It was a very hurtful situation. I was angry that my emails were no longer being answered, that i felt like I had just gotten back into my sons life and now it was taken away from me AGAIN. when my son was 12 I got back into contact with him. He got a cell phone of his own and his Amom was okay with me calling and texting him freely. I was able to send him cards and gifts, have unlimited contact over the phone etc etc. It wasnt until my mom passed away when my son was 13 that I was allowed to go down to visit him and have a whole weekend with him unsupervised. I get annoyed sometimes with Amom when I dont hear how he is doing. Now that my son is a teenager, hes moody and sometimes unresponsive to my texts... so i get upset because i try to talk to amom about it and she shrugs it off. Dont get me wrong , I AM eternally greatful that Amom has let me and my wife have contact, pictures, unsupervised visits etc etc. for that I am eternally thankful. But sometimes there is a shadow of a doubt, that maybe i will say something wrong unintentionally and she will cut off contact, or she will say something about me to my son and he wont like me anymore. It keeps me up at night. I am just thankful I am in in his life and scared at the same time. Everytime we lost contact, its like im in court again being forced to give up my rights.