Guardianship to adoption.

I don't know if y'all can give me any advice, but I'm really at my wits end on how to proceed with my situation. Here's a little bit of backstory: We live in Louisiana. My husband and I have court ordered guardianship of and would love to adopt his 2 year old cousin. He refers to me and my husband as mommy and daddy, my parents, grandparents, inlaws ( his biological great aunt) all as his grandparents. His biological mom is my husbands first cousin, I have a good relationship with her. She sees the baby when it is convenient and on some holidays. She was 16 at the time she had her daughter (which was taken by child services, and then given back, and she still has her). She was also in child services custody while pregnant with my son, and living with my mother in law. She had him at 17. We started keeping our little man when he was 2 1/2 months old. Of course with it being family, we saw him at least once a week. As time passed, a day trip would turn into an overnight trip, an overnight trip would turn into the weekend which would eventually turn into the week. A week would turn into two weeks, which would turn into months at a time. The problem? She was still considered his legal parent. This went on until he was 11 months old. My husband, the baby, and myself, along with my family took a trip to Tennessee, he went with her permission for course. She didn't call or text to check on him at all. Her grandmother, who was also her financial support, passed away while we were in Tennessee. She finally figured out she could not care for both children, even with government assistance. She said that we could "have" him. Nothing was done permanently. We tried calling the department of child and family services the entire time. Their reply, "as long as the child is left with someone responsible that can care for him, it isn't abandonment or neglect." It finally took her not filing paperwork to renew his Medicaid to get something done. DCFS didn't want a long drawn out process because she didn't fight it, so instead of putting him in foster care, we were given legal guardianship. We obtained legal custody through the state on November 5, 2011. His biological mom would allow us to adopt him in a second. She has no issue with this at all. What we're having trouble with now is, the convicted felon who is on the birth certificate, but isn't his biological father. He has also been in jail since my son was born, with a brief 2 month stent out but was then put back in on a parole violation. He's in prison until September 2014 because he escaped from his work release program in October of 2012. He came to the first court appearance when the baby was placed with us, but because he was on the run he didn't come to the second one. He was found in our area and made threats against us and the baby. The stress that he cause during this time caused me to have a stress-induced miscarriage. He claims that as soon as he gets out if prison and on his feet, he is going to come after his son. Everyone tells me not to worry, that there isn't a judge on the planet that would give my son back to him, but I can't help but worry. I gave birth to my daughter in September 2013, and I'm concerned about what happens if either of them regain custody. How that will affect my children, myself, our families, and even our friends? DCFS considers the case closed, so our case worker will not get involved or even give me advice on how we should proceed. My husband is a bi-vocational youth minister, I am a stay at home mom and student so we do not have thousands of dollars to spend on lawyers fees. I just need advice on what I need to do.