Husband and I disagree on visits - advice??

Hi! My husband and I adopted my 7 year old niece just last November, but she has been living with us since May 2011, placed with us from CPS. Currently my brother (her bio father) is still using and looks very much like an addict. My SIL (her bio mom) has been in jail since July 2012 and is getting out on Monday. My husband feels we should let her bio parents visit, and that it's not a problem until it becomes a problem. I have a lot of anger at them and am trying to let it go, but I think in no circumstances should any of my children (I have 4) be around a using addict. I have a picture of Christmas 2011 when I allowed both of them to my house and they look horrible and have their arms around my two little girls and it disgusts me and I never want to let that happen again. Bio Mom completed treatment in jail and graduated from it, but I am scared she will go back to my brother and start using again. But she is clean, so should I allow her to see my now daughter (who was my niece). And what about my brother? Does anyone agree with a using addict being around a child? Even if he is appropriate with behavior? Thanks for listening, and I'd love any advice anyone has for me!