Anyone Have Experience with Removing a Child from a Relative?

We are in Riverside County (well, our child was placed from Riverside). Bio parents had children already in care so baby was taken soon after birth. Grandfather (maternal) has older brother (3yrs) and has had him for over 10 mos. He was not considered for the baby as he was already struggling with the brother. DPSS has given him multiple warnings and "chances" to get it together and are now concerned for his ability to care for the brother long-term. When we were placed with baby we were asked whether or not we'd take the brother as well if it became necessary. We said yes. Now that TPR is the goal for both children, grandfather wants both children. Baby has been with us for over six months and we were granted De Facto Parent Status. My question: Has anyone else been through this? SW is not 100% sold on grandpa's ability to care for the brother, let alone the baby. They are not recommending the baby be removed from us, however, removing the boy from grandfather now is problematic for them due to him being a relative and in his care for over 10 months. We'd like to adopt the brother as well. And now baby's TPR has been moved up BEFORE the brother's! Is this normal? We doubt we'd be able to keep contact with grandpa if the kids were split between us because we believe he still allows bio parents access to the brother which is a big no-no. Also, now that TPR is scheduled, his behavior has become more erratic and unpredictable. The SW tried to initiate visits with us and the brother outside of the scheduled supervised visits but grandpa didn't cooperate...