Could my job prevent me from fostering?

My wife and I are considering fostering, with the goal being to adopt eventually, and we want to focus exclusively on older kids because of my disabilities. I deal with a seizure disorder that causes chronic nerve pain, and we don't ever want to put a child in a position where they aren't able to do something and I can't help them with it. However, my disability also prevents me from working most jobs. A few years ago, I stumbled into being a phone sex operator. I don't use any of my own information, I work with a reputable company, and no one knows my real name except for my bosses and supervisor. I usually work in my basement office, which isn't soundproof but certainly keeps me separate from the rest of the house thanks to a door. I am an independent contractor, and I am hired through my own LLC. What it boils down to is- will I have to give up my job in order to be a foster parent? I'd never do anything in the same room as the child and ideally would adjust my schedule to work when they weren't there or were asleep. Will my history of working in the sex industry disqualify me even if I do quit my job? I would hate to lose this, because being able to work again and contribute to the household has boosted my mood and self esteem so much. But it would break my wife's heart if we couldn't pursue being parents.