Should I stop fostering?

I agree with happymomminy. It's not enough time. Any new person in your household will have something irritating about them. (I'm sure they see things about you that irritate them.) But, it takes time to fall in love with a kid. I think that you can see this in yourself means you make a good FM. Don't stop. If you got a Foster-to-Adopt, I bet you would feel different right off the bat! Keep going!