Playdate problem?

My daughter has playdates. Sometimes the kids do drop off playdates (without the parent). Theres are a couple moms I feel comfortable with having my daughter there without me but theres one mom I wouldnt feel comfortable with. I know the day will come when she wants to invite my daughter and Im not going to want to but itll be clear that she's being alienated. She's just not attentive enough. How do I handle this? Or should I stop doing drop off playdates with anyone so this wont be a concern?