Can I shoo people out of White Forest?

White Forest on my copy of Pokemon White is currently full of people. However, I want to get everyone to leave except for two of them with the intention of only inviting residents who will make evolutionary stones appear in the forest. I figured talking only to the two people I want to stay would get the others to leave, but it's been days since I started doing this and no one has left yet. Does anyone know of a quicker solution to shoo people out of White Forest if there is one?