You have to kill the dust or you'll just keep fighting it. Blow some bedding out on the bare ground you know they like to use and if you have a water wagon or sprinklers spray it down as well. If it's a dirt floor barn and there's not enough of them to wet the floor themselves, I'd bed the barn as well.