Where are you and do you know what kind of wasps they are?
Wasps are beneficial when they are outside away from people, but not welcome indoors. Do you have a guess as to if they're in a wall, in the attic, or perhaps coming in from under pier and beam? If they're under the house they could be in the ground.
A soapy spray with orange oil will kill them if you can find the nest (
here is a general discussion of treating them) and there are some baits. Please note that Howard stipulates that it might be best to have a pest control operator take care of this. There are organic pest control companies, search the
Organic Businesses and Products listings.
Good luck.