Cass-
You can turn the pile at night, they will not come out when it is dark. Also, if you don't want to turn the pile you can sprinkle some compost tea or molasses over the pile and water it in at night. That will hasten the breakdown of the pile. It may take until winter to be able to use this pile without disturbing or getting disturbed by the bees/wasps.
Tony