On the underside of one of my crepe myrtle leaves I see a few hundred larvae. They have clear bodies with black heads and are about the size of mosquito larvae, except that these aren't in water.

Should I squish them or preserve them?

If they are not harming the foliage or the rest of the tree, they are harmless beneficial insects.

However, if they are, just spray the tree with neem oil, or a homemade garlic, hot pepper, canola oil, liquid soap spray.

Are they in some sort of web or cocoon or just sitting on the underside of the leaf? ..and they have completely clear bodies?

I don't see any coccon or web. Their bodies are clear, but their heads round and black.
I have a few aphids on the crepe myrtle and was wondering if these are aphid larvae or maybe a predator.

Also, should I ALWAYS spray once I see aphids or should I just let a few have a little snack?

Don't spray automatically. Give the beneficial bugs a chance to take care of it for you. In the spring, when the aphids usually appear on my rosebuds I just wait. The ladybugs never let me down. The appear out of "no where" and have a buffet.

If it turns out that these mystery bugs are not beneficial then you need to take a look at why the plants are being attacked. Howard always says that healthy, happy plants don't have this problem. Pest attacks, of that magnitude, don't happen unless your plants are in stress. So take a look at the irrigation (too much, too little water), has there been a grade change etc..