Properly formatting and compressing images can save many bytes of data.
Images saved from programs like Fireworks can contain kilobytes of extra comments, and use too many colors, even though a reduction in the color palette may not perceptibly reduce image quality. Improperly optimized images can take up more space than they need to; for users on slow connections, it is especially important to keep image sizes to a minimum.
You should perform both basic and advanced optimization on all images. Basic optimization includes cropping unnecessary space, reducing color depth to the lowest acceptable level, removing image comments, and saving the image to an appropriate format. You can perform basic optimization with any image editing program, such as GIMP. Advanced optimization involves further (lossless) compression of JPEG and PNG files. You should see a benefit for any image file that can be reduced by 25 bytes or more (less than this will not result in any appreciable performance gain).
Avoid filters
The IE-proprietary AlphaImageLoader filter aims to fix a problem with semi-transparent true color PNGs in IE versions < 7. The problem with this filter is that it blocks rendering and freezes the browser while the image is being downloaded. It also increases memory consumption and is applied per element, not per image, so the problem is multiplied.
The best approach is to avoid AlphaImageLoader completely and use gracefully degrading PNG8 instead, which are fine in IE. If you absolutely need AlphaImageLoader, use the underscore hack _filter as to not penalize your IE7+ users.