1. Compile the code, I mean it. Not bytecache (like use APC with PHP or DLL with VB) but really compile the whole thing to a native code to the language of the processor. Facebook does it with HipHop, Google does it with Java. Whatever you choose make sure the native image (assembler) is running on your machine as a result of whatever you come up with but just compile the whole thing. Not all of the technology stacks would let you do that but if you can, make sure the whole thing is compiled, not parts of it. Once you’re over this point you’d come up with different and much more accurate optimisation strategies when you do it correctly.
Also going along this path you might also want to combine your JS/CSS stack for a lower footprint. Combining has nothing to do with compilation but fewer HTTP requests do a great favour, at least until you implement some HTTP replacement strategy, if really required.
2. Bring stuff closer together. Loose coupling of multiple remote services makes things slower as often times it also means stuff are running on servers running at different speeds, different technologies, not always optimised. Okay, you can’t really do anything if Google Analytics start slowing things down. Many services are not essential for your website so with those cases you can defer loads with some caution and care. The rest of the stack should be sitting closely together as at some point of your optimisations the geography start to matter (once again?). Therefore have a look at CDNs as you not only need your services to live together but your content to live close to your visitors.
Also going along this path you might also want to minimise your JS/CSS stack for a lower footprint. Minimising means less content to be shuffled around and faster delivery. Ideally you’d like to combine and minimise (compress) stuff, making sure there’s some dynamic compression (GZIP?) supported by your web server.
To sum it up: Compile the code and bring stuff together and you’re in much better place to start serious optimisations from here.