WordPress can be very fast. Here is the server-side setup for achieving the 97/100 Page Speed score (it is only because of Google Analytics that the score is not 100/100):
- Nginx web server;
- PHP with FastCGI process management (FPM) with APC and Memcache extensions for op-code and general object cache;
- Memcached object cache server.
And on the WordPress side:
- A custom HTML caching layer (wp-content/advanced-cache.php) for reading and writing HTML cache to Memcache with a timestamp based version number used for cache invalidation (which changes every time content is inserted or updated);
- APC Object Cache Backend plugin for WordPress object cache.
- Google Appspot as a free origin/pull type CDN for off-loading all static assets. A few simple WordPress filters are used to rewrite all asset URLs;
- Image Optimizer plugin for reducing the size of JPG and PGN type of images during uploading.
To have such a setup, you must have a Virtual Private Server (VPS). If you are looking for WordPress performance consulting and optimization, please do get in touch.