However, Magento is fundamentally using an EAV data structure which by necessity requires a lot of caching and indexing with consequently periodic heavy server loads.
In this respect, Magento was in my opinion doomed to failure right from the beginning (at least as a deployed system) as these problems are not trivial to fix for companies without unlimited budgets and there is no instruction manual that helps.
No. There simply are things that you cannot do without using the EAV model, it’s just impossible. The alternative to the EAV model is to have dozen columns in your tables and be able to add/remove columns depending on your data, which fails big time if you’re modeling your data in a MVC architecture where you need to write queries depending on your columns, and you do need to do that. So no, EAV isn’t bad, it’s just slow by nature and there’s no replacement in rdbms world.
As for using hhvm/hack, this is questionable if all what you are doing is to use the original php files and not writing your code in hack where the compiler can then optimize your code. You could as well use an opcode cache and get same, if not better, performance improvements.
For the OP, wordpress itself isn’t really the fastest cms out there and it will easily cripple small vps’es. But again, if performance is your concern, throw it on powerful machine(s) and be done with it, hardware is cheap nowadays. Anyway, i personally am against using things on top of things that were not designed to do those things in the first place, if you get what i mean. It’s like using PHP for doing networking stuff, surely it’s possible, slow as hell and restrictive, but possible. On the other hand C is a better tool for this.
Somebody recommended setting up varnish, but last time when i checked(this might have been changed from then), varnish over ssl was a nightmare to setup and when you get it going, you get all sort of nasty surprizes.
Just my 2 cents.