Okay, I have just spent 10+ hours on a single site, the site was for almost 12 hours.. It is basically a BuddyPress, BBPress and WordPress MU installation. Everything was working fine so far but suddenly in the morning, the site stopped working. And since then, I have been banging my head on it to see what could be wrong.
First, I noticed around 3000 odd tables in the DB, thought it might be the reason (spambot attack) but later found out that this is how BuddyPress RC 1 behaves.. it creates 3 activity tables for each user.. and we have around 1400 users. (I figured it out quite late actually haha)
So, after spending numerous hours finding the issue, I got on to IRC channel of BuddyPress, got in touch with the user Burt (Burtadsit), We spent around 2 odd hours, tried out few things and then we moved on to the plugins section, disabled all the plugins, enabled every plugin one by one and finally found the issue.
The plugin Sexybookmarks was trying to connect to its server and the server was throwing back 500 error code (actually, 403 error, this means mostly mod_security rules are blocking it). So, our server was retrying or getting stuck and causing the page to load very slow.. (it used to take around 12 minutes to load the page).
Finally I traced the issue and fixed the things up.
So, the purpose of this post is to make you guys aware that, whenever you feel something wrong with your WordPress installation, first thing you should do is, check the plugins. (Apart from the normal routine checks, DB check and all)
That’s it guys..
Note: If you wish to see detailed issue, you can check this thread on BuddyPress forums