I have thought of migrating to WordPress but have no idea how to port the 800+ posts and who knows how many comments over.  I simply have not been able to take the time to figure out EE as well as I’d hoped.

But at the moment it’s on the back burner.  I can see the traffic you get is a mixed blessing to say the least.

Les, do what works best for you. There’s no rush to get a dedicated server, either.

I have never used MT or WP, but I have reason to believe that migrating to either one would be relatively painless and each of these choices has advantages and disadvantages.

Regarding drupal, I have a script to migrate content and users from EE to drupal. You can even migrate your multi-site setup and use a shared account database between all of your weblogs. You’d need to port your templates or start with new themes (at least initially, perhaps).

As far as the technicalities of migrating off EE are concerned, I suppose MT, WP, or drupal are all feasible options, each having their ups and downs.

The other obvious solution is get serious about trouble-shooting EE

DOF, the answer is probably MT’s export format (or whatever it’s called).

There’s an EE template that exports all of your content into that format and if WP can import it as I believe it does, you should be all set.

Some caveats apply, like comment ownership (I think)…

Drop me an email when you’re getting serious about migrating.

Elwed’s right about the MT export format. It’s pretty easy to set up an EE template to list off all the entries and comments in MT’s export format which WP can then import using it’s MT import option.

It’s pretty much how I’d handle migrating back to MT if I ended up doing that.

Well, I used to enjoy the technical goodies you dug up when you were on MT, so I wouldn’t mind seeing you come back to that fold.  But whatever works for you—I’m by no means a True Believer in any particular blogging platform.  MT works for me (and I was too lazy to change away from it), so if it works for you, welcome back.

(Reminds me—I need to do my 3.3 upgrade and keep digging into my Notifier problem ...)

Have you thought about moving to a different service provider?  I’m not sure of what your current provider costs or offers, but for $20 a month at dreamhost you get 60GB of hard drive space and 1.6TB of bandwidth a month.  The drive space and bandwidth also increase weekly.

The servers seem pretty good, too.  Should have no problems running your EE, I would think.

Les is hosted in a VPS these days, a move precipitated by his sites melting down in a shared hosting environment. The only shared hosting that I would currently consider an option is a custom plan of PMachine Hosting - the hosting subsidiary run by the makers of EE.

Sam_Charette: Believe me, once you start using near that amount of Dreamhost’s resources, you’ll start having problems (especially with a very dynamic website). If you look at the costs (just hardware and network costs), you’ll realise that even with overselling their prices are unsustainable. Once you look at the CPU and memory usage, I’ll give Les around 20 minutes on Dreamhost before he’s kicked off.

Moving from MovableType to Wordpress isn’t that difficult (I don’t know about EE though) - and Wordpress does have a caching plugin available.

For a damn decent dedicated server, you are looking around $120 per month - that’s for 250Gb HD and 2,000Gb data transfer per month: but it’s well worth it for a heavy CPU usage site. I’ve got a dedicated server for my sites, but that’s mainly because I run very CPU (mySQL) intensive processes once a week which would take down a shared hosting server or even a VPS.

Richy C:

I don’t agree with you.

Now, granted, I don’t know what the specs on the machine Les is hosted on are, nor what the CPU and memory usages are, nor how much of that usage is due directly to SEB, but I can’t see what you say as being terribly accurate.

I’m a software developer myself.  I know how computationally expensive database servers are.  I know what goes into a dynamic site.  Hell, I’ve made my own free comic host (comicdish.com), though it’s still in its infancy.

The reason why I’m skeptical about your information is because I know of another free comic host, SmackJeeves, that has over 2,000 hosted comics on it, and is easily hosted at DreamHost with no issues whatsoever.  It is a 100% dynamic application, and isn’t as slow as SEB when it’s doing image manipulation.  (’course he may not be doing image manipulation on the part that I think he is, but still

Point is that, without specific knowledge showing why SEB eats up a whole server’s reasources on it’s own, and knowledge that said server is particularly beefy, I still have to fully reccommend DreamHost.

mySQL isn’t really that CPU intensive.  Sure, it does definitely take up some CPU cycles, but every SQL query on this page I’m looking at now should total less than a second on a decent database server.

If there is a problem on DreamHost, I suggest that it is the blogging software itself, because there is no way that it should kill a DH server unless it’s poorly written.

And besides… if the service here isn’t up to snuff, what’s the big deal in giving DH a shot to see if it can handle it?  They have a 97 day money back guarantee on their service, and is much cheaper than getting a dedicated host (especially considering that you’d start, for 1/6th the price, with almost as much bandwidth, and automatic backups, which many dedicated server options don’t allow).

So yeah, long story to short: I can’t see SEB killing a DH server.  I’ve seen big apps work fine there, and worst case scenario you try and it doesn’t work.

Hell, I’ll pay for the first month, I’m that confidant.  Get the SEB software running on DreamHost, switch the domain over, and if it dies, switch it back.  What have you lost then?

Sam, your line of argument sounds like Dreamhost’s Wager.

You do not know the specs of the VPS SEB and the rest of Les’s family blogs are hosted on, you do not know the traffic patterns of the site, you do not know the resource requirements of the sites, you are probably not familiar with Expression Engine and you do not know how the software is configured and customized. Unless you’ve read some old threads or looked carefully at some recent error messages, you do not even know if SEB will even fit within DH’s PHP memory limits.

I have no opinion on DH, never having used them myself, but your advocacy is based on faith and not supported by factual knowledge.

SEB’s resource usage is clearly outrageous. Sadly, I haven’t had much time in recent months to troubleshoot and throwing more resources at the problem (i.e. upgrading the VPS plan) worked well enough until recently - although the VPS has been under strain for a long time. And no, it’s not mysql that does it.

The options boil down to:

a) More resources, a.k.a. a dedicated server.

b) Tweaking the site (and finally trying Richy’s suggestion of using squid as an http accelerator)

c) Tackle EE and either fix the root problem or confirm that it’s expected behaviour

d) Move the sites to a different and leaner package.

To state the obvious, options c) and/or d) are prerequisites for fitting Les’s sites back into shared hosting; the single exception is the hosting service of the company that also develops EE, because they know it inside out and could jumpstart c).

elwedriddsche:

You’re right, I don’t know those specs.  Of course, I admitted to that before you mentioned it, but that’s neither here nor there

I do take issue to the idea that my opinion is based on faith.  That’s not true.  I’ve seen DreamHost in action on SmackJeeves.  Over 2,000 comics hosted on that site, and it’s faster than SEB is.  That is an example of a large, dynamic application working absolutely fine on DreamHost.

EE is a blogging system, and even given SEB’s remarkable amount of traffic it shouldn’t be totally consuming a server.  If it is, there’s something very wrong with the underlying software.

Again, I don’t know the specifics of the system, the amount of resources being consumed or the verified SEB usage (as I don’t even know if there’s anything else running on said server contributing to the load).  What I DO know, however, is that DreamHost manages at least one large dynamic site, with thousands of users, without having an ounce of problem.  That tells me that one of the following is likely to be true:

1) SEB would be fine on DH
2) EE is horribly written
3) There is some third-party add-on and/or other software configuration/interaction that is poorly written/configured

You may disagree with me if you like, but saying “you don’t know the situation” without explaining what said situation specifically is doesn’t really constitute an argument.  Give me the computer specs, along with the CPU and memory usage, traffic, and what else is in use on the machine and maybe I’ll have a different opinion.  Maybe not.

Les, I think it’s an option you should consider.  It’s not the be all and end all of server solutions, by any stretch of the imagination, but it’s solid, is capable of handling large systems, is inexpensive and has a risk-free trial.  The worst that could happen is that I’m proved wrong

In the end, though, unless we’re talking about consumer-level machines, there’s absolutely no reason why a blogging system, even one as popular as SEB, should be eating up all of the resources.

Sam,

you recommend DH on the strength of it handling at least one big site and I am well aware that you concede your lack of familiarity with the particulars of SEB and companion blogs. I myself have not claimed that EE is not to blame or that Les’s pain isn’t self-inflicted.

You claim that DH can handle SEB, yet are not in a position to back up this claim and the only way to decide the veracity of it is for Les to try. This is what I call Dreamhost’s Wager - he has nothing to lose by trying, except for the hassle of doing it and probably backing out again.

To be absolutely clear, it is your assertion that DH can handle SEB that I object to, which is not the same as arguing that they can’t. And no, I won’t humor your desire for advocacy by spelling out the requirements.

By the way, what is your relationship to Dreamhost?

So let me get this straight…

You dislike the fact that I advocate DreamHost… not because they can’t handle it, as you aren’t owning up to an opinion on that, but simply because I advocate DreamHost?

What on earth kind of statement is that?  Why would you even argue something like that? I think it can handle it, you won’t say, so why can’t I offer that suggestion to Less anyways?

More importantly, why would you feel the need take my assessment of the situation in such a negative way?  I simply offered what I feel is a very viable solution, and you cut it down right away, just because I say that they can handle it?

Yeah, you’re impartial here alright.  Perfectly reasonable, too.

You say I can’t back up my claim.  Well, honestly, you don’t know that to be fact, and since you refuse to do the same, I see no reason why I should be kept to any different a standard than you.  Don’t ask for numbers if you aren’t prepared to give some.

And as for what my relationship to them is, I’ve already stated it.  I have a DreamHost account for my up and coming comic hosting software.  I think it’s great.  They’ve done a fabulous job thus far, and any problems I’ve had were both minor and quickly fixed.  It’s fast, affordable, and offers plenty of bandwidth for my needs (though in time I will have to increase that, I’m sure).

I’m also a reader over at SmackJeeves, watching it run ever so smoothly on a daily basis, with thousands of comics, each with dozens of pages on average. It’s no small system, far larger than a blogging system to be sure, and it runs fine.  It is a comparison I draw.  Empirical evidence, if you will.  An example of a large system successfully running there can’t be so easily discounted.

So my relationship, you could say, is as a satisfied DreamHost customer.

More importantly though, I very much like SEB, and when I see an opportunity to help out, even if it’s just by giving some friendly advice, I’ll do so.  (if you search waaaaaay back in the archives, you may see me even offering my own server space for SEB to use… I think… was a long time ago, though I KNOW my server won’t really cut the mustard for long, hence why I use DH for my larger system).

Les, I don’t intend to push anything on you here.  To you, I apologize for this becoming more than it should have.  I just wanted to mention what is very likely a viable alternative, and as I said, if you’re skeptical I’ll pony up the cash myself for the first month so that you can give it a try.  If you don’t want to, that’s fine too It was just some advice from someone who believes he knows of something that could help.

elwedriddsche, get a grip.  When someone offers up a possible solution to someone else’s problem, don’t be so quick to cut it down, especially when you’re not willing to back up your own argument.

Unless this discussion turns to open-mindedness, I won’t respond again.

Sam, you failed to get it straight.

I hate the politics of this site, but dammit, it ought to be up and running! All this blabber about whether DreamHost does or does not suck ignores the core issue for Les Jenkins—is ExpressionEngine gobbling up his server resources? If so, what are the ways to tune or tweak EE to improve performance and/or reduce server overhead?

Presumably you’re using EE’s page caching to reduce CPU cycles. You are, aren’t you? What about excluding from your queries the extraneous, unused capabilities of EE using the exclude= parameter to the weblog tag?

Take a look at each use of {exp:weblog:queries} tag and think about what data you need to pull for that tag. Then exclude all the extraneous information so you don’t pay the processing overhead to get it by SQL query.

You can exclude categories, member_data, custom_fields, pagination, and trackbacks. For abbreviated content (for example, a list of entry titles, or recent comments), you don’t need any of this data and ought to exclude it.

Being a novice I have so little idea about what’s being discussed it’s frightening but its language sure a hell sounds exciting.
Hope y’all manage to help Les speed the site up. I’d like that.

Brendon, given the access pattern of SEB and the rest of the sites running off a single install of EE, I’m not even sure if caching helps or hinders. It might be worth a try turning it off…

exlude= and limit= are simple optimizations, well worth checking out. As is double-checking the sprinkling of custom PHP on the templates.

Having said that, chatting about EE’s interna ignores the core issue that Les stated in the opening post - would his needs be better served by moving his and his family’s sites off EE? There are plenty of things that could be done to tweak EE, but at some point that effort outweighs the cost of moving to another software package.

OK let me toss a little water onto this heated discussion (should’ve checked my email before bed last night).

First up, Sam, thank you very much for the recommendation as I do appreciate the feedback. As it turns out I’m a former DreamHost customer myself as it was where I originally ran my anime websites (The Casual Otaku and AnimeLinks.com) for five years back when the deals weren’t as good as they are today. Overall I like the company and I did consider switching back to them before opting to go with the VPS. I thought the VPS solution would be better in part because I’d have more control over what was being run. There’s quite a few EE sites running on DH servers that I’m aware of so I don’t doubt that they’re capable of handling it, I’m just not sure they’re capable of handling my sites without going dedicated.

Elwed, let’s not discourage folks from mentioning services they’re happy with even if they may not fit our needs. If nothing else it’s something to check into that might lead to something that does work. I’d hate to have folks not speak up for fear of being shot down. Sam was only trying to be helpful.

Brendon, yes I’m using EE’s page caching and the exclude and limit options. The comment pages would benefit greatly if I were to make use of pagination, but doing that breaks the links to the comments in the sidebar and underneath the entries as there’s currently no way to have a comment listing take into account the pagination. I’d have to write a plugin to do that and I’ve not gotten around to it yet.

I have to be honest and admit that I’m not a great webmaster and this is the first time I’ve ever tried to run the whole thing myself. In addition to running Apache and mySQL for the blogs, the VPS is being called on to do a minor amount of DNS work (via tindydns) and is also processing email generated by the site. A big part of the problem is that SEB has a number of pages with very high pagerank that attract a lot of spammers that attempt to hit our comments, trackbacks, and referrer logs. I’ve not had trackbacks enabled in months and yet if I sit and monitor the server log I’ll see an average of 1 or 2 trackback attempts every 5-20 seconds. 

EE is a pretty well written package from what I’ve seen (and I’ve spent a lot of time looking at the code) and it’s being used to run some major sites out there. That said it does have higher minimums than some of the other packages out there. How much of the problem is from me trying to run it on a minimal system configuration, how much is just my abuse of the templating system, and how much is constant spam attacks is something I’ve never been entirely able to figure out. Truth is it’s probably a bit of all of those combined to make the perfect storm of webmaster incompetence.

But bear with me as I am learning and I’m also possibly moving to a new job soon that may allow me to step up to a better server solution. I’m also looking at all the possibilities out there to see if anything else would be a better fit. With a little luck and education on my part I should be able to straighten this all out soon.

You ought to explore pMachine Hosting, great service and a reasonable price—plus staff knowledgeable in the ins and outs of optimizing their servers for their own product.

I can also recommend Pair.com without reservation, but being a managed host they are very expensive for a dedicated server (starting $249/month). You should not be on a VPS under any circumstances—it sounds great but then you’re your own system administrator. And as you say, fighting off the spammers, hackers, and other miscreants can be a full-time job. Let someone else do it full-time.

I disagree with that Brendon.  I am running my site on godaddy.com, and I have liked it so far (besides some FTP quirks), but not having access to install whatever I want sucks flat out.  I wanted to set up Ruby on Rails for my brother, but I can’t because I do not have access to install apps since my server is just a webserver. 

When I get out of college, and get a job, I am going to switch all my hosting over to my own server, and do everything in-house.  It is actually becoming a lot easier to implement a safe, easy, and affordable in-home network with a web server.  The only real expense you have in doing this is the cost of business class internet from an ISP.