Chad Perrin: SOB

7 July 2007

website failure earlier this week

Filed under: Metalog — apotheon @ 12:39

On Thursday this week, this domain ran up against a 50,000 SQL query per hour limit imposed by the webhost. This is not a limit that is advertised before webhosting service is paid, and it’s not something that is clarified in the control panel software for the webhosting account. I found out the hard way, when I apparently got enough traffic at SOB — somewhere in the neighborhood of 1,000 to 2,000 views per hour would suffice — to exceed the secret SQL query limit and destroy basic functionality for WordPress for a period of about six to eight hours.

I’m working on a fix for the problem to ensure this doesn’t happen again in the future. I’ll fill you in on details when I’m finished shuffling things around. In the meantime, I’ll try to keep up the posting here and at Ameliorations at a rate something above zero, but my free time is pretty well taken up between offline stuff, fixing this problem, and playing WoW with the SigO at the moment. I had the time, until the queries per hour limit reared its ugly head. . . .

11 Comments

  1. Et tu, ‘poth? WoW is taking over the world, one person at a time.

    Comment by Mina — 7 July 2007 @ 09:33

  2. It was in part the discovery that WoW would run on Linux and FreeBSD using Wine without having to pay for Cedega that convinced me to start playing WoW. There was also the simple fact of who it was that suggested it in the first place.

    WoW is a surprising lot of fun. It’s not as good as a true, old-fashioned roleplaying game, but it’s a lot more convenient.

    Comment by apotheon — 7 July 2007 @ 10:23

  3. That’s how I got into it as well- my brother was looking for an MMORPG to play and told me to pick one, any one. I played the trial for a few, and WoW seemed like it had a bit more freedom than the other popular ones on the market- at least the crafting system looked promising. Well that turned out to be a huge error in judgment, but I played it for about a year nonetheless.

    It surprises me that it’s so popular, because try as I might there’s very little in it’s design philosophy that I can honestly consider to be… er, sane. It’s a little suspicious when all my complaints about the game are in fact conscious design decisions. I had a very difficult time projecting imagination onto the game, I could never play the game I wanted with their mechanics, I could only play the game they wanted me to play. Which sounds like a strange complaint, but that’s what RPGs are really, immersion. At first I thought that whoever made the game had absolutely no idea how suspension of disbelief works, it seemed like I encountered something that broke it about every five minutes, but somewhere along the line I realized they probably didn’t care. That most players play it for the mechanics, they like having the rigidly defined path to climb and the window dressing makes little difference to them. Someone on the WoW boards once called it a “virtual achievement simulator,” which I think is pretty accurate.

    So I quit and went back to pen and paper games. But I still don’t really understand why people enjoy WoW so much. Of course, it’s not so bad when you have someone to play with, but outside of that I don’t see much in the game itself that merits it’s popularity.

    Comment by Mina — 7 July 2007 @ 11:11

  4. I surprised that no one has question why, for something as simply as WordPress, it needs 25 to 50 SQL queries per page view! Either that, or your math is wrong… but I doubt it.

    WordPress is, bar none, one of the absolute worst pieces of trash I have ever experienced. For all of the millions of people using it and the fact that it is F/OSS, it really makes me wonder how many (and whose) “eyeballs” are on it. Seriously.

    What is even more surprising to me is that tech people use it, despite the fact that it is impossible to but any kind of code or mathematical expressions in it (it loves to choke on less than and greater than signs) and it mangles the formatting.

    The even more sad thing is, I have not found anything significantly better. I am truly stunned that 10 years after Web development started really occuring, we cannot get a decent piece of software that can handle simple text entry.

    J.Ja

    Comment by Justin James — 8 July 2007 @ 09:26

  5. Mina:

    My biggest disappointment with WoW is the fact that they call these damned things “MMORPGs”. They are not, in any useful sense of the term, “roleplaying games”. The monumental scope of that misapplication of a name to something is staggering.

    I guess WoW is my football. Marvin Minsky once said that it amazes him how a trillion synapse computer can sit around on Sundays watching football. At least football provides a sort of battlefield-analogous tactical component, unlike baseball — but I digress. WoW is quite a bit more immersive than football, of course — more interactive, and more engaging for the mind. It’s just nothing anywhere near a real roleplaying game.

    I don’t have to “go back to” real RPGs. I’m involved in a Star Wars RPG using modified 7th Sea rules, so I still get my fix. WoW is just something else I do with my free time.

    Justin James:

    WordPress is, for the most part, an excellent collection of features — and it’s this that makes it one of the better weblog applications out there, for most purposes. There are some that are better, but setup is generally a bear for them, whereas most shared hosting providers worth their salt provide automated WordPress installation (as well as a great many shared hosting providers that aren’t worth much).

    That being said . . . the code on the back-end of WordPress is pretty ugly. That’s kind of an unavoidable consequence of doing any “serious” programming in PHP, however. There are also some other issues, like the fact that WordPress is obviously not designed for people who like to talk about programming (complete with code samples), or to be read easily by people on limited resource computers (using, say, Lynx without Unicode support).

    Unfortunately, PHP was also basically the biggest win for WordPress, because it’s friggin’ everywhere — so basically everyone can use it. Also, considering the braindead syntax of PHP, it’s absurdly easy for people to create extensions. It’s only recently that stuff like eRuby has started to become more widely available for similar purposes and, frankly, Ruby still looks like it’s out of range for most users because for web development it’s difficult to find information on how to do anything like using eRuby for markup-embedded code. Instead, Rails is everywhere — but that’s pretty high-end for the kinds of people most attracted to PHP.

    There’s also the simple fact that, of the major weblog websites (including Blogger, LiveJournal, and so on), WordPress.com is pretty much the least odious. In other words, it’s sorta the best of a bad bunch for people who don’t want to have to install their own weblog applications or maintain their own webhosting accounts.

    I feel kinda stuck now that I’m using WordPress. I’d like to move to something better, but that requires two things:

    1. something better
    2. ability to migrate to it without losing anything along the way

    That’s kind of a problem, at present.

    The even more sad thing is, I have not found anything significantly better.

    That’s really the key, right there.

    By the way, for simple text entry we’re well past excellence. There are a couple of markup translators out there known as Textile and Markdown that are incredible. Markdown is more well-suited to things like weblog comments than Textile, I think, because it’s so much more prose-oriented than Textile (which is, in turn, better suited for something approaching simple web design using an easier syntax than XHTML). The problem isn’t in the text entry and formatting, but in the collection of other functionality surrounding it that has become “necessary” for weblogs these days — things like trackbacks and pingbacks, RSS and Atom syndication feeds, and comment spam filtering.

    To give you an idea of how nice Textile and Markdown have made text entry and formatting, here are examples of

    • entered text using Markdown syntax (ignoring the weird effects of justified text, of course):
      #### Example:

    This is a demonstration of how Markdown works. It’s pretty slick, and the syntax that it uses is quite familiar to anyone that has used text-based email (such as on technical mailing lists) much. The following is an example of a blockquote.

    I am truly stunned that 10 years after Web development started really occuring, we cannot get a decent piece of software that can handle simple text entry.

    It’s very intuitive and, unlike actual XHTML, it doesn’t degrade the readability of the text. You can learn more about Markdown markdown syntax. Those were two examples of links, the first of which uses a footnote-like means of organizing URLs.

  6. Markdown-generated output:

    Example:

  7. This is a demonstration of how Markdown works. It's pretty slick, and the syntax that it uses is quite familiar to anyone that has used text-based email (such as on technical mailing lists) much. The following is an example of a blockquote.

    I am truly stunned that 10 years after Web development started really occuring, we cannot get a decent piece of software that can handle simple text entry.

    It's very intuitive and, unlike actual XHTML, it doesn't degrade the readability of the text. You can learn more about Markdown here. At the same site, there's a link for markdown syntax. Those were two examples of links, the first of which uses a footnote-like means of organizing URLs.

    Posting the generated XHTML code so it’s readable here in WordPress would of course be a severe pain in the butt, but you can view it with a View Source on this page if you really want to. The rendered text formatting should make the point, though, I think. Markdown even provides an easy means of providing the equivalent of sane <code></code> tags — just indent every line containing the code to be unmolested by translation by at least four spaces.

    Markdown is what’s used in the discussion threads at reddit, and it works quite well — despite the fact that I think reddit uses a slightly modified, less capable version of Markdown. I just finished incorporating Markdown into the text formatting tools I’m using for a new website, by way of the Ruby implementation of Markdown, called BlueCloth (while the Ruby implementation of Textile is called RedCloth). The syntax for BlueCloth is pretty dead easy. Assuming a variable foo that contains a block of text you want to format for output, as a string:

        require 'rubygems'
        require 'bluecloth'
    
        formatted_text = BlueCloth.new(foo).to_html

    Of course, you can do other things with it if you like. You could create the new BlueCloth object using the foo string, using say bar, then do the final translation to XHTML at your leisure:

        require 'rubygems'
        require 'bluecloth'
    
        bar = BlueCloth.new(foo)
        puts bar.to_html

    You could also just dump it to STDOUT immediately:

        require 'rubygems'
        require 'bluecloth'
    
        puts BlueCloth.new(foo).to_html

    Obviously, in any given script, you only need the library require statements once.

    Anyway, that’s my long and rambling digression on how the text input and formatting problem is already solved. It’s just getting the toolset you need combined with the existing solutions that is proving difficult. Obviously, WordPress doesn’t cut the mustard for text input and formatting, but it has all the attendant tools.

    Comment by apotheon — 8 July 2007 @ 03:40

  8. Mina: I can understand why you felt a bit restricted when playing WoW. The first MMORPG I played was EverQuest back in 1999, which I ended up playing for 5 years. I finally got burned out (although it did take me a long time) and quit the game.

    I’ve played several other games since then and none of them have the same magic that EverQuest had for me. The interesting thing is that even though I really liked EverQuest, there were things about it that made it very frustrating and those aspects finally got too much for me. I don’t think I could go back to playing that game again.

    I think that the designers World of Warcraft went out of their way to try to resolve all of those really frustrating aspects of EverQuest, such as the horrible corpse runs and experience loss from deaths, difficulty to get new spells at higher levels, locating quests, kill stealing, and mob camping. All of those difficulties were taken out of WoW, but in the process it also feels like you’ve lost a lot of freedom to do what you want.

    In EverQuest, you were kind of just thrown out to the wolves when you started and had to figure out things for yourself. There weren’t little convenient exclamation points to indicate if an NPC had a quest. You had to just go up to NPCs and figure out the right thing to say to them to get them to give you the quest. In trade skills, there were no recipe books where you automatically create the item every time. You just had to know what items combined to make the crafting item you wanted, and you had to hope that you made it successfully.

    Anyway, I find WoW enjoyable but it does have a lot of hand holding and safety nets in place that forces you down a more linear path. It definitely doesn’t have the same charm as when I first started playing EverQuest. Of course, that could also be the same phenomenon that happens where you think a movie was awesome as a kid and when you go back and watch it again years later, it isn’t that great and you wonder what made it seem so cool back then.

    I think part of World of Warcraft’s popularity is that casual gamers can play and get something out of it. You don’t have to spend hours of time looking for a group or running across a continent to get somewhere to get experience. It is a much easier game and that probably appeals to a greater amount of people.

    Comment by medullaoblongata — 9 July 2007 @ 09:20

  9. Sigh. One day, one day I will have the time to write something better!

    It is sad, actually, (I agree 100% with your analysis of the situation, BTW) that current development tools cannot even get something this simple right. At least in ASP.Net, the cache object would slash your query count to only refreshing the page once for each page read after it has changed. I suspect J2EE has a similar component, but I have not bothered looking. I am baffled as to how this functionality flew under my radar for 4+ years now, but it did.

    In any event, I am of the mind that a better CMS that actually did very little, but actually worked, and could do more than blogging would be quite popular indeed. But as you’ve mentioned, it would most likely need to be in PHP (or Perl), since that is that one thing youc can count on existing on an inexpensive host. :( Reason #2 that I have my own server, despite the hassles…

    J.Ja

    Comment by Justin James — 10 July 2007 @ 07:59

  10. There’s a caching plugin for WordPress — unfortunately, it is apparently buggy (I’d post a link to the relevant discussion of its shortcomings, but I don’t have it handy right now) and its portability is kinda limited. I’m pretty sure it’ll break at least temporarily with every WordPress version upgrade, too — WordPress is not exactly well-designed in terms of its plugin API and similar compatibility functionality. Or, y’know, at all.

    Again, that’s probably largely the fact that it has, despite being a not-very-large application, grown well past PHP’s sweet spot for scalability (which ends just after the point where you’re only using it as a templating language).

    Comment by apotheon — 11 July 2007 @ 01:47

  11. I actually read a very interesting article a few days ago about PHP and its scalability was brought up (http://www.oracle.com/technology/pub/articles/phpexperts/rasmusphp.html). Yes, something useful on Oracle’s Web site… It was a good read that explains a lot. I think that its scalability model does not work well for enterprises, unless they are working exclusively on what I choose to call “dynamic Web sites”, not true “applications”.

    J.Ja

    Comment by Justin James — 14 July 2007 @ 08:18

  12. When I say that PHP isn’t scalable, I don’t mean that it doesn’t perform well under load. With mod_php, it does okay. It’s true that mod_perl is more efficient and performs better, all else being equal, but the difference is not significant (and, in any case, shared hosting providers shy away from mod_perl, forcing you to use Perl/CGI instead).

    My reference to scalability was the scalability of the language itself, not of its (relatively efficient) implementation. Your comment about the difference between a “dynamic website” and an “application” is quite emblematic of the problem:

    Object oriented programming techniques are valuable as a way to allow code to scale, so that a given project is still maintainable as the volume of code increases and as greater numbers of developers are employed (assuming you don’t misuse OOP techniques, of course — which almost everyone does). Unfortunately, the scalability in terms of code volume of PHP kinda craps out right about at the point where OOP starts to become an attractive solution. Part of the reason for this is that PHP’s OOP constructs are so atrocious that you’re better off continuing to use standard procedural code. The only value PHP’s OOP constructs (or POOHP, as Sterling puts it) provide is a way to simulate namespaces.

    . . . which brings me to another problem. The procedural code capabilities of PHP don’t scale as well as they should for a number of reasons, including the complete lack of certain capabilities that other languages possess. I refer to such details as proper namespaces, a sane model of scope, and proper anonymous functions, for instance. This eliminates local/lexical functions, closures, and — perhaps most importantly for code scalability — the ability to organize code meaningfully, among other important needs of complex project development.

    Keep in mind, by the way, that when PHP core developers refer to how great PHP performance is, they’re referring to PHP performance after some commercial optimization help from Zend. The open source implementation without the proprietary optimization is intentionally crippled so someone can extort money to make the code perform better. Ignoring that for the moment, however, the issue to which I referred was scalability of the project, not of the software’s ability to handle high-load processing or deployment scalability.

    Comment by apotheon — 15 July 2007 @ 03:25

  13. I’ll buy that for a dollar.

    Every PHP project that I have looked at that tries to do more than handle a single page’s worth of functionality is an utter mess. Includes are abused to the point where it is reminiscent of GOTO in BASIC. I mean, when it takes you having four levels of “include” to get around the lack of true OOP or even a concept as simple as a library, there is trouble.

    Yeah, I mistook what you meant by “scalability”, and PHP fails both in terms of the technical angle (shared memory within a cluster, message queues, etc.), but in terms of the scalability of the development process, it is a rough ride too. Even something as simple as WordPress gets messy… you cannot even persist (as far as I know) to the level of the server (think the Application object in .Net or J2EE) without acrobatics.

    Ah well. I guess that is reason #4,578 why I rarely see PHP coding jobs, let alone PHP jobs at big shops.

    J.Ja

    Comment by Justin James — 15 July 2007 @ 08:06

RSS feed for comments on this post.

Sorry, the comment form is closed at this time.

All original content Copyright Chad Perrin: Distributed under the terms of the Open Works License