See other 4play Articles
Title: Recent Problems on 4um
Source:
[None]
URL Source: [None]
Published: Mar 4, 2009
Author: Neil
Post Date: 2009-03-04 13:05:15 by Pinguinite
Ping List: *4um Admin News* Subscribe to *4um Admin News*
Keywords: None
Views: 714
Comments: 26
Lately there have been slowdowns and outages on 4um. I do believe that one of them recently was a DOS (denial of service) attack, which worked by simply overwhelming the server with bogus requests with lots of web pages that were more difficult to respond to. That was on the 14th or 15th of Feb. That's the first and only instance where I am confident 4um was attacked. There may have been more but this is the only one I'd certify as such. BUT... there has apparently been another innocent factor which is likely the cause of most of the problems of late (and made the attack more effective). In a nutshell, the ever growing size of 4um's DB seems to have broken a limit which exceeded the capacity of the DB to handle quickly. This type of weakness makes DOS type attacks much more effective, but I believe this one has been remedied in a very reasonably acceptable fashion. Here's a hopefully layman's explanation of what has been happening. First, mysql is a program that runs on the server. It's responsible for storing all the data on 4um. Articles, comments, PMs, everything about the site that ordinary use by 4um members and admins can change. Mysql is a widely used program in the linux world and is very useful for storing and retrieving data like 4um's articles, comments, etc. When a web page on 4um is visited, my own forum software takes requests for various articles and threads on 4um and in the case of someone visiting the LC or LA page, asks mysql for information about what was previously posted. Mysql takes those requests, processes them to dig up from (in the case of 4um) its vast DB and sends the raw articles, comments etc it finds to my software, which then formats it into a nice looking web page and sends that to your browser. One of the features I built into Pinguinite long ago was the ability to filter out categories that people may not be interested in. The category filter. Works similar to the bozo filter but works against the category instead of the person who posted it. But because of a nuance in mysql, I found asking it for articles that were NOT in a category took it much longer to process than asking it for articles that WERE in every other category. So that's how it's been working. So even if someone had NO category filtering at all, the request put to mysql was "send me articles in category" and then proceeded to list every category in the DB. Up until recently, mysql could respond to this request by doing everything in it's memory. And apparently, at around 96,000 articles which is what 4um has now, the request started to go over the limit for what mysql could process in it's memory, and it then needed to use the hard disk to figure out what to send which takes a LOT longer. As the number of articles approached a limit, it more and more often had to struggle to service those requests, hense the increasing problems of late. It seems that at about 5:30 AM this morning, it crossed that threshold for good and not even "rebooting" either mysql alone or the entire server returned things to normal as has been the case in the recent past. But the good news there is it makes it very easy for me to see what the problem is and address it. So, I've disabled the Category filter. This allows mysql to respond to requests for information without using the hard disk which is much, much faster, and, it seems, it can now easily keep up with the demands from 4um. Normally, mysql should be able to serve requests from much bigger DBs without using the HD and it can -- if the conditions are right. The fix today was rather drastic in one sense, because it involved NOT asking mysql to do as much work -- my software is now being nicer to it. Maybe mysql should have been able to do the category filtering and the current mysql version just couldn't handle it due to some inefficiency in it. I've read of one discussion on the net where it was debated whether a similar issue was the fault of mysql OR the fault of a program that asked it for data, so the question of "blame" in this case can be a bit fuzzy. Since it appears no one was using the category filter, I think disabling it entirely can be considered an appropriate fix. But the next step is to just wait and over the next month or two and see if there are any more problems.
Post Comment Private Reply Ignore Thread
Top • Page Up • Full Thread • Page Down • Bottom/Latest
Begin Trace Mode for Comment # 17.
#2. To: Pinguinite (#0)
96,000 articles? I know that you know more about this than I ever will, but sometimes a private can offer a good suggestion to a general... Can that number be pared down? Perhaps put them into a separate and searchable archive? Would that help?
And half of them have the word "Turtle" in them!
daggone, time for me to start charging you half for this joint! ;)
There are no replies to Comment # 17. End Trace Mode for Comment # 17.
Top • Page Up • Full Thread • Page Down • Bottom/Latest
#14. To: PSUSA (#2)
96,000 articles?
#16. To: Turtle (#14)
96,000 articles? And half of them have the word "Turtle" in them!
#17. To: christine, turtle (#16)
daggone, time for me to start charging you half for this joint! ;)
Replies to Comment # 17.