The LibrarianCentre contains a script called liststats that trawls back though editing events in the LUMP database and tries to determine how many lists in each department have been edited by which types of user.  Our library staff use the resulting statistics for a variety of tasks, such as distributing workload amongst teams and planning future loads.

The trouble is that liststats can be a bit slow, especially on a busy production database.  Today I’ve been doing some tweaking to speed it up, and got some pretty good results (mostly by creating some new methods to find the maximum priority a user has, either associated with a particular SU ID or in general).

All well and good, but it still takes some time to run so I decided to be flash and put a progress bar on the page whilst the script is running.  Hmmm… but the liststats script is just a plain old bog standard CGI script written in Perl, rather than one of Jason’s whizzy AJAX Javascript jobs.  A bit of hacking reveals that you can run Javascript whilst a page is loading though… so you can do a bit of sneaky Javascript and CSS to give us a nice graphical progress bar (with a textual incrementing percentage completed for the folk with no images/CSS/screen readers).  Cool – worked fine on our test server.

Then I moved it to our live server and all my progress bar loveliness disappeared.  What?!  Turns out that whilst Apache 2.x does unbuffered output by default (unlike some of the older Apache 1.x web servers of yesteryear that needed nph- scripts to do it), if you’ve got on the fly compression turned on the buffering is turned back on.  Rats.  Luckily there’s a handy way of turning it back off for particular scripts (so I don’t cock up Jason’s AJAX calls that benefit from compression, espeically on IE with its limited connection limit):

SetEnvIfNoCase Request_URI liststats.test no-gzip dont-vary

This just needs to sit somewhere appropriate in the HTTPd config files – on our CentOS box its in the included ssl.conf config file fragment.  Everything works again – hoorah!