If you've never seen Gaslight, go see it. If you can't drop everything to see a fantastic film right now, suffice it to say that Gaslight is about driving someone mad in very subtle ways.
Redhat 9 and Apache 2 just did the trick.
I'm working on a little CGI to produce pretty HTML diffs between module versions. Someday it might be part of search.cpan.org, but for now it's just a gleam in my eye.
Anyway, I decide the best way to test it is to get Apache setup to execute CGIs out of ~/public_html on my brand new Redhat 9 installation. I open up httpd.conf and do the obvious - uncomment the AddHandler for .cgi, setup userdirs with ExecCGI, make sure permissions are all setup right and start up Apache. Hit the CGI and wham, I get a 500 error and this in the error_log:
[Fri Apr 18 01:11:25 2003] [error] [client 127.0.0.1] Premature end of script headers: example.cgi
I double, triple, quadruple check everything. At this point example.cgi is just a single 'print' with perfect headers. I run it as the Apache user and it works fine. But from a browser, I get nothing but 500s.
An hour later I find myself searching the web for "Premature end of script headers". I've been developing CGIs with Apache for almost 5 years now. At this point I am totally mad.
Finally it dawns on me. Maybe there are new logs in Apache 2! And there it was, in suexec.log of all the damn places:
[2003-04-18 01:14:32]: uid: (500/sam) gid: (501/501) cmd: example.cgi [2003-04-18 01:14:32]: directory is writable by others: (/home/sam/public_html)
That's a problem!? And how did suexec get turned on? I look through the conf file which I've already read line-by-line. There's nothing in there about suexec! I hit the Apache docs and find this lie:
If Apache finds a properly configured suEXEC wrapper, it will print the following message to the error log: [notice] suEXEC mechanism enabled (wrapper: /path/to/suexec)
I can tell you with 100% certainty that no such line is in my error_log. To make matters worse, turning on suexec is a matter of removing the suexec binary from the path, not making a change to the conf file! Check it:
If you want to disable suEXEC you should kill and restart Apache after you have removed the "suexec" file.
Holy crap! I have to delete /usr/sbin/suxec in order to turn off suexec in Apache?
So I did it.
And everything started working nornally.
And I'm totally insane now.
-sam