All the Perl that's Practical to Extract and Report

File Handles on Linux/Apache

ajt on 2003-02-28T13:50:59

I'm having a torrid time with File::Temp. I like the module and it works mostly well, but I'm not doing something correctly with it.

In an Apache/mod_Perl PerlRun applicatiion I have the following happening many times:

{  # start scope
  ($SFH, $softer) = tempfile(SUFFIX => ".data",
                             DIR => "/tmp/",
                             UNLINK => 1);
  ... stuff ...
  unlink0 ($SFH, $softer);
}  # end scope

This appears to works on Windows (Perl 5.6.1, Apache 1.3.26, File::Temp 0.12), without a problem, I don't run out of file handles, and I don't have files left over, but Windows isn't used as much. On the Linux box, same spec, much higher load levels, I've had miserable problems.

  • Files left behind, and I eventually run out of file handles and strange things happen on the box. I restart Apache, and delete the temp files, and all is well.
  • I added the unlink0 option, which removes the files, but does not remove the file-handle. I get no files left over, but I still get strange file-handle problems.
  • I'm currently trying an explicit close on the filehandle after the unlink to see if that helps.

As far as I can tell it's simply confused because it's running under mod_Perl, and the Perl processor never ends, so the normal end of Perl clean up isn't happening. The way I would expect.

Trying to find out how many file handles a Linux system can or cannot have hasn't been easy either. I've done quite a bit of Goolging, but there the same thing out there many times, but so far I've not discovered any deep magic.

For example on two nearly identical RedHat 7.x systems cat /proc/sys/fs/file-nr gives:

  • 8192, 3110, 16384
  • 1891, 1349, 8192
  • allocated, free, max file handles

It took me a long time to figure out that the middle number is the number of free, not the number of used file handles, and hence a small number is bad.

It's worse than that

Dom2 on 2003-02-28T14:19:57

Have a look at the code. I seem to recall that those numbers don't mean what you think that they mean...

From /usr/src/linux/Documentation/sysctl/fs.txt:

The kernel allocates file handles dynamically, but as yet it doesn't free them again. The value in file-max denotes the maximum number of file- handles that the Linux kernel will allocate. When you get lots of error messages about running out of file handles, you might want to increase this limit. The three values in file-nr denote the number of allocated file handles, the number of used file handles and the maximum number of file handles. When the allocated file handles come close to the maximum, but the number of actually used ones is far behind, you've encountered a peak in your usage of file handles and you don't need to increase the maximum.

-Dom

tempfile

Dom2 on 2003-02-28T14:37:11

I've found tempfile() easiest to work with when you don't specify any parameters. That way, you just get back a filehandle and a name. If you only want a filehandle to play with, it gets even easier as the autodelete should happen automatically when the $fh goes out of scope.

What happens if you put a close $fh in there before you leave the routine?

-Dom

Re:tempfile

ajt on 2003-02-28T14:59:37

Dom2,

Useful comments. I agree it's a great module, the problem started when I was working with it on NT and Linux. When I'd finished NT was working okay, but Linux was not....

If you have an explicit close $fh, then what I found was that the file was NOT deleted, and I can't remember if the file handle was. I've even tried an explicit unlink too.

Like I said I feel part of the problem is that it's running under mod_perl PerlRun, which is a bit of an odd place!

Since I posted, I'm now at: 8192, 2781, 16384 on the problem box. I had over 6000 free file handles this morning, before people came in to use the web server.

Re:tempfile

Dom2 on 2003-02-28T16:55:40

As I said in the first reply, I don't necessarily think those kernel figures are anything to worry about. But I would be concerned about the undeleted files.

The docs state that a file is not auto deleted, if a filehandle is requested as well as a filename. You have to unlink it yourself.

One more thing; You might wish to check whether or not the unlink fails: unlink( $fname ) or warn "unlink($fname): $!\n"

Oh, and to see whether filehandles are leaking in real, useful terms, the best thing to do is run "lsof -p" on the pid of the web server. That's more useful than the system-wide kernel measurements.

I suppose that at worst, you just write a cron job to clean them...

-Dom

Re:tempfile

ajt on 2003-03-01T12:32:02

Dom,

I think I know what is going on! I created the following little script, popped it into my PerlRun folder on my home Linux box and tested all the permutations.

#!/usr/bin/perl
$|++;

use strict;
use CGI;
use File::Temp qw(tempfile unlink0);

my $q = CGI->new;

print $q->header(-type => "text/plain");

print "PID: $$\n";
print `cat /proc/sys/fs/file-nr`;

for (1..50) {
    my ($TMP_FH, $tmp_file) = tempfile(SUFFIX => ".f", UNLINK => 0);
    print "Created: $tmp_file\t";
    print `cat /proc/sys/fs/file-nr`;
    unlink0 ($TMP_FH, $tmp_file);
    close $TMP_FH;
}

What I found was that with UNLINK=>1 on, my Linux box wasn't releasing file handles until the parent Apache process was terminated. I'd turned this on to get round files that were not being removed on NT (I thought at the time - though I may now be wrong).

With UNLINK=>0, if I do an explicit close of the filehandle, or let it go out of scope then I'm okay with file handles, but not files, they get left over.

Most recently I tried the unlink0 option that comes with File::Temp, it should be smarter and more secure than a plain unlink. With UNLINK=>1 it doesn't work, but with UNLINK=>0 it removes the files, and the file-handles are returned to the system.

My elderly RedHat 6 system didn't have lsof on it, so I went off and found it and built it - seems a useful tool.