Date: Mon, 24 Sep 2001 14:12:05 +0200
From: peter pilsl <pilsl_@goldfisch.at>
Subject: Re: eval-statement fools garbage-collection ?
Message-Id: <3baf2319@e-post.inode.at>


Martien Verbruggen wrote:
>> So the memory is *not* freed at the undef-statement ..
> 
> You don't know that. When memory is freed, it is not necessary for the
> process to hand it back to the OS immediately, or at all. All it means
> is that it is available to the application for subsequent requests for
> memory.
> 
> On many Unix systems it is possible for programs to return memory to the
> OS while still running (this has not always been the case on most
> Unices), but it can only do that under certain circumstances, and it
> most of the time as very little control over whether or not the OS gets
> the memory back.
> 

thnx,

Thats an interesting point. However this makes things even worse. My 
program runs in a mod_perl-environment and if your statement is true on my 
system, that means, that every mod_perl-script using such structures is 
memleaking, cause it never releases the memory back to the OS unless the 
task terminates. 
And this is why I discovered my problem: apache-webserver seemed to be 
memleaking when running my application on it.

I made subsequent tests and found out, that the memory is released to the 
OS when I "manally" delete the structure.   (see below, dest($ptr) instead 
of undef $ptr) So I think its a problem of perl (or better: a problem of my 
way to use perl)

I also extended my little script to loop around the allocate-part to see if 
the mem is reclaimed to the OS when it needs it. At least on my linux2.4.10 
(or 2.4.9) it does not, even if the thread gets bigger than 400M and 
swapping makes my harddisks shake.
The same on BSD3.4.

perl on both machines is 5.6.0, and

> 
> $ perl -V | grep malloc
>     alignbytes=4, usemymalloc=n, prototype=define
>                   ^^^^^^^^^^^^^

is the same here. Maybe I should have a look at perl 5.6.1.

here my extended script, including a loop-feature and a (very dumb) 
manually destruction-routine.

using dest instead of undef I get the expected results. The mem is released 
immediately when using the eval-statement.
using undef I get in big troubles with heavy swap ...

case1: loop 0..20, use dest and eval
<skip first 20 loops>
===============
 1928 1452
9
21460 21000
 1928 1452

case 2: loop 0..20,  use undef and eval
<skip first 19 loops>
===============
373976 173480
9
393512 193016
393512 193016
===============
393512 193016
9
413044 212548
413044 212548


the script:

#!/usr/bin/perl -w
use strict;

foreach (0..2) {   # extend to 20 later to get the full 400MB ;)
  allocate();
}

sub allocate
{
  print "===============\n";
  print `ps -o vsize,rss -p $$ | tail -1`;
  my $ptr={};
  $ptr->{big}=[];
  $#{$ptr->{big}}=5000000;
  def2($ptr);
  print &{$ptr->{test}->{func2}}(5),"\n";
  print `ps -o vsize,rss -p $$ | tail -1`;
#  undef $ptr;        # let perl do the job
  dest($ptr);      # let the script to the job
  print `ps -o vsize,rss -p $$ | tail -1`;
}

sub def2 {
  my $ptr=shift;
  my $cmd='$ptr->{test}->{func2}=sub{return $_[0]+4};';
#  $ptr->{test}->{func2}=sub{return $_[0]+4};
  eval $cmd;
}

sub dest {
  my $ptr=shift;
  my $ptype=ref($ptr);
  if ($ptype) {
    if ($ptype eq 'ARRAY') {
#  uncomment only if arrayelements can be references again
#      foreach (@{$ptr}) {
#        dest($_);
#      }
      @{$ptr}=();
    };
    if ($ptype eq 'HASH') {
      foreach (keys %{$ptr}){
        dest($ptr->{$_});
        delete($ptr->{$_});
      }
      %{$ptr}=();   
    }
  }
}
  




-- 
peter pilsl
pilsl_@goldfisch.at
http://www.goldfisch.at