Date: Mon, 24 Sep 2001 23:22:47 +1000
From: mgjv@tradingpost.com.au (Martien Verbruggen)
Subject: Re: eval-statement fools garbage-collection ?
Message-Id: <slrn9quct7.8ai.mgjv@martien.heliotrope.home>

On Mon, 24 Sep 2001 14:12:05 +0200,
	peter pilsl <pilsl_@goldfisch.at> wrote:
> 
> Martien Verbruggen wrote:
>>> So the memory is *not* freed at the undef-statement ..
>> 
>> You don't know that. When memory is freed, it is not necessary for the
>> process to hand it back to the OS immediately, or at all. All it means
>> is that it is available to the application for subsequent requests for
>> memory.
>> 
>> On many Unix systems it is possible for programs to return memory to the
>> OS while still running (this has not always been the case on most
>> Unices), but it can only do that under certain circumstances, and it
>> most of the time as very little control over whether or not the OS gets
>> the memory back.
>> 
> 
> thnx,
> 
> Thats an interesting point. However this makes things even worse. My 
> program runs in a mod_perl-environment and if your statement is true on my 
> system, that means, that every mod_perl-script using such structures is 
> memleaking, cause it never releases the memory back to the OS unless the 
> task terminates. 

It may not release the memory back to the OS, but the freed memory is
always available to the application itslef, if it needs new memory. It
just isn't available to other processes.

> And this is why I discovered my problem: apache-webserver seemed to be 
> memleaking when running my application on it.

It is possible that on your platform, your malloc(3) or sbrk(2) calls
are slightly broken. It is possible that the Perl you are running isn't
entirely kosher. As I showed, on my platform, it seems to work ok..

> I made subsequent tests and found out, that the memory is released to the 
> OS when I "manally" delete the structure.   (see below, dest($ptr) instead 
> of undef $ptr) So I think its a problem of perl (or better: a problem of my 
> way to use perl)

Nono. If this is the case, then I'd look at a problem in Perl. But
first, I'd make sure that perl is indeed not freeing the memory at all,
instead of just not giving it back to the OS. 

Do something like this in your test program:

my $ptr={};
$#{$ptr->{big}}=5000000;
def2($ptr);
print $ptr->{test}->{func2}(5),"\n";
print `ps -o vsize,rss,args -p $$ | tail -1`;
undef $ptr;
print `ps -o vsize,rss,args -p $$ | tail -1`;
# Make sure this allocation is smaller than the previous one
$#{$ptr->{big}}=2500000;
print `ps -o vsize,rss,args -p $$ | tail -1`;

If the memory has been correctly freed, then the memory used in the last
ps output should not be much more than the first ps output (ideally, it
should be less, but we've already established that that isn't
happening).

If you find that the memory has grown, by a significant factor, then you
do have a memory leak, and you need to investigate, maybe recompile Perl
with perl's malloc instead of the system (although I doubt it's that).

> I also extended my little script to loop around the allocate-part to see if 
> the mem is reclaimed to the OS when it needs it. At least on my linux2.4.10 
> (or 2.4.9) it does not, even if the thread gets bigger than 400M and 
> swapping makes my harddisks shake.
> The same on BSD3.4.

> perl on both machines is 5.6.0, and

Ok, I just tested with a perl 5.6.0 I have here, and it leaks like a
sieve, using the eval. It is a copy of Perl that was compiled with
threading support, while the 5.6.1 copy doesn't have threading support.
I do not have a 5.6.0 without threads, or a 5.6.1 with threads, but it
looks like the problem was either fixed between 5.6.0 and 5.6.1 or it is
related to thread support. If you don't have thread support, then you
probably need to upgrade. Otherwise, you probably need to get rid of
thread support.

Oh, what the hell. I'll compile a 5.6.1 with threads, one minute.
Actually, more than one minute...

Nope. No memory leak with perl 5.6.1 with threads enabled.

While reading the Changes file, I found this:

[  8894] By: gsar                                  on 2001/02/22 19:06:18
        Log: integrate changes#6162,6163 from mainline (missing leak fixes!)
             
             fix memory leak in method call optimization (change#3768);
             made C<eval "$x->foo()"> leak
             
             fix memory leak in C<eval "BEGIN {}"> (bug in change#4579)
     Branch: maint-5.6/perl
          !> op.c

Maybe that's the one... 5.6.1 was released on 2001-Apr-08.

> is the same here. Maybe I should have a look at perl 5.6.1.

That sounds like a good option to me.

Martien
-- 
Martien Verbruggen              | 
Interactive Media Division      | In the fight between you and the
Commercial Dynamics Pty. Ltd.   | world, back the world - Franz Kafka
NSW, Australia                  |