The function rtf_parse_keyword, at line 502 of src/protocols/novell/nmrtf.c, defines a 30-byte keyword array, which it writes to through a pointer pch. How can an attacker arrange for pch to move past the end of the array? Gaim reads data from the network (i.e., from the attacker) with the UNIX read() function; what does that data have to be to control pch? Your assignment, due 2004.09.03, is to identify and understand the relevant lines of the Gaim program.
You will have to look at files other than nmrtf.c. I have all the source files online, but I recommend that you download gaim-0.81.tar.gz onto a UNIX machine and unpack it:
gunzip < gaim-0.81.tar.gz | tar -xf -
(Warning: you need about 32MB of disk space free.)
Then you can look at the files on that machine.
If you want to know (for example)
where nm_rtf_strip_formatting appears inside Gaim,
and in particular which functions call the
nm_rtf_strip_formatting
function,
you can type
find gaim-0.81 -type f | xargs grep -n nm_rtf_strip_formatting /dev/null
and then use your favorite editor (maybe vi or emacs)
to look at the files identified by grep.