Virenk's Blog

Archive for March 2011

A script to measure peak memory usage of a process on Linux shell.

Source : http://stackoverflow.com/questions/1080461/peak-memory-measurement-of-long-running-process-in-linux

#!/bin/bash

ppid=$$
maxmem=0

$@ &
pid=`pgrep -P ${ppid} -n -f $1` # $! may work here but not later
while [[ ${pid} -ne "" ]]; do
    #mem=`ps v | grep "^[ ]*${pid}" | awk '{print $8}'`
        #the previous does not work with MPI
        mem=`cat /proc/${pid}/status | grep VmRSS | awk '{print $2}'`
    if [[ ${mem} -gt ${maxmem} ]]; then
    	maxmem=${mem}
    fi
    sleep 1
    savedpid=${pid}
    pid=`pgrep -P ${ppid} -n -f $1`
done
wait ${savedpid} # don't wait, job is finished
exitstatus=$?   # catch the exit status of wait, the same of $@
echo -e "Memory usage for $@ is: ${maxmem} KB. Exit status: ${exitstatus}\n"

Recently i have been trying to read a large text file (1.8 G) in tcl using following code:

set fn [open “filename.txt” r];

set data [read $fn];

This works well for file sizes < 1 G. but crashes with larger files with core dump.

Another better approach recommended is to use command “file size” along with “read” so as to pre-allocate memory for required buffer. This should be faster than above approach:

set fn [open “filename.txt” r];

set data [read $fn [file size “$filename.txt”] ];

But this still does not read files > 1G.

Solution:

Instead of reading the entire file into buffer, read line by line:

set filename “myfile.xml”
set f [open $filename]
while {[gets $f  line] >= 0} {
# work with $line here …
}
close $f

Ref: http://en.wikibooks.org/wiki/Tcl_Programming/Working_with_files

Tags: , , ,

Debugging a crash point in a program is usually a very tedious job. Life gets even more difficult when you have to wait for hours to reach the crash point. And u fix something and again test if it worked. Imagine trying to debug a test case running for 100 hours! That happen quite often in routing/placement tools in EDA (Electronic Design Automation) domain.

It would be good to have a debugging tool that can help me in restarting my program just before the crash point (call it critical point) (a point where things were working fine).

Here i am assuming  that i have not changed anything in  functions/data objects  in my code, that were used in reaching the critical point of my choice.  Changes are permissible only  in functions/data called after the critical point.

We need to capture the entire state of the program for such a thing to work.  The state would include stack, opened files, files acessed/read/written to disk, network files and so on.  It would be interesting to see explore the possibility of this task. I need to do some R&D to here :P.

To my knowledge there in no tool as of now which can do that.  Imagine how much hours of debugging time will be saved if we had such a tool!



  • None

Categories