[Laszlo-dev] instructions for running performance analysis tests

[Benjamin Shine]

To gather the data for performance analysis:
On each architecture of interest:
In $LPS_HOME/test/lfc/perf/build.xml edit these lines to indicate  
which browsers, branches, test suites, and runtimes you want to test:
     var suites = ["sprite", "functions", "viewperf",  
"imageperformance", "textperformance"];
     var runtimes = ["swf8", "dhtml"]; // swf7 numbers are boring,  
almost identical to swf8 numbers
     var browsers = ["Firefox"];
     var webapps = ["trunk"];

Then invoke ant to drive the tests... leave your machine alone for  
around 10 minutes while it runs the test.
$ cd $LPS_HOME/test/lfc/pref
$ ant -Dbranch=trunk hitallbrowsers storelog

This will result in a lot of log data being stored in svn -- the  
latest one (which I just stored) is here:
http://svn.openlaszlo.org/QA/performance/raw/7656-openlaszlo-trunk.txt
Each of those lines represents the results from one test (executed  
several times) in each suite. That's the raw data.

......

The tool for analyzing the raw data is in $LZ_TOOLS_ROOT/performance/ 
cruncher. It is a java program. To compile it, see instructions in
http://svn.openlaszlo.org/tools/trunk/performance/README.txt

cd $LZ_TOOLS_ROOT/performance/cruncher
ant -Dbuild.id=7656-openlaszlo-trunk go

performance/cruncher/build.xml fetches the raw performance logs for  
the specified build from subversion, runs the java performance  
analysis tool on it, and mails the results to a few interested  
parties. "ant go" is all you need.

The cruncher analyzes whatever data is in the checked-in raw log; it  
expects to analyze data recorded by test/lfc/perf/reporter.lzx via  
test/lfc/perf/reporter.jsp. The reporting itself does not need to  
happen on the same machine as the analysis.

Standard usage is from within the laszlosystems firewall (so we can  
send mail) and as a logged-in svn user with commit access to the  
repository. Then just
$ ant go

To run analysis on a particular build, pass in
-Dbuild.id=thebuildid for example
$ ant -Dbuild.id=2469-openlaszlo-branches-legals go

To prevent any interaction with subversion at all, pass in
-Dno.svn=true for example
$ ant -Dno.svn go

If you've already got the data locally and you just want to analyze  
it, with no network communication, do this
$ ant -v -Dno.svn=true -Dbuild.id=2469-openlaszlo-branches-legals  
analyze
$ more localanalysis/2469-openlaszlo-branches-legals-analysis.txt

Use the "clean" target to tidy up all local files. This is, of  
course, destructive.