PROOF

The proofcluster.py script can be used to configure, start and stop a PROOF cluster on the batch system. The configuration is stored between sessions, so there is no need to configure again if the same configuration should be used again.

The following lines are an example workflow:

~samson1/proofcluster/proofcluster.pl config

~samson1/proofcluster/proofcluster.pl start
~samson1/proofcluster/proofcluster.pl status

# do something with PROOF

~samson1/proofcluster/proofcluster.pl stop

SFrame

SFrame can be run locally, as PROOF Lite and full PROOF.

PROOF Lite

For testing purpose or short runs, you can run on the work group servers. For longer runs please use the batch system via qrsh using the multicore parallel environment. The following two examples should give you an idea how to do this.

Example 1: Request a session on the batch system, setup and run SFrame:

# get session on batch 
qrsh -pe multicore 8 -now n -R y -l site=hh -l h_cpu=01:00:0 -l h_vmem=1G

# setup SFrame
cd atlas/SFrame
ini pyroot526
source setup.sh

cd user/run
sframe_main ../config/FirstCycle.xml

You can run another sframe_main in the same session.

Example 2: Setup SFrame and run it directly on the batch system

cd atlas/SFrame
ini pyroot526
source setup.sh

cd user/run
qrsh -pe multicore 8 -now n -R y -l site=hh -l h_cpu=01:00:0 -l h_vmem=1G -V -cwd sframe_main ../config/FirstCycle.xml

When SFrame ends, you are back on the work group server.

Some comments on the qsub options:

• The option -R y tells SGE to reserve slots, even if not all will be directly available.
• As always for batch jobs you need to specify the needed CPU time and memory. Memory limits are given per core.
• You should specify a site using -l site=hh or -l site=zn, if you know where your data is. For Lustre the directory name contains the site, e. g. /scratch/hh. For dCache the DDM site name contains the site, e.g. DESY-ZN_LOCALGROUPDISK. If you do not care or do not know about the site, leave this option out.
• The option -V will import the full shell environment to the batch session.
• If option -cwd given the batch session will change to the current working directory. Please note that this needs to be either in AFS or /scratch.

PROOF

You can also run SFrame in full PROOF mode and get many more cores than in the PROOF Lite mode. Set up a PROOF cluster as described above. Add the PROOF master (hostname:port) to the cycle XML file under ProofServer. Then run SFrame as in local mode. e. g.:

cd atlas/SFrame
ini pyroot526
source setup.sh

~samson1/proofcluster/proofcluster.pl start
~samson1/proofcluster/proofcluster.pl status

cd user/run
sframe_main ../config/FirstCycle.xml

~samson1/proofcluster.pl stop

Do not forget to stop your proof cluster, when done.