#acl SaschaMehlhase:read,write,admin All:read
{{{
#!sidebar
== Quicklinks ==
 * [[http://isscvs.cern.ch/cgi-bin/cvsweb.cgi/Pixel/mehlhase/?cvsroot=atdesyz#dirlist|CVS at CERN]]
 * [[http://cern.ch/analysepixeldcs|Example plots]]
}}}
<<TableOfContents(3)>>
----
<!> There are some major changes from 090702-1010. The command line parameters changed <!>

<!> There are some major changes between 090421-1510 and the current head. See [[http://isscvs.cern.ch/cgi-bin/cvsweb.cgi/Pixel/mehlhase/analysePixelDCS/ChangeLog?rev=HEAD;content-type=text%2Fplain;cvsroot=atdesyz|ChangeLog]]! <!>
----
I started working on tools that allow for advanced offline monitoring of the Detector Control System (DCS) and Data Quality (DQ) variables.

All software explained below can be found in [[http://isscvs.cern.ch/cgi-bin/cvsweb.cgi/Pixel/mehlhase/analysePixelDCS/?cvsroot=atdesyz#dirlist|CVS at CERN]].

This is work in progress, so if you check out the head version be aware of that fact! Cheers

== People involved ==
 * [[SaschaMehlhase|Sascha Mehlhase]]

== Analyses carried out using these tools ==
 * [[http://atlas-project-pixel-analysepixeldcs.web.cern.ch/atlas-project-pixel-analysepixeldcs/TOptoAndIPin/|Topto/Ipin analyses (Sep-Dec 2008)]]
 * [[http://atlas-project-pixel-analysepixeldcs.web.cern.ch/atlas-project-pixel-analysepixeldcs/BackpressureTest/|Backpressure test (Dec 7th 2008)]]
 * [[http://atlas-project-pixel-analysepixeldcs.web.cern.ch/atlas-project-pixel-analysepixeldcs/Bakeout|Bakeout test (Sep 2008)]]
 * [[http://atlas-project-pixel-analysepixeldcs.web.cern.ch/atlas-project-pixel-analysepixeldcs/Bakeout-ThermalTests|Thermal tests (Sep 2008)]]
 * [[http://atlas-project-pixel-analysepixeldcs.web.cern.ch/atlas-project-pixel-analysepixeldcs/Cosmics-Sep-Dec-08/|Offline DQ Monitoring]]

== dumpDCS ==
dumpDCS reads a user defined collection of DCS variables from the ATLR database and puts them into a ROOT file.
Each given DPE alias is put into a separate branch of a tree that also contains a timestamp.

For now the code scans all systems in pixels and IDE databases. This will be extended to SCT and TRT soon.

=== Get the code and get it running ===
set up you CVS environment by issuing e.g.

{{{
export CVSROOT=:ext:isscvs.cern.ch:/local/reps/atdesyz
export CVS_RSH=ssh
}}}
check out the package by typing

{{{
cvs co Pixel/mehlhase/dumpDCS
}}}
set up your ROOT environment, here shown for LXPLUS

{{{
export ROOTSYS=/afs/cern.ch/sw/lcg/external/root/5.20.00/slc4_amd64_gcc34/root
export LD_LIBRARY_PATH=$ROOTSYS/lib:$LD_LIBRARY_PATH
export PATH=$ROOTSYS/bin:$PATH
}}}

{{{
make
}}}
and off you go

=== Usage ===

fire up a ROOT session and open one of the output files, let's say you created the file issuing
{{{
./dumpDCS -d D3C_B01_S._TOpto
}}}
then you can do e.g.
{{{
dumpDCS->Draw("D3C_B01_S1_TOpto:timestamp")
dumpDCS->Draw("D3C_B01_S1_TOpto:D3C_B01_S2_TOpto")
}}}

== analysePixelDCS ==
analysePixelDCS reads a collection of DCS variables from the ATLR database and puts them into a object based ROOT file.

To get an understanding of the variable names read from the data base there is a TWiki page on that topic:<<BR>> https://twiki.cern.ch/twiki/bin/view/Atlas/PixelDCSCondDB

=== Get the code and get it running ===
set up you CVS environment by issuing e.g.

{{{
export CVSROOT=:ext:isscvs.cern.ch:/local/reps/atdesyz
export CVS_RSH=ssh
}}}
check out the package by typing

{{{
cvs co Pixel/mehlhase/analysePixelDCS
}}}
set up your ROOT environment, here shown for LXPLUS

{{{
export ROOTSYS=/afs/cern.ch/sw/lcg/external/root/5.20.00/slc4_amd64_gcc34/root
export LD_LIBRARY_PATH=$ROOTSYS/lib:$LD_LIBRARY_PATH
export PATH=$ROOTSYS/bin:$PATH
}}}
open up ROOT and create the shared library (you have to use the same version as for compiling the code itself), in ROOT type

{{{
.L PixelDCSclasses.h++
}}}
at last leave ROOT again and create a symbolic link to the class library to make the compilation/linking a bit easier

{{{
ln -s PixelDCSclasses_h.so libPixelDCSclasses.so
}}}
issue

{{{
make
}}}
and off you go

=== Usage ===
{{{
./analysePixelDCS -i 300 -s 20081019:000000 -e 20081019:235959 -o 081019_05min.root -env -d PIX/ENV/Side./Oct./PP1/NTC+/Temp
}}}
 * {{{-i}}} followed by a number gives the time interval that will be used in the file
 * {{{-s}}} followed by a time stamp pattern (YYYYMMDD:hhmmss) sets the stating time
 * {{{-e}}} followed by a time stamp pattern (YYYYMMDD:hhmmss) sets the end time
 * {{{-sGMT}}} followed by a time stamp pattern (YYYYMMDD:hhmmss) sets the stating time
 * {{{-eGMT}}} followed by a time stamp pattern (YYYYMMDD:hhmmss) sets the end time
 * {{{-o}}} followed by a string sets the output filename
 * {{{-env}}} to add some extra data points on the environment
 * {{{-d}}} followed by a string (reg expr) to dump specific data points in the file<<BR>>they can be accessed by name later on, e.g. {{{Dump.GetDump("PIX/ENV/SideC/Oct7/PP1/NTC_PP1_R/Temp")}}}

This would create a roughly 8 Mbyte ROOT file containing the data from October 19th 2008 with intervals of 5 minutes in about 22 minutes at LXPLUS.

Several ROOT files created with this software can be merged using the {{{hadd}}} command that comes along with ROOT.

The matching between PP0s and various other information is done by a [[http://spreadsheets.google.com/ccc?key=pm8famaLk6XlhkjdTLL0KGA|look-up table that can be found here]] and in the CVS repository.

{{{
hadd targetfile source1 [source2 source3 ...]
}}}
=== 2Do List ===
 * !! automatically read list of good runs from data base
 * ! improve the way run numbers are stored for timestamps (maybe do no consider last interval of run, cause it might include the reason for the end of the run)
 * ! get ROD matching done (connectivity database, extend the analysePixelDCSlookup table accordingly)
 * switch from ROOT to Oracle API routines (don't see the full need yet though)

=== Batch usage ===
to run a mass production of ROOT files you can set up scripts to use analysePixelDCS on e.g. the LXPLUS batch farm

there are now two new files ({{{analysePixelDCSbatch.sh}}} and {{{submitBatch.sh}}}) that you can get inspiration from

'''use this very careful, as you might stress the database a bit too much'''

=== ROOT usage ===
to look at the ROOT files generated by this software you need to include the shared library by issuing the following command within ROOT

{{{
gSystem.Load("PixelDCSclasses_h.so")
}}}
then you can draw the object using e.g. to draw the module temperature of module L2_B20_S1_C7_M4C over time

{{{
PixelDCS->Draw("Tmod[][4]:timestamp","Tmod[][4]!=-88. && location==012201")
}}}
where {{{location}}} is encoded in the following way

{{{
ABCDDE
||| ||
||| |stave (1/2)
||| bistave (1-4, 1-11, 1-19, 1-26)
||layer (1-3, 0-2)
|side (0/1)
barrel/ec (0/1)
}}}
and  {{{position}}} as given here

{{{
ABCDEF
|||||top/bottom (0, 1)
||||octant (1-8)
|||quadrant (1-4)
||pipe (0-old, 1-new, 2-reworked)
|inlet/outlet
institue (1-Genova, 2-Marseille, 3-Wuppertal, 0-others)
}}}
=== Variables ===
A value of -88 is filled in case the variable hasn't been measure at least once in the interval.

All values that might be measured multiple times with different values in one interval have the RMS for each interval stored in a variable which name is equal but with an additional 'RMS' at the end. {{{PP0s.Tave}}} -> {{{PP0s.TaveRMS}}}

<<latex($RMS_{x}=\sqrt{\frac{\sum_{i}^{N}{(x_{i}-\bar{x})^2}}{N}}=\sqrt{\frac{\sum_{i}^{N}{x_{i}^{2}}}{N}-\left(\frac{\sum_{i}^{N}{x_{i}}}{N}\right)^2}$)>>

|| '''Name''' || '''Content/Info''' || '''Switch for optional values''' ||
|| timestamp || UNIX timestamp (GMT) || ||
|| runnumber || vector of runnumbers in that interval || ||
|| PP0s.location || encoded location as explained above || ||
|| PP0s.position || encoded position as explained above || ||
|| PP0s.coolloop || number of the connected cooling loop || ||
|| PP0s.rod || number of the connected ROD crate/slot || ||
|| PP0s.Tave || average module temperature for this stave || ||
|| PP0s.Tmax || maximum module temperature for this stave || ||
|| PP0s.Topto || temperature of the optoboard || ||
|| PP0s.Popto || power of the optoboard || ||
|| PP0s.VVDC_I || || ||
|| PP0s.VVDC_V || || ||
|| PP0s.VVDC2_I || Vmon current || ||
|| PP0s.VVDC2_V || Vmon voltage || ||
|| PP0s.VPin_I || pin diode current || ||
|| PP0s.VPin_V || pin diode voltage || ||
|| PP0s.VISet_I || || ||
|| PP0s.VISet_V || || ||
|| PP0s.HV_I || high voltage current || ||
|| PP0s.HV_V || high voltage || ||
|| PP0s.Tmod || vector of module temperatures || ||
|| PP0s.Pmod || vector of module powers || ||
|| PP0s.VDD1_I || digital module current || ||
|| PP0s.VDD1_V || digital module voltage || ||
|| PP0s.VDDA1_I || analog module current || ||
|| PP0s.VDDA1_V || analog module voltage || ||
|| ENV.Texh(loopnumber) || exhaust temperature of the connected cooling loop || -env ||
|| ENV.Tsup(loopnumber) || supply temperature of the connected cooling loop || -env ||
|| ENV.Tsur(loopnumber) || heater surface temperature || -env ||
|| ENV.Texl(loopnumber) || exhaust liquid temperature || -env ||
|| ENV.!ToptoBotAve(side, octant) || e.g. A1BOT_Opto_Tave || -env ||
|| ENV.!ToptoBotMax(side, octant) || e.g. A1BOT_Opto_Tmax || -env ||
|| ENV.!ToptoTopAve(side, octant) || e.g. A1TOP_Opto_Tave || -env ||
|| ENV.!ToptoTopMax(side, octant) || e.g. A1TOP_Opto_Tmax || -env ||
|| ENV.!ToptoIspAve(side, octant) || e.g. A1ISP_Opto_Tave || -env ||
|| ENV.!ToptoIspMax(side, octant) || e.g. A1ISP_Opto_Tmax || -env ||
|| ENV.!TbundleIsp(side, octant) || e.g. PIX/ENV/SideA/Oct1/INT/ISP_BUNDLE_R/Temp || -env ||
|| ENV.!TbundleOsp(side, octant) || e.g. PIX/ENV/SideA/Oct1/INT/OSP_BUNDLE_R/Temp || -env ||
|| ENV.Tnose(side, quadrant) || e.g. IDE/ENV/PixelNose/SideA/Q1/Temp<<BR>>quadrant=5 is IDE/ENV/PixelNose/SideA/Bottom/Temp<<BR>>quadrant=6 is IDE/ENV/PixelNose/SideA/Top/Temp<<BR>>quadrant=7 is IDE/ENV/PixelNose/SideA/Outside/Temp || -env ||
|| Dump.!GetDump("some_dpe_name") || returns the value of that dpe, does not work with TTree::Draw || -dsome_dpe_name ||

== plotPixelDCS ==
Please understand this code only as an example!

plotPixelDCS reads the ROOT files generated by analysePixelDCS and generates some general purpose plots.

The usage is explained in the following ...

Create some symbolic links to the main directory of analysePixelDCS

{{{
ln -s ../PixelDCSclasses.h .
ln -s ../PixelDCSclasses_h.so .
ln -s ../libPixelDCSclasses.so .
}}}
build it by issuing

{{{
make
}}}
and run by typing

{{{
./plotPixelDCS inputFilename.root
}}}
== loopPixelDCS ==
Please understand this code only as an example!

loopPixelDCS reads the ROOT files generated by analysePixelDCS and generates some general purpose plots.

in difference to plotPixelDCS this tool loops over the file on an event (timestamp) by event basis which allows for more advanced analyses with more complex cuts etc ...

Create some symbolic links to the main directory of analysePixelDCS

{{{
ln -s ../PixelDCSclasses.h .
ln -s ../PixelDCSclasses_h.so .
ln -s ../libPixelDCSclasses.so .
}}}
build it by issuing

{{{
make
}}}
and run by typing

{{{
./loopPixelDCS inputFilename.root
}}}
some examplary plots created with this tool can be found here http://cern.ch/analysepixeldcs

the webpage presenting the plots is also created by the script and offers features like error highlighting and tooltips ;)