DAQ system for BCM1F Test Beam Jan. 2014 at DESY-Hamburg
Contents
Schematic of DAQ Elements
Introduction
The TLU (Trigger Logic Unit) generates triggers for data readout. During running, it takes coincidences of hits in the scintillators to trigger. In test mode it generates triggers itself. Triggers are sent to the telescope and to the DUT (Detector Under Test); in our case the "DUT" is the V1495_TLU interface board programmed by Hans. The interface board is necessary to conduct the TLU's most complex handshake mode, which in addition to the trigger sends the TLU event number. This event number is sent to the DUT and telescope and used to synchronize events between the two systems (instead of using a timestamp). The V1495_TLU stores the event number in a register until it is read out (e.g. by a DAQ program, or Serivas). It also passes a simple trigger signal on to the ADC to trigger the ADC readout.
The V1495_TLU has a veto setting in which no more triggers can be sent until the current trigger number is read out. This veto setting will not work with the TLU's test mode; this is the only (known) functional difference between the TLU's running and test modes.
DAQ for Test Beam
There are two (mostly-independent) DAQ systems: the ADC system used to take the sensor data, and the EUDAQ telescope system to reconstruct the tracks that pass through the sensor. The only connection is that both receive the same event number from the TLU.
Useful Information
We are in Test Beam Area 21, Building 27 (aka Hall 2). Telephone number: (+49 40 8998) 3721
IP addresses |
|
lccmstmp |
131.169.61.81 |
cms-alibava |
131.169.61.78 |
The two-pad diamond is ADC channels 1 and 2, the single-pad diamond is ADC channel 3.
Team member telephone numbers are posted above the desk.
The telescope support number is (0+) 0151-72713427. The phone number of the beam area is 4221. This (and other) information is posted on the whiteboard in the counting room.
Starting Up the DAQ Systems
On the counting room telescope machine (atlaslab01, but with "zenpixell2" monitor), go to the remote desktop. (If not already open, use Remote Desktop to login to 192.168.1.2 with username and pw posted in the counting room.) This is the Windows machine that controls the telescope.
Set the pixel thresholds of the telescope planes:
- Open M126.exe on (Windows) desktop
- Go to Open
- Open threshold7 file/setting
- Click "All" (towards the bottom right)
- Click "Read"
- Click "Start"
Start the telescope:
- Double-click the anemone_v1.2.vi icon on the remote desktop (Windows machine)
- In window that opens, check that all error markers (down left side) are black
- If not, restart or call telescope support
Run it by pushing the white arrow (Labview) --> should be already running
Start the EUDAQ software:
Double-click startGUIEudaq.bat - Shortcut on the Windows machine desktop. This will open the EUDAQ window.
- Make sure everything in the status window is OK (green).
Choose config setting ni_coins_FCAL then press "Config".
Prepare the ADC DAQ window:
- Open a terminal (icon in left bar)
Do ssh -X daq@131.169.61.81 (this is lccmstmp)
Go to /home/daq/TMP/TB_HH_2014_Jan/DAQ_v1721+1495/
To Do Each Run
1. Start a Run
In the terminal window, start the ADC with ./runDAQ.sh start
You can check the status of the process with: ./runDAQ.sh status (not necessary)
Check data-taking progress and the number of ADC triggers with: tail -f logs/Log_[Date]_[Time].log
- On the Windows machine, start the EUDAQ by clicking on "Start" button in EUDAQ window
- Lines in status window should turn green, Events Built and Triggers should start counting
- The number of triggers in the ADC log file should also count up by 1000
- Keep an eye on the EUDAQ Log Collector window. If there is a message about an unexpected event number, record the run and event number.
!!! It is important that the ADC DAQ is started before the EUDAQ/TLU, because the TLU hangs otherwise !!!
Record in the logbook:
- Time of run start
- Run number (from EUDAQ window)
- Trigger rate (from EUDAQ window)
- ADC ROOT data filename
2. End the Run
Stop the tail -f process in the terminal window with Ctrl+C
Stop ADC readout with ./runDAQ.sh stop
- Stop EUDAQ by clicking on the "Stop" button in the EUDAQ window
Things to check:
- Make sure that the final number of triggers in ADC DAQ and EUDAQ are the same:
Check number of triggers at end of ADC log file (less Log_[Date]_[Time].log)
- Check number of triggers in the EUDAQ window
Record in the logbook:
- Number of Events Built
- Number of Triggers (EUDAQ window)
- Time of run stop (at end of ADC log file if you forget to look at the clock)
- Any relevant quick data checks (see next section)
Checking the Data
In the EUDAQ window check that #particles > #triggers
With ROOT open the ADC file (located in /data/TB_HH_2014_Jan/ADC_data_diamond/)
- Open new TBrowser
Check spectrum_ch3 histogram (for single-pad diamond. For two-pad diamond, look in channels 1 and 2.)
- You should see two well-separated peaks (Pedestal and Signal), peak positions should be separated by ~100 a.u.
- Optional: look at the raw data to see the signal peak
In ROOT type .L ~/TMP/TB_HH_2014_Jan/DAQ_v1721+1495/show_Event.cxx
Type show_Event("/path/to/data/file.root",X+N*Y);
- X = channel number, Y = event number, N = number of ADC channels enabled (5)
- In some events there should be signals visible
For X-Y position scan
In ADC ROOT file look at amplitude_ch3 (for single-pad diamond. For two-pad diamond, look in channels 1 and 2.)
- Write down the ratio of signal peak height to pedestal peak height (the scan is optimizing this)
For high voltage scan:
In ADC ROOT file look at amplitude_ch3 (for single-pad diamond. For two-pad diamond, look in channels 1 and 2.)
- Write down the mean signal amplitude (zoom in on the signal peak and take the mean from the statistics box)
Diamond Calibration Validation
SHOULD BE DONE ONCE EACH SHIFT
- Switch on the generator CH2 (voltage of 50mV is set, DO NOT CHANGE IT!!!)
(Stop run and) kill EUDAQ with killEUDAQ.bat - Shortcut desktop shortcut (TLU can't be controlled both by EUDAQ and TLUControl.exe).
Go to the VIM window with the file configure.sh
Change the POST_TRIGGER_NUMBER value to 60
Save configure.sh
- Start test run
In the Linux terminal do ./runDAQ.sh start
In the Windows PowerShell do ./TLUControl.exe -t 5 -d 1 -i RJ45 -hm 2 -b 1
If no PowerShell open, open one and go to C:\eudaq\Windows Binaries then execute above command
- Run for 1000 events
- The TLUControl.exe output and the ADC output show very different numbers of events taken, we are not yet sure why. The ADC output is accurate in terms of events in the root file.
In the Linux terminal do ./runDAQ.sh stop
In the PowerShell window, kill TLUControl.exe with Ctrl-C.
Check the ADC file in ROOT (located in /data/TB_HH_2014_Jan/ADC_data_diamond/)
Make sure the mean of amplitude_ch3 is 12.6 ADC counts (for single-pad diamond. For two-pad diamond look in channels 1 and 2)
In configure.sh change POST_TRIGGER_NUMBER back to 40
Save configure.sh
- Remember: Only do it for 50mV!
- Turn off the channel 2 output on the generator. Do not change something on the generator, only switch it off!!!!
Restart EUDAQ from startGUIEudaq.bat - Shortcut on Windows desktop.
Other Things You Might Need To Do
1. Change the HV on the Diamond
The HV box is on the NIM crate to the left of the VME crates, next to the telescope assembly. The important number on the display is in the lower-left corner, showing the current voltage in kV. To change it, select which digit you want to change with the right/left arrow buttons (towards the right side of the box). Then change the digit with the up/down arrow buttons just above the right/left arrows. Remember that the display is in kV, so 10 V = 0.01 kV. IMPORTANT: The voltage should ONLY be changed in increments of 10 V = 0.01 kV.
2. Change the x-y Position of the Diamond
Double-click the PIMikroMove icon on the desktop of the remote desktop (Windows machine). There will be two lines in the lower section of the window, one for the X direction and one for the Y direction. Change the step size if necessary (e.g. 1.) and then use the single right and left arrows to the side of the "Target" column entries to change the position in each direction to the desired value.
Data Storage
The ADC and telescope data files are being backed up in three places.
cms-alibava (ubuntu)
1-TB external hard disk (WD MyPassport, FAT format)
Zeuthen CMS group AFS space: /afs/ifh.de/group/cms/TestBeam/HH2013/HH2014
We will use rsync to keep the contents in each location updated.
ADC data
- From lccmstmp to cms-alibava
(On cms-alibava) rsync -avzr daq@131.169.61.81:/data/TB_HH_2014_Jan/ADC_data_diamond/ /home/daq/Data/TB_HH_2014_Jan/ADC_data_diamond/
(On cms-alibava) rsync -avzr daq@131.169.61.81:/data/TB_HH_2014_Jan/ADC_data_saphire/ /home/daq/Data/TB_HH_2014_Jan/ADC_data_saphire/
- From cms-alibava to hard disk
- Plug hard drive into cms-alibava
Will be mounted at /media/E613-88AD/
(On cms-alibava) rsync -avzr /home/daq/Data/TB_HH_2014_Jan/ADC_data_diamond/ /media/E613-88AD/ADC_data_diamond/
(On cms-alibava) rsync -avzr /home/daq/Data/TB_HH_2014_Jan/ADC_data_saphire/ /media/E613-88AD/ADC_data_saphire/
umount /media/E618-88AD/
- Unplug hard drive
- Plug hard drive into cms-alibava
- From cms-alibava to AFS space
- If you have write access in AFS storage area:
rsync -avzr /home/daq/Data/TB_HH_2014_Jan/ADC_data_diamond/ <username>@pub.ifh.de:/afs/ifh.de/group/cms/TestBeam/HH2013/HH2014/ADC_data_diamond/
rsync -avzr /home/daq/Data/TB_HH_2014_Jan/ADC_data_saphire/ <username>@pub.ifh.de:/afs/ifh.de/group/cms/TestBeam/HH2013/HH2014/ADC_data_saphire/
- Otherwise ask Olena/Maria/Jessica
- If you have write access in AFS storage area:
Telescope data
- From telescope computer (tb-datura) to hard disk
- Plug hard drive into atlaslab01 (the computer with the "ZENPIXELL2" monitor)
Will be mounted at /media/E613-88AD/
(On atlaslab01) rsync -azvr telescope@131.169.37.240:/mnt/datura_raid/data/2014w2/ /media/E613-88AD/Telescope_data_diamond/
copy telescope files to /media/E613-88AD/Telescope_data_saphire/ above run number 469
umount /media/E613-88AD/
- Unplug hard drive
- Plug hard drive into atlaslab01 (the computer with the "ZENPIXELL2" monitor)
- From hard disk to cms-alibava
- Plug hard drive into cms-alibava
Will be mounted at /media/E613-88AD/
(On cms-alibava) rsync -azvr /media/E613-88AD/Telescope_data_diamond/ /home/daq/Data/TB_HH_2014_Jan/Telescope_data_diamond/
copy telescope files to /home/daq/Data/TB_HH_2014_Jan/Telescope_data_saphire/ above run number 469
umount /media/E613-88AD/
- Unplug hard drive
- Plug hard drive into cms-alibava
- From cms-alibava to AFS space
- If you have write access in AFS storage area:
rsync -avzr /home/daq/Data/TB_HH_2014_Jan/Telescope_data_diamond/ <username>@pub.ifh.de:/afs/ifh.de/group/cms/TestBeam/HH2013/HH2014/Telescope_data_diamond/
rsync -avzr /home/daq/Data/TB_HH_2014_Jan/Telescope_data_saphire/ <username>@pub.ifh.de:/afs/ifh.de/group/cms/TestBeam/HH2013/HH2014/Telescope_data_saphire/
- Otherwise, ask Olena/Maria/Jessica
- If you have write access in AFS storage area:
NOTE: rsync is sensitive to the trailing "/" at the end of the source directory -- it will either copy the contents or the whole directory. Because of the different folder names for the telescope data there are two copies of some of the files (inside and outside the directory 2014w2), but this is fine so far because there is no space issue yet.
General instructions to use the external hard disk with the Linux machines. Remember to unmount with umount before removing it!
- Connecting
Plug into lccmstmp or cms-alibava
It will automount, and a folder will show up under /media/ containing the files
- Disconnecting
Unmount with umount /media/DiskFolderName
- The disk can now be unplugged
Troubleshooting
If something is wrong (e.g. triggers are not accumulating) and you can't find anything wrong, try killing and restarting the whole DAQ setup. To kill the EUDAQ there is the killEudaq.bat - Shortcut on the desktop of the remote-desktop-connected Windows machine, next to the start GUI shortcut.
Zeuthen Lab Setup for Testing
Running the Software (Test Mode)
To run a test DAQ setup with the TLU test mode:
- Make sure all hardware is on and connected (Wiener crate and TLU).
- Start the DAQ program first (or else the TLU hangs).
- Change to root (su -) and start the TLU test mode in the directory
/home/daq/TMP/itamar/TMPEUDAQ/eudaq/bin
with a command like
./TLUControl.exe -t 5 -d 1 -i RJ45 -hm 2 -v 1 -b 1
The -t argument is the time interval (in ms) between generated triggers. The TLU seems to have a maximum trigger-generating frequency of roughly 400 Hz, so changing e.g. between -t argument of 1 and 2 makes no difference.
The full list of commands can be found by doing ./TLUControl.exe -h. In the example above, -d is the DUT mask, -i is the interface to the DUT, -hm is the handshake mode, -v is the veto mask for vetoing external triggers, and -b is the mask for enabling veto of external triggers.
The program can be stopped with Ctrl-C.
Check the frequency in the last line of the periodic output. If it says 0 Hz, the TLU is hanging. With a time interval of 100 ms, the frequency should be about 10 Hz.
Full output of ./TLUControl.exe -h:
TLU Control Utility version 1.0 A comand-line tool for controlling the Trigger Logic Unit usage: ./TLUControl.exe [options] options: -f --bitfile <filename> (default = ) The bitfile containing the TLU firmware to be loaded -t --trigger <msecs> (default = 0) The interval in milliseconds for internally generated triggers (0 = off) -hm --handshakemode <nohandshake> (default = 0) In this mode the TLU issues a fixed-length pulse on the trigger line (0 = no hand shake) -d --dutmask <mask> (default = 0) The mask for enabling the DUT connections -v --vetomask <mask> (default = 0) The mask for vetoing external triggers -a --andmask <mask> (default = 255) The mask for coincidence of external triggers -o --ormask <mask> (default = 0) The mask for ORing of external triggers -i --dutinputs <value> (default = ) Selects the DUT inputs (values: RJ45,LEMO,HDMI,NONE) -e --error-handler <value> (default = 2) Error handler (0=abort, >0=number of tries before exception) -r --fwversion <value> (default = 0) Firmware version to load (0=auto) -w --wait <ms> (default = 1000) Time to wait between updates in milliseconds -p --strobeperiod <cycles> (default = 1000) Period for timing strobe in clock cycles -l --strobelength <cycles> (default = 100) Length of 'on' time for timing strobe in clock cycles -b --dutveto <mask> (default = 0) Mask for enabling veto of triggers ('backpressure') by rasing DUT_CLK -pw --powervctrl <mV> (default = 800) LV power, range from 250 to 900 (in mV) -n --notimestamp Do not read out timestamp buffer -q --quit Quit after configuring TLU -u --wait-for-user Wait for user input before starting triggers -s --save-file <filename> (default = ) The filename to save trigger numbers and timestamps -z --trace-file <filename> (default = ) The filename to save a trace of all usb accesses, prepend - for only errors, or + for all data (including block transfers)
DAQ Software for Data-taking
In
~/TMP/TB_HH_2014_Jan/DAQ_1721+1495/
do
./runDAQ.sh start
to start the DAQ program.
Stop it again with
./runDAQ.sh stop
In addition, you can check whether or not it's running with
./runDAQ.sh status
This version of the program will check the V1495 register to see if a trigger has been received. If it has, the program will read out the event number and then read out the triggered data from the ADC. It will create an output file with the information from all the triggered events, including a TTree with the event number as a leaf (TluTriggerNumber).
The DAQ program is configured by the parameters in configure.sh in the same directory. Important DAQ parameters include:
- CHANNEL_MASK, bitwise ordered 7-0 (0xff to pass through all channels, e.g. 0x1f to pass channels 0-5)
- CUSTOM_SIZE, BUFFER_ORG -- defines the data-taking period. Don't change BUFFER_ORG; CUSTOM_SIZE determines how many blocks the acquisition window is (e.g. 10 vs 100 gives a window one-tenth the size).
- POST_TRIGGER_NUMBER -- defines how many blocks to store after the trigger vs before the trigger
- INTEGRATION_START, INTEGRATION_STOP -- define the window of peak-finding (in samples)
- BASELINEGATE -- defines the window to determine the baseline (in samples)
The triggers relative to the data will need to be timed in at the beginning of the test beam running. Acquisition window parameters for previous test beam are currently in configure.sh. This corresponds to a window of 1600 ns (1.6 us). Maybe doesn't need to be that long? Adjustable later, in any case. It should be adjusted such that the peak is within the integration start and stop. (There will be some jitter between the trigger and the peak, so the window should be wide enough to deal with that.)
Output files get put into
~/TMP/TB_HH_2014_Jan/DAQ_1721+1495/results/
This output file location is specified in configure.sh.
Data File Analysis
The output files have various data contained within. A TTree exists containing all the raw data (leaf data) along with various headers and other information. In addition, the program already does some peak-finding, and the resulting histograms for each channel are included.
To look at a single event, use the ROOT script show_Event.cxx in the main directory. It is used as follows in ROOT (the data file does not already have to be open):
.L show_Event.cxx show_Event("/path/to/data/file.root",X+N*Y)
where X is the channel number (0-7), Y is the event number (but not specifically TLU trigger number) you wish to look at, and N is the number of ADC channels enabled (currently 5).