ACR Tile Training Overview

Tile ACR training
June 2011
Intended for regular training on Thursdays,
after Tile Operations Weekly Meeting
1 Friday, July 29, 2011
Material we will cover
How to start your shift

-
Arrival time and Tile desk
-
Interaction with previous shifter and shift leader
-
Applications to open and sign on
Monitoring:
-
Tile busy
-
Tile trips and status
-
Data quality: digital errors
Calibrations
How to write useful e-log entries
When to call the experts
Other information
DAQ = Data AcQuisition
DQM = Data Quality Monitoring
DCS = Detector Control System

What to check before taking shifts (I)
To access the Atlas Control Room (ACR) you need :
Safety courses 1, 2, 3, 4A
ATLAS Control room ATL_CR
Check your access detail sheet on EDH
https://edh.cern.ch/Document/General/ACRQ
It should look like :
More informations on access :
https://twiki.cern.ch/twiki/bin/view/Atlas/TileCalShifterPages
For newcomers : check
your access rights at least
2 weeks before your shift!
Request your access via EDH. More information in this twiki
https://twiki.cern.ch/twiki/bin/viewauth/Atlas/OtpAtlasControlRoomShifts
For newcomers: check
your access rights at
least 2 weeks before
your shift!
Exercise Self-training
Minimum of 2 shadow shifts, where you ask to be

shown how things are done.
-
Take shadow during the morning shift: ~9am
-
What to do when Tile is busy
-
What to do after a module trips
This presentation is a subset of what you should read

here:
https://atlasop.cern.ch/twiki/bin/view/Main/
NewTileCalorimeterOperationManualShifter
Start of shift
You sit here
He controls ATLAS
partition
Shift Leader:
Ask him/her about
the plan for today
He knows about
L1Calo trigger
Arrive 10 min before your
shift starts, we HAVE to talk
to the previous shifter
Start of shift
Ask the previous Tile Shifter for a brieng and the Shift Leader for the plan of
the day
Log-in to RunCom.
Open (if not already open) the following:

-
GeneralATLAS Logbook. (Login as yourself.)
-
Whiteboard:
https://atlasop.cern.ch/twiki/bin/view/Main/TileCalorimeterWhiteBoard
-
TileCalDCSTile DCS Alarms
-
TileCalDCSTile DCS FSM
-
TileCalL1Calo Map
-
TDAQDAQPanel
FSM panel
Legend
Clickable / Browsable!
DCS Alarm Panel
Voltage trips
Temperature alarms
Yellow = warning.
Often some value ickers in/out of legal range.
Browse the source of the warning in FSM. Copy-paste it to the e-log.

(Screenshot can be taken using snapshot linux tool)
Red = Alarm!
https://atlasop.cern.ch/twiki/bin/view/Main/ShifterToolsDCS
L1Calo Map
Tile Modules
Masked
Tile Modules/
Channels
without signal
A side C side
L1 Calo Map
C side A side
Partition
Its an environment, appropriately congured, in which data are

taken. It has a name, and it uses some parts of ATLAS.
Partitions can exist in parallel, as long as they dont try to use the
same part of the detector.
-
TILE partition: We use it for calibrations. You control it!
-
ATLAS partition: Run Manager uses this for collisions,
cosmics, or any other combined runs. You only watch.
Examples: ATLAS partition, TILE partition.
Monitor/Start Partition
MRS
BUSY
OHP
DQMD
...
DAQPanel: Set partition info & laun
1
2
3
1. Congure, from whiteboard.
2. Read Info.
3. Launch the panel you need.
How to monitor ATLAS partition?
Congure DAQPanel according to parameters in the whiteboard.
Click Monitor Partition.

https://atlasop.cern.ch/twiki/bin/view/Main/ShifterToolsMonitorPartition
The TDAQ software (IGUI)
Look out for...
Load Panels TileIguiPanel to see how Tile is doing.
Is Tile well-congured? That will depend, but typically for physics

we use Reconstruction, and not Raw Data, especially if the L1
event rate is over ~30-40 kHz.
Conguration
for stable beams
Look out for...
MRS ERROR messages. TDAQ DAQPanel MRS

https://atlasop.cern.ch/twiki/bin/view/Main/ShifterToolsMrs
Look out for...
BUSY Panel: TDAQ DAQPanel BUSY
Is Tile BUSY? Serious incident. See:

https://atlasop.cern.ch/twiki/bin/view/Main/TroubleShooting#TileCal_is_BUSY
These would be red.
https://atlasop.cern.ch/twiki/bin/view/Main/MiscellaneousTools#Busy_Monitor
Tile is busy
Look out for...
Is Tile BUSY? Serious incident. See:

https://atlasop.cern.ch/twiki/bin/view/Main/TroubleShooting#TileCal_is_BUSY
Some ROD(s) would be red,
similar to this.
Please re-load this
panel if all crates are in
RED and no busy is
shown in the BUSY
panel
Ritmo Panel
Re-congure a module
DQM and OHP
Data Quality Monitor
Online Histogram Presenter
Keywords
-
Digital errors
-
Occupancy plots
-
Noisy channels
-
Masked channels
-
Known problems (in the whiteboard)
25
New histogram to monitor digital errors
Last bin is always
current LB
Time when module
was recovered
Fraction of
ANY type of
digital errors
integrates
over LB
One histogram per module and run
Browser:TileGATH-EF->SHIFT->DMUErrors->FracTileDigiErrorsEBA07
Back - up
26
TRIP LBC04
5 lumiblocks ago
TRIP LBA45
75 lumiblocks ago
EBC22
Known problem see
whiteboard
LBC03
Small percentage,
just report this in the
e-log
DCS &
Detector Control System
Keywords
-
High Voltage
-
Low Voltage Power Supply (LVPS or fLVPS, where f
is for nger )
-
Power trip!
-
Temperature
-
Alarm Panel
FSM panel
Legend
Clickable / Browsable!
Alarm screen
!"#$%&'(&'$)%*'+,-$+*+./$
01+$23"$4+)1,+'-$'%$'5-$6%..-7(%48-4'$/%8&*-$+48$8.+9-.$
:(-4$%&'(&'$)%*'+,-$(*%'$;%.$+<-6'-8$!"#$
!"#$%&'(&'$)%*'+,-$+*+./$012$
3%$'%$45"$(+6-*$+67$8*98:$;&<%6$!"#$%&#'()*$
=.9'-$+6$-*%,$-6'.>$?9'@$+,&-./$+67$+A-8'-7$012B$96798+'-$
8&..-6'$)%*'+,-$+67$%AC-'$)+*&-D$E@-8:$!"#$96$'@-$!"#$%&#'()*$
1rlpped Lv examples
A message ln Alarm panel !"#$%&'"#(()* wlll appear!
uCS Lakes auLomauc acuons and recovers Lv of Lhe Lrlpped modules
- lf module falls Lo sLarL +&,-). consecuuvely, auLomauc acuon ls .'/(()*&
AuLomauc recovery ls only for Lhe Lv.

Calibrations
Tile Calibration: TileCal TileCalibration

-
Pedestal, CISmono, CISscan, laser-,
laser LG, laser HG.
L1Calo: TileCal Level1 L1 Calibration

https://atlasop.cern.ch/twiki/bin/view/Main/TileCalibrationRuns
Manuall Step-by-step instructions:
Check schedule in Whiteboard Check schedule in Whiteboard Check schedule in Whiteboard Check schedule in Whiteboard Check schedule in Whiteboard
MONDAY TUESDAY WEDNESDAY THURSDAY FRIDAY
L1Calo Tile Calibration TileCalibration L1Calo
To take L1Calo calibration
To Take Tile Calibration
TileCal TileCalibration
Monitoring is not enable so,
in case histograms need to
be check you should take
manually calibration
We use TILE partition for calibrations
Conguration from Whiteboard.
Her we dont just monitor; we START the partition.
Step-by-step in the twiki. Please skim it.
To log or not to log?
When to write an e-log:
End of each run.
Include detector and data quality information.
For any significant problem requiring the intervention of an expert.
Power cycle is a significant problem, please report this
At the end of your shift, make a summary.
Rough chronology of what was done, new problems, general comments.
The goal is to have one e-log report per run for routine operations, one
shift summary, and additional posts only in case of serious problems.
E-log TEMPLATES on the whiteboard
https://atlasop.cern.ch/twiki/bin/view/Main/
NewTileCalorimeterOperationManualShifter
Hot issues
100% Tile Busy begin of a run:

-
Could be due to missing ECR at the start of run.
-
Tile will be busy after ~16000 events at Level1.
-
Faster solution of problem:
Stop / Un-congured (Tile) / Start run
Try at least 2 times if doesnt work call Tile DAQ

expert
-
Medium term solution is to update the software to
avoid this
HOT issues
Module Trips:
-
Look into FSM and ALARM panel to identify if a module trips and auto-recovers.
-
Check in FSM if module is ON again.
-
Check MRS for auto-reconguration is launched (with 3-5 minutes after module is
ON).
-
Check L1Calo for auto-reconguration succeeds.
-
Check RITMO for increasing number of discard events, if that is the case:
re-congure again:
DAQ Panel Tile Stopless Recovery
if still discarding events:
Check OHP histograms: Browse TileGather DMUErrors

FracDigErrorsXXX . If more than 20% in Last lumiblock (number 0 in plot):
Call DCS on-call to ask for a power-cycle

-
Please, write all this information in your RunSummary e-log for all the trips in the run.
If MRS messages from TileLaserRCD complaining about absence of laser TT

Lutin detecter that although the number of laser pulses emitted is changing, the
number of Laser Trigger Types received by LASTROD has not changed for a long
time
Check in DAQPanel TriP (TriggerPresenter) the L1_CALREQ2 PreScale

value:
-
if PS = -1 , it is trigger problems, there is nothing to do (except
complaining)
-
if PS=1 then:
-
Laser should be disable (tell Run Control Shifter, see next slide for
emergency stop of laser)
-
SHAFT conguration should be checked (send an e-mail to laser
experts)
HOT issues: LASER
!"##$"%$!" &'()*+,-./&01(2-342. 5
How to disable the laser in a run ?
!
On the Tile FSM, if you log in, there is a
button "Laser stop" (or similar)
!
Push the button:
"
Laser will not be fired anymore
"
A stopless removal will be triggered to
remove the associated ROBlN
When things go wrong
Be prepared!
https://atlasop.cern.ch/twiki/bin/view/Main/TroubleShooting
More information
TileLBA_resetshafts
This application is
ABSENT during
RUNNING
is OK it should
be like this
Masking a super-drawer
ECR Mismatch
TileCal
Basic electroncs
Basics of Tile Electronics
In order to properly understand the various problems that can occur in TileCal, it is important to understand the basics of the TileCal front end
electronics. In case of a trip of a module or barrel, the shifter must be able to identify the nature of the trip and take appropriate action. The
nature of the trip is given by the error messages in the DCS alarm panel and the colored status display on the DCS.
Each TileCal drawer has a High Voltage (830V) input that provides the voltage bias for all pmts in that drawer.
In addition, low voltage is supplied to each drawer via a two-tier system:
1. A 200 V Bulk Power Supply channel serves four tile drawers. These are the represented by the innermost ring of each barrel in the DCS
schematic.
2. The fLVPS (nger Low Voltage Power Supply) transforms the 200V into 8 different output voltages necessary for the functioning of the
TileCal Front-End electronics inside Super-Drawer(S-D). The fLVPS provides two sets of voltages: a low voltage for the readout
electronics (digitizers and motherboard) on on side of S-D and the low voltage for the High Voltage distributor system on the other side of
S-D. These two sets - "MB_side of fLVPS" and "HV_side of fLVPS" has to be distinguish / mentioned in the case of fLVPS trip.
Given this, drawer trips can be classied as follows:
Trip of the HV input channel to the TileCal drawer. One drawer becomes nonoperational and must be restarted by the DCS expert.
Trip of the fLVPS. One TileCal module becomes nonoperational and must be restarted by the DCS expert and in then the case of "MB_side
of fLVPS" recongured for data taking by the DAQ expert.
A drawer can still be operated in so called "emergency mode" if the "HV_side" of the fLVPS is tripped.
Trip of the 200V power supply. In this case, four drawers are made inoperable and each one must be treated as in a fLVPS trip. The DCS
expert must restart the 200 V Bulk Power Supply channel and fLVPS in all 4 drawers and the DAQ expert must recongure all 4 drawers for data taking.

ACR Tile Training Overview

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

ACR Tile Training Overview

Uploaded by

Copyright:

Available Formats

Tile ACR training

How to start your shift

How to write useful e-log entries

When to call the experts

Minimum of 2 shadow shifts, where you ask to be

This presentation is a subset of what you should read

Open (if not already open) the following:

Often some value ickers in/out of legal range.

Browse the source of the warning in FSM. Copy-paste it to the e-log.

Its an environment, appropriately congured, in which data are

Congure DAQPanel according to parameters in the whiteboard.

Click Monitor Partition.

Load Panels TileIguiPanel to see how Tile is doing.

Is Tile well-congured? That will depend, but typically for physics

MRS ERROR messages. TDAQ DAQPanel MRS

BUSY Panel: TDAQ DAQPanel BUSY

Is Tile BUSY? Serious incident. See:

Is Tile BUSY? Serious incident. See:

Tile Calibration: TileCal TileCalibration

L1Calo: TileCal Level1 L1 Calibration

100% Tile Busy begin of a run:

Stop / Un-congured (Tile) / Start run

Try at least 2 times if doesnt work call Tile DAQ

DAQ Panel Tile Stopless Recovery

if still discarding events:

Check OHP histograms: Browse TileGather DMUErrors

Call DCS on-call to ask for a power-cycle

If MRS messages from TileLaserRCD complaining about absence of laser TT

Check in DAQPanel TriP (TriggerPresenter) the L1_CALREQ2 PreScale

You might also like