[u/mrichter/AliRoot.git] / PMD / pmd_da_info.txt

DETECTOR ALGORITHM for PMD
++++++++++++++++++++++++++
This is a brief note on pmd da:
 - how to get info at P@2
 - how it works
 - the codes
 - Producing DA codes
 - Deployment procedures.  

Author: Satyajit Jena
Date: 14 Nov 2009
Updated: Wed Aug  4 11:03:37 CEST 2010
++++++++++++++++++++++++++

Information is added at:
https://twiki.cern.ch/twiki/bin/view/ALICE/DA

Detail of Central DA Framework can be found here:
https://aliceinfo.cern.ch/DAQ/index.php?q=products/detector-algorithms

++++++++++++++++++++++++++
Important Note:
++++++++++++++++++++++++++
Please note that: The informatoins are also kept in the DA code
header, it is the requirements for DA framework. Please nver change 
the values. it should look like:

PMD DA for online calibration

contact: basanta@iitb.ac.in, Satyajit.Jena@cern.ch
Link: https://twiki.cern.ch/twiki/bin/view/ALICE/DA
Reference run:/afs/cern.ch/user/s/sjena/public/run83496.raw
Run Type: PHYSICS
DA Type: MON
Number of events needed: 1 million for PB+PB, 200 milion for p+p
Input Files: PMD/Calib/Mapping/Run0_999999999_v0_s0.root, PMD_PED.root, 
             PMD_GAIN_CONFIGFILE, pmd_gain_tempfile.dat
Output Files: PMDGAINS.root, to be exported to the DAQ FES
Trigger types used: PHYSICS_EVENT


and 

/*
PMD DA for online calibration

contact: basanta@iitb.ac.in, Satyajit.Jena@cern.ch
Link: https://twiki.cern.ch/twiki/bin/view/ALICE/DA
Reference Run:/afs/cern.ch/user/s/sjena/public/run83496.raw
Run Type: PEDESTAL
DA Type: LDC
Number of events needed: 1000
Input Files: PMD/Calib/Mapping/Run0_999999999_v0_s0.root
Output Files: pmd_ped.root, to be exported to the DAQ FXS, pedestal230*.ped
Trigger types used: PHYSICS_EVENT

*/

This infomration can be accessed in P@2 machince via rpm -qip command

e.g. 
rpm -qip daqDA-PMD-GAIN-36451-trunk.36552.i386.rpm

it will show all the following information it has:

[pmdldc01] /home/pmd/AliSoft/AliRoot >  rpm -qip  daqDA-PMD-PEDESTAL-36451-trunk.36552.i386.rpm
Name        : daqDA-PMD-PEDESTAL           Relocations: /opt/daqDA-PMD-PEDESTAL
Version     : 36451                             Vendor: (none)
Release     : trunk.36552                   Build Date: Mon 09 Nov 2009 01:51:47 PM CET
Install Date: (not installed)               Build Host: pmdldc01.cern.ch
Group       : Applications/Alice            Source RPM: daqDA-PMD-PEDESTAL-36451-trunk.36552.src.rpm
Size        : 65331468                      License: CERN Alice DAQ/Offine
Signature   : (none)
Summary     : PMD Detector Algorithm
Description :
This is the PMD PEDESTAL DA for online calibration.
It uses data from PMD and  detectors at run time.
Build requires: daqDAlib, date, AliRoot trunk.36552, ROOT 5.24.00.
Runtime requires: date.
Contact: basanta@phy.iitb.ac.in, Satyajit.Jena@cern.ch
Link:
Reference Run: /afs/cern.ch/user/s/sjena/public/run83496.raw
Run type: PEDESTAL
DA type: LDC
Number of events needed: 1000
Input files: PMD/Calib/Mapping
Output files: pmd_ped.root, to be exported to the DAQ FXS, pedestal230*.ped
Trigger types used: PHYSICS_EVENT

================================
A Brief Infromation about PMD DA
================================

PMD has two types of detector algorithm. One algorithm is for the 
pedestal data and the other one is for the online gain calibration.

Pedestal DA
============
Pedestal DA is to calculate the mean pedestal value and the root 
mean square value for each channel. The output is written in a 
root file and finally goes to the OCDB

for offline reconstruction.
--------------
At the same time this pedestal da also writes the pedestal values 
in ASCII file corresponding to each DDL. This file is appropriately 
converted to the right format and is loaded

to the MARC for pedestal suppression while taking data. The ASCII files are:

     pedestal2304.ped 
     pedestal2305.ped
     pedestal2306.ped
     pedestal2307.ped
     pedestal2308.ped
     pedestal2309.ped

Six files corresponding to 6 DDL. This is our original configuration.

However, in the year 2009, we have 4 DDL. The corresponding files are:

     pedestal2304.ped
     pedestal2305.ped
     pedestal2308.ped
     pedestal2309.ped

  --Two DDLs are for PREshower plane (2304, 2305) and two DDLs (2308, 2309) 
    are for CPV plane.

  --The input to this da code is the mapping file which can be found here 
    (Run0_999999999_v0_s0.root).

This file can be taken from here or from 
$ALICE_ROOT/OCDB/PMD/Calib/Mapping/Run0_999999999_v0_s0.root.

* Run0_999999999_v0_s0.root: Run0_999999999_v0_s0.root

This file has to be stored in the DAQ database and will be fetched from the 
DAQ database and kept in a local directory as

localOCDB/PMD/Calib/Mapping/Run0_999999999_v0_s0.root.

The output of pedestal da is "PMD_PED.root" which contains mean and rms of 
each channel in a tree. One copy goes to the DAQ database for pedestal 
subtraction while running the gain DA and another copy goes to FES and 
ultimately to the OCDB for offline reconstruction.

The pedestal run for validation can be found here

/afs/cern.ch/user/s/sjena/public/run83496.raw

PMDGAINda:
=========
PMDGAIN da calculates the relative gain of individual channel.

The input to this program are the following files.

    Mapping file which is mentioned above and can also be taken from here 
    * Run0_999999999_v0_s0.root: Run0_999999999_v0_s0.root. During the 
    run this file is fetched from the DAQ database and The other input 
    is "PMD_GAIN_CONFIGFILE" which can be either taken from here 
    PMD_GAIN_CONFIGFILE: PMD_GAIN_CONFIGFILE or from 
    $ALICE_ROOT/PMD/PMD_GAIN_CONFIGFILE. This file has 5 words. 

    filestatus    : 0 or 1 (to start a new run or not) 
    xvar      : variance define (say 5 sigma) to define the hot cell 
    totevt    : incremented event number 
    maxevt    : Total number of Physics event required to find gain 
    hotevtsize : after these many events, the hot cell file will be generated

    PMD_PED.root : A copy of this file is already in the DAQ database. 
    Before every run this is copied from the DAQ database and kept in the 
    local directory for further use. 

Output files:
------------
PMDGAINS.root
PMD_MEAN_SM.root

Input/Output files:

1. PMD_HOT.root : This is the output of gain DA and it is again reused by the DA.

2. pmd_gain_tempfile.dat : This is an ASCII file and generated by gain DA 
   and again taken as the input in the next run.

    If filestatus = 0, the temporary file "pmd_gain_tempfile.dat" 
    is generated. 

    If filestatus = 1, the temporary file "pmd_gain_tempfile.dat" 
    is fetched from the DAQ database. 


The file for gain da validation can be found here
/afs/cern.ch/user/s/sjena/public/run83496.raw

Detail logics and implimentation can be understood by looking at the
code with a minimal knowledge of PMD data structures. 
The Link to Codes:

PEDESTALda http://alisoft.cern.ch/viewvc/trunk/PMD/PMDPEDESTALda.cxx?root=AliRoot&view=log
GAINda http://alisoft.cern.ch/viewvc/trunk/PMD/PMDGAINda.cxx?root=AliRoot&view=log 

=======================
How to build PMD DA
=======================
Building of the DAs is handled in Aliroot. You can find below some hints on 
how to build DAs, but this is no official documentation (such documentation 
has been requested). In case of problems please report directly to the 
Offline team. Build targets for DAs are created by cmake. Before invoking cmake, 
in addition to the standard Aliroot settings (see README_CMake in Aliroot), 
one should make sure it will detect the DA dependencies, or DA compile errors may follow:

    DATE: launch DATE setup with . /date/setup.sh
    AMORE: make sure amore-config is in the path, e.g. export PATH=${PATH}:/opt/amore
    daqDA-lib: export DAQDALIB_PATH=/opt/daqDA-lib

Sourcing the following example environment should allow to build DAs using 
default dependencies installation path:
. DAbuildenv.sh

Afte cmake is invoked, one can build DAs with one of the following targets:

   make daqDA-PMD-all : build all DAs for detector 
   make daqDA-PMD-rpm : build DA RPM packages 
  
   make daqDA-PMD-PEDESTAL : build a given DA
   make daqDA-PMD-PEDESTAL-rpm : build a given DA RPM

   make daqDA-PMD-GAIN : build a given DA
   make daqDA-PMD-GAIN-rpm : build a given DA RPM

RPMs are created in the build directory. Binaries are stored under ./lib/tgt_linuxx8664gcc

======================
DA testing information
======================
It is possible to fully test the DAs outside of the production framework. 
It can be done by setting a number of environment variable, as described 
in the example DAtestenv.sh


    # This is an example DA test environment
    # Put DB files needed by DA in $DAQDA_TEST_DIR
    # and then launch the DA with DAexe DAtestfile.
    # DA local output files will be in $TESTDIR directory
    # and exported files (FXS, AMORE, DaqDetDB) will go to $DAQDA_TEST_DIR.

    # to change according to local settings
    TESTDIR=/tmp/DAtest
    export        DAQDALIB_PATH=.../daqDAlib

    # to be left unchanged
    export        PATH=/bin:/usr/bin
    export        ROOTSYS=NULL
    unset         ALICE_ROOT
    export        DAQDA_TEST_DIR=${TESTDIR}/db
    export        AMORE_NO_DB_DIR=${TESTDIR}/db
    export        DATE_RUN_NUMBER=12345
    export        DATE_DETECTOR_CODE=TST
    export        AMORE_NO_DB=true
    export        DATE_ROLE_NAME=myrole
    export        DIM_DNS_NODE=noname

    mkdir -p $TESTDIR
    mkdir -p $DAQDA_TEST_DIR
    cd $TESTDIR

=================================
Detector DA deployment procedure (cc from DA page)
=================================
The detector team should contact the DAQ team (alice-datesupport@cern.ch) 
and the following steps should be done (in parenthesis, the responsible 
of each action is indicated):

   1. define the calibrations tasks, DAs and corresponding run types in 
      the ECS (detector team)
   2. commit code in Aliroot and test it (detector team)
   3. contact DAQ team to ask for DA release and provide raw data 
      test files (detector team)
   4. build RPM (DAQ team)
   5. compliance checks with DA framework packaging and documentation 
      guidelines. (DAQ team)
   6. in case of fulfillment of the requirements, the DA is considered 
      validated for deployment and the detector team is informed (DAQ team)
   7. deploy and exercise the DA on a real setup with the full DAQ chain 
      and the detector hardware (detector team)
   8. all the above steps are mandatory before allowing the usage of the 
      detector DA in production, ie during cosmic runs
   9. changes in detector DA code (detector team) imply the repetition of 
      steps 2 to 7

Please feel free to contact me (sjena@cern.ch, satya.ino@gmail.com) to know
more or in case of any trouble.
Commit	Line	Data
2d2b959e	1	DETECTOR ALGORITHM for PMD
2d2b959e	2	++++++++++++++++++++++++++
5b12fe37	3	This is a brief note on pmd da:
	4	- how to get info at P@2
	5	- how it works
	6	- the codes
	7	- Producing DA codes
	8	- Deployment procedures.
	9
2d2b959e	10	Author: Satyajit Jena
	11	Date: 14 Nov 2009
	12	Updated: Wed Aug 4 11:03:37 CEST 2010
	13	++++++++++++++++++++++++++
5b12fe37	14
5b12fe37	15	Information is added at:
2d2b959e	16	https://twiki.cern.ch/twiki/bin/view/ALICE/DA
	17
	18	Detail of Central DA Framework can be found here:
	19	https://aliceinfo.cern.ch/DAQ/index.php?q=products/detector-algorithms
	20
5b12fe37	21	++++++++++++++++++++++++++
	22	Important Note:
	23	++++++++++++++++++++++++++
2d2b959e	24	Please note that: The informatoins are also kept in the DA code
	25	header, it is the requirements for DA framework. Please nver change
	26	the values. it should look like:
	27
	28	PMD DA for online calibration
	29
	30	contact: basanta@iitb.ac.in, Satyajit.Jena@cern.ch
	31	Link: https://twiki.cern.ch/twiki/bin/view/ALICE/DA
	32	Reference run:/afs/cern.ch/user/s/sjena/public/run83496.raw
	33	Run Type: PHYSICS
	34	DA Type: MON
	35	Number of events needed: 1 million for PB+PB, 200 milion for p+p
5b12fe37	36	Input Files: PMD/Calib/Mapping/Run0_999999999_v0_s0.root, PMD_PED.root,
5b12fe37	37	PMD_GAIN_CONFIGFILE, pmd_gain_tempfile.dat
2d2b959e	38	Output Files: PMDGAINS.root, to be exported to the DAQ FES
	39	Trigger types used: PHYSICS_EVENT
	40
	41
	42	and
	43
	44	/*
	45	PMD DA for online calibration
	46
	47	contact: basanta@iitb.ac.in, Satyajit.Jena@cern.ch
	48	Link: https://twiki.cern.ch/twiki/bin/view/ALICE/DA
	49	Reference Run:/afs/cern.ch/user/s/sjena/public/run83496.raw
	50	Run Type: PEDESTAL
	51	DA Type: LDC
	52	Number of events needed: 1000
	53	Input Files: PMD/Calib/Mapping/Run0_999999999_v0_s0.root
	54	Output Files: pmd_ped.root, to be exported to the DAQ FXS, pedestal230*.ped
	55	Trigger types used: PHYSICS_EVENT
	56
	57	*/
	58
2d2b959e	59	This infomration can be accessed in P@2 machince via rpm -qip command
	60
	61	e.g.
	62	rpm -qip daqDA-PMD-GAIN-36451-trunk.36552.i386.rpm
	63
	64	it will show all the following information it has:
	65
	66	[pmdldc01] /home/pmd/AliSoft/AliRoot > rpm -qip daqDA-PMD-PEDESTAL-36451-trunk.36552.i386.rpm
	67	Name : daqDA-PMD-PEDESTAL Relocations: /opt/daqDA-PMD-PEDESTAL
	68	Version : 36451 Vendor: (none)
	69	Release : trunk.36552 Build Date: Mon 09 Nov 2009 01:51:47 PM CET
	70	Install Date: (not installed) Build Host: pmdldc01.cern.ch
	71	Group : Applications/Alice Source RPM: daqDA-PMD-PEDESTAL-36451-trunk.36552.src.rpm
	72	Size : 65331468 License: CERN Alice DAQ/Offine
	73	Signature : (none)
	74	Summary : PMD Detector Algorithm
	75	Description :
	76	This is the PMD PEDESTAL DA for online calibration.
	77	It uses data from PMD and detectors at run time.
	78	Build requires: daqDAlib, date, AliRoot trunk.36552, ROOT 5.24.00.
	79	Runtime requires: date.
	80	Contact: basanta@phy.iitb.ac.in, Satyajit.Jena@cern.ch
	81	Link:
	82	Reference Run: /afs/cern.ch/user/s/sjena/public/run83496.raw
	83	Run type: PEDESTAL
	84	DA type: LDC
	85	Number of events needed: 1000
	86	Input files: PMD/Calib/Mapping
	87	Output files: pmd_ped.root, to be exported to the DAQ FXS, pedestal230*.ped
	88	Trigger types used: PHYSICS_EVENT
	89
5b12fe37	90	================================
2d2b959e	91	A Brief Infromation about PMD DA
	92	================================
	93
	94	PMD has two types of detector algorithm. One algorithm is for the
	95	pedestal data and the other one is for the online gain calibration.
	96
	97	Pedestal DA
	98	============
	99	Pedestal DA is to calculate the mean pedestal value and the root
	100	mean square value for each channel. The output is written in a
	101	root file and finally goes to the OCDB
	102
	103	for offline reconstruction.
	104	--------------
	105	At the same time this pedestal da also writes the pedestal values
	106	in ASCII file corresponding to each DDL. This file is appropriately
	107	converted to the right format and is loaded
	108
	109	to the MARC for pedestal suppression while taking data. The ASCII files are:
	110
	111	pedestal2304.ped
	112	pedestal2305.ped
	113	pedestal2306.ped
	114	pedestal2307.ped
	115	pedestal2308.ped
	116	pedestal2309.ped
	117
	118	Six files corresponding to 6 DDL. This is our original configuration.
	119
	120	However, in the year 2009, we have 4 DDL. The corresponding files are:
	121
	122	pedestal2304.ped
	123	pedestal2305.ped
	124	pedestal2308.ped
	125	pedestal2309.ped
	126
	127	--Two DDLs are for PREshower plane (2304, 2305) and two DDLs (2308, 2309)
	128	are for CPV plane.
	129
	130	--The input to this da code is the mapping file which can be found here
	131	(Run0_999999999_v0_s0.root).
	132
	133	This file can be taken from here or from
	134	$ALICE_ROOT/OCDB/PMD/Calib/Mapping/Run0_999999999_v0_s0.root.
	135
	136	* Run0_999999999_v0_s0.root: Run0_999999999_v0_s0.root
	137
	138	This file has to be stored in the DAQ database and will be fetched from the
	139	DAQ database and kept in a local directory as
	140
	141	localOCDB/PMD/Calib/Mapping/Run0_999999999_v0_s0.root.
	142
	143	The output of pedestal da is "PMD_PED.root" which contains mean and rms of
	144	each channel in a tree. One copy goes to the DAQ database for pedestal
	145	subtraction while running the gain DA and another copy goes to FES and
	146	ultimately to the OCDB for offline reconstruction.
	147
	148	The pedestal run for validation can be found here
	149
	150	/afs/cern.ch/user/s/sjena/public/run83496.raw
	151
	152	PMDGAINda:
	153	=========
	154	PMDGAIN da calculates the relative gain of individual channel.
155
156	The input to this program are the following files.
157
158	Mapping file which is mentioned above and can also be taken from here
159	* Run0_999999999_v0_s0.root: Run0_999999999_v0_s0.root. During the
160	run this file is fetched from the DAQ database and The other input
161	is "PMD_GAIN_CONFIGFILE" which can be either taken from here
162	PMD_GAIN_CONFIGFILE: PMD_GAIN_CONFIGFILE or from
163	$ALICE_ROOT/PMD/PMD_GAIN_CONFIGFILE. This file has 5 words.
164
165	filestatus : 0 or 1 (to start a new run or not)
166	xvar : variance define (say 5 sigma) to define the hot cell
167	totevt : incremented event number
168	maxevt : Total number of Physics event required to find gain
169	hotevtsize : after these many events, the hot cell file will be generated
170
171	PMD_PED.root : A copy of this file is already in the DAQ database.
172	Before every run this is copied from the DAQ database and kept in the
173	local directory for further use.
174
175	Output files:
176	------------
177	PMDGAINS.root
178	PMD_MEAN_SM.root
179
180	Input/Output files:
181
182	1. PMD_HOT.root : This is the output of gain DA and it is again reused by the DA.
183
184	2. pmd_gain_tempfile.dat : This is an ASCII file and generated by gain DA
185	and again taken as the input in the next run.
186
187	If filestatus = 0, the temporary file "pmd_gain_tempfile.dat"
188	is generated.
189
190	If filestatus = 1, the temporary file "pmd_gain_tempfile.dat"
191	is fetched from the DAQ database.
192
193
194	The file for gain da validation can be found here
195	/afs/cern.ch/user/s/sjena/public/run83496.raw
196
5b12fe37	197	Detail logics and implimentation can be understood by looking at the
	198	code with a minimal knowledge of PMD data structures.
	199	The Link to Codes:
2d2b959e	200
	201	PEDESTALda http://alisoft.cern.ch/viewvc/trunk/PMD/PMDPEDESTALda.cxx?root=AliRoot&view=log
	202	GAINda http://alisoft.cern.ch/viewvc/trunk/PMD/PMDGAINda.cxx?root=AliRoot&view=log
	203
5b12fe37	204	=======================
5b12fe37	205	How to build PMD DA
2d2b959e	206	=======================
	207	Building of the DAs is handled in Aliroot. You can find below some hints on
	208	how to build DAs, but this is no official documentation (such documentation
	209	has been requested). In case of problems please report directly to the
	210	Offline team. Build targets for DAs are created by cmake. Before invoking cmake,
	211	in addition to the standard Aliroot settings (see README_CMake in Aliroot),
	212	one should make sure it will detect the DA dependencies, or DA compile errors may follow:
	213
	214	DATE: launch DATE setup with . /date/setup.sh
	215	AMORE: make sure amore-config is in the path, e.g. export PATH=${PATH}:/opt/amore
	216	daqDA-lib: export DAQDALIB_PATH=/opt/daqDA-lib
	217
	218	Sourcing the following example environment should allow to build DAs using
	219	default dependencies installation path:
	220	. DAbuildenv.sh
	221
	222	Afte cmake is invoked, one can build DAs with one of the following targets:
	223
5b12fe37	224	make daqDA-PMD-all : build all DAs for detector
5b12fe37	225	make daqDA-PMD-rpm : build DA RPM packages
2d2b959e	226
	227	make daqDA-PMD-PEDESTAL : build a given DA
	228	make daqDA-PMD-PEDESTAL-rpm : build a given DA RPM
	229
	230	make daqDA-PMD-GAIN : build a given DA
	231	make daqDA-PMD-GAIN-rpm : build a given DA RPM
	232
	233	RPMs are created in the build directory. Binaries are stored under ./lib/tgt_linuxx8664gcc
	234
5b12fe37	235	======================
2d2b959e	236	DA testing information
	237	======================
	238	It is possible to fully test the DAs outside of the production framework.
	239	It can be done by setting a number of environment variable, as described
	240	in the example DAtestenv.sh
	241
	242
	243	# This is an example DA test environment
	244	# Put DB files needed by DA in $DAQDA_TEST_DIR
	245	# and then launch the DA with DAexe DAtestfile.
	246	# DA local output files will be in $TESTDIR directory
	247	# and exported files (FXS, AMORE, DaqDetDB) will go to $DAQDA_TEST_DIR.
	248
	249	# to change according to local settings
	250	TESTDIR=/tmp/DAtest
	251	export DAQDALIB_PATH=.../daqDAlib
	252
	253	# to be left unchanged
	254	export PATH=/bin:/usr/bin
	255	export ROOTSYS=NULL
	256	unset ALICE_ROOT
	257	export DAQDA_TEST_DIR=${TESTDIR}/db
	258	export AMORE_NO_DB_DIR=${TESTDIR}/db
	259	export DATE_RUN_NUMBER=12345
	260	export DATE_DETECTOR_CODE=TST
	261	export AMORE_NO_DB=true
	262	export DATE_ROLE_NAME=myrole
	263	export DIM_DNS_NODE=noname
	264
	265	mkdir -p $TESTDIR
	266	mkdir -p $DAQDA_TEST_DIR
	267	cd $TESTDIR
	268
5b12fe37	269	=================================
2d2b959e	270	Detector DA deployment procedure (cc from DA page)
	271	=================================
	272	The detector team should contact the DAQ team (alice-datesupport@cern.ch)
	273	and the following steps should be done (in parenthesis, the responsible
	274	of each action is indicated):
	275
	276	1. define the calibrations tasks, DAs and corresponding run types in
	277	the ECS (detector team)
	278	2. commit code in Aliroot and test it (detector team)
	279	3. contact DAQ team to ask for DA release and provide raw data
	280	test files (detector team)
	281	4. build RPM (DAQ team)
	282	5. compliance checks with DA framework packaging and documentation
	283	guidelines. (DAQ team)
	284	6. in case of fulfillment of the requirements, the DA is considered
	285	validated for deployment and the detector team is informed (DAQ team)
	286	7. deploy and exercise the DA on a real setup with the full DAQ chain
	287	and the detector hardware (detector team)
	288	8. all the above steps are mandatory before allowing the usage of the
	289	detector DA in production, ie during cosmic runs
	290	9. changes in detector DA code (detector team) imply the repetition of
	291	steps 2 to 7
	292
5b12fe37	293	Please feel free to contact me (sjena@cern.ch, satya.ino@gmail.com) to know
5b12fe37	294	more or in case of any trouble.