]> git.uio.no Git - u/mrichter/AliRoot.git/blame - SHUTTLE/AliShuttle.cxx
git://git.uio.no / u / mrichter / AliRoot.git / blame
? search:
summary | shortlog | log | commit | commitdiff | tree
blob | blame (incremental) | history | HEAD
Pad masking ok (0x3F)
[u/mrichter/AliRoot.git] / SHUTTLE / AliShuttle.cxx
CommitLineData
73abe331 1/**************************************************************************
2 * Copyright(c) 1998-1999, ALICE Experiment at CERN, All rights reserved. *
3 * *
4 * Author: The ALICE Off-line Project. *
5 * Contributors are mentioned in the code where appropriate. *
6 * *
7 * Permission to use, copy, modify and distribute this software and its *
8 * documentation strictly for non-commercial purposes is hereby granted *
9 * without fee, provided that the above copyright notice appears in all *
10 * copies and that both the copyright notice and this permission notice *
11 * appear in the supporting documentation. The authors make no claims *
12 * about the suitability of this software for any purpose. It is *
13 * provided "as is" without express or implied warranty. *
14 **************************************************************************/
15
16/*
17$Log$
a3ca69fe 18Revision 1.81 2007/12/20 14:24:59 jgrosseo
19Do not increase count in case of StoreError
20
c42350c2 21Revision 1.80 2007/12/20 13:31:28 acolla
22Bug fix (Jan Fiete): recovering from StoreError, if the store to OCDB is successful,
23the Shuttle sets current detector's status=done
24
a0aaf247 25Revision 1.79 2007/12/19 14:03:01 acolla
26
27detector name to build the lhcPeriod_DET is to be looked in "detector" column, not "partition"
28
2694f253 29Revision 1.78 2007/12/19 11:50:41 acolla
30
31Raw data tag merged files is written in /alice/data/.../lhcPeriod_DET/runNb/raw if partition is made of DET only
32
a1c5723b 33Revision 1.77 2007/12/19 11:16:16 acolla
34More meaningful log message added in GetFileSources
35
8884d71e 36Revision 1.76 2007/12/19 07:45:20 acolla
37bug fix in the name of the raw tag files (Raw instead of raw)
38
955d2abc 39Revision 1.75 2007/12/18 15:42:14 jgrosseo
40adding number of open runs to monitoring
41
fbc112e3 42Revision 1.74 2007/12/17 03:23:32 jgrosseo
43several bugfixes
44added "empty preprocessor" as placeholder for Acorde in FDR
45
4508ab8b 46Revision 1.73 2007/12/14 19:31:36 acolla
47Sending email to DCS experts is temporarily commented
48
b5f1c82e 49Revision 1.72 2007/12/13 15:44:28 acolla
50Run type added in mail sent to detector expert (eases understanding)
51
6a1146c4 52Revision 1.71 2007/12/12 14:56:14 jgrosseo
53sending shuttle_ignore to ML also in case of 0 events
54
97e3c167 55Revision 1.70 2007/12/12 13:45:35 acolla
56Monalisa started in Collect() function. Alive message to monitor is sent at each Collect and every minute during preprocessor processing.
57
ee6f7523 58Revision 1.69 2007/12/12 10:06:29 acolla
59in AliShuttle.cxx: SHUTTLE logbook is updated in case of invalid run times:
60
61time_start==0 && time_end==0
62
63logbook is NOT updated if time_start != 0 && time_end == 0, because it may mean that the run is still ongoing.
64
1abfbb60 65Revision 1.68 2007/12/11 10:15:17 acolla
66Added marking SHUTTLE=DONE for invalid runs
67(invalid start time or end time) and runs with totalEvents < 1
68
b0e53b15 69Revision 1.67 2007/12/07 19:14:36 acolla
70in AliShuttleTrigger:
71
72Added automatic collection of new runs on a regular time basis (settable from the configuration)
73
74in AliShuttleConfig: new members
75
76- triggerWait: time to wait for DIM trigger (s) before starting automatic collection of new runs
77- mode: run mode (test, prod) -> used to build log folder (logs or logs_PROD)
78
79in AliShuttle:
80
81- logs now stored in logs/#RUN/DET_#RUN.log
82
7d4cf768 83Revision 1.66 2007/12/05 10:45:19 jgrosseo
84changed order of arguments to TMonaLisaWriter
85
468df1ce 86Revision 1.65 2007/11/26 16:58:37 acolla
87Monalisa configuration added: host and table name
88
b832ec02 89Revision 1.64 2007/11/13 16:15:47 acolla
90DCS map is stored in a file in the temp folder where the detector is processed.
91If the preprocessor fails, the temp folder is not removed. This will help the debugging of the problem.
92
b1d18693 93Revision 1.63 2007/11/02 10:53:16 acolla
94Protection added to AliShuttle::CopyFileLocally
95
7d43a416 96Revision 1.62 2007/10/31 18:23:13 acolla
97Furter developement on the Shuttle:
98
99- Shuttle now connects to the Grid as alidaq. The OCDB and Reference folders
100are now built from /alice/data, e.g.:
101/alice/data/2007/LHC07a/OCDB
102
103the year and LHC period are taken from the Shuttle.
104Raw metadata files are stored by GRP to:
105/alice/data/2007/LHC07a/<runNb>/Raw/RunMetadata.root
106
107- Shuttle sends a mail to DCS experts each time DP retrieval fails.
108
675f64cd 109Revision 1.61 2007/10/30 20:33:51 acolla
110Improved managing of temporary folders, which weren't correctly handled.
111Resolved bug introduced in StoreReferenceFile, which caused SPD preprocessor fail.
112
d524ade6 113Revision 1.60 2007/10/29 18:06:16 acolla
114
115New function StoreRunMetadataFile added to preprocessor and Shuttle interface
116This function can be used by GRP only. It stores raw data tags merged file to the
117raw data folder (e.g. /alice/data/2008/LHC08a/000099999/Raw).
118
119KNOWN ISSUES:
120
1211. Shuttle cannot write to /alice/data/ because it belongs to alidaq. Tag file is stored in /alice/simulation/... for the time being.
1222. Due to a bug in TAlien::Mkdir, the creation of a folder in recursive mode (-p option) does not work. The problem
123has been corrected in the root package on the Shuttle machine.
124
c88ad5db 125Revision 1.59 2007/10/05 12:40:55 acolla
126
127Result error code added to AliDCSClient data members (it was "lost" with the new implementation of TMap* GetAliasValues and GetDPValues).
128
1790d4b7 129Revision 1.58 2007/09/28 15:27:40 acolla
130
131AliDCSClient "multiSplit" option added in the DCS configuration
132in AliDCSMessage: variable MAX_BODY_SIZE set to 500000
133
542b6cc8 134Revision 1.57 2007/09/27 16:53:13 acolla
135Detectors can have more than one AMANDA server. SHUTTLE queries the servers sequentially,
136merges the dcs aliases/DPs in one TMap and sends it to the preprocessor.
137
1d172743 138Revision 1.56 2007/09/14 16:46:14 jgrosseo
1391) Connect and Close are called before and after each query, so one can
140keep the same AliDCSClient object.
1412) The splitting of a query is moved to GetDPValues/GetAliasValues.
1423) Splitting interval can be specified in constructor
143
b41b252a 144Revision 1.55 2007/08/06 12:26:40 acolla
145Function Bool_t GetHLTStatus added to preprocessor. It returns the status of HLT
146read from the run logbook.
147
4859271b 148Revision 1.54 2007/07/12 09:51:25 jgrosseo
149removed duplicated log message in GetFile
150
4f0749a8 151Revision 1.53 2007/07/12 09:26:28 jgrosseo
152updating hlt fxs base path
153
42fde080 154Revision 1.52 2007/07/12 08:06:45 jgrosseo
155adding log messages in getfile... functions
156adding not implemented copy constructor in alishuttleconfigholder
157
1bcd28db 158Revision 1.51 2007/07/03 17:24:52 acolla
159root moved to v5-16-00. TFileMerger->Cp moved to TFile::Cp.
160
a986b218 161Revision 1.50 2007/07/02 17:19:32 acolla
162preprocessor is run in a temp directory that is removed when process is finished.
163
5bac2bde 164Revision 1.49 2007/06/29 10:45:06 acolla
165Number of columns in MySql Shuttle logbook increased by one (HLT added)
166
db99d43e 167Revision 1.48 2007/06/21 13:06:19 acolla
168GetFileSources returns dummy list with 1 source if system=DCS (better than
169returning error as it was)
170
6297b37d 171Revision 1.47 2007/06/19 17:28:56 acolla
172HLT updated; missing map bug removed.
173
dc25836b 174Revision 1.46 2007/06/09 13:01:09 jgrosseo
175Switching to retrieval of several DCS DPs at a time (multiDPrequest)
176
a038aa70 177Revision 1.45 2007/05/30 06:35:20 jgrosseo
178Adding functionality to the Shuttle/TestShuttle:
179o) Function to retrieve list of sources from a given system (GetFileSources with id=0)
180o) Function to retrieve list of IDs for a given source (GetFileIDs)
181These functions are needed for dealing with the tag files that are saved for the GRP preprocessor
182Example code has been added to the TestProcessor in TestShuttle
183
4a33bdd9 184Revision 1.44 2007/05/11 16:09:32 acolla
185Reference files for ITS, MUON and PHOS are now stored in OfflineDetName/OnlineDetName/run_...
186example: ITS/SPD/100_filename.root
187
2d9019b4 188Revision 1.43 2007/05/10 09:59:51 acolla
189Various bug fixes in StoreRefFilesToGrid; Cleaning of reference storage before processing detector (CleanReferenceStorage)
190
546242fb 191Revision 1.42 2007/05/03 08:01:39 jgrosseo
192typo in last commit :-(
193
8b739301 194Revision 1.41 2007/05/03 08:00:48 jgrosseo
195fixing log message when pp want to skip dcs value retrieval
196
651fdaab 197Revision 1.40 2007/04/27 07:06:48 jgrosseo
198GetFileSources returns empty list in case of no files, but successful query
199No mails sent in testmode
200
86aa42c3 201Revision 1.39 2007/04/17 12:43:57 acolla
202Correction in StoreOCDB; change of text in mail to detector expert
203
26758fce 204Revision 1.38 2007/04/12 08:26:18 jgrosseo
205updated comment
206
3c2a21c8 207Revision 1.37 2007/04/10 16:53:14 jgrosseo
208redirecting sub detector stdout, stderr to sub detector log file
209
3d8bc902 210Revision 1.35 2007/04/04 16:26:38 acolla
2111. Re-organization of function calls in TestPreprocessor to make it more meaningful.
2122. Added missing dependency in test preprocessors.
2133. in AliShuttle.cxx: processing time and memory consumption info on a single line.
214
886d60e6 215Revision 1.34 2007/04/04 10:33:36 jgrosseo
2161) Storing of files to the Grid is now done _after_ your preprocessors succeeded. This is transparent, which means that you can still use the same functions (Store, StoreReferenceData) to store files to the Grid. However, the Shuttle first stores them locally and transfers them after the preprocessor finished. The return code of these two functions has changed from UInt_t to Bool_t which gives you the success of the storing.
217In case of an error with the Grid, the Shuttle will retry the storing later, the preprocessor does not need to be run again.
218
2192) The meaning of the return code of the preprocessor has changed. 0 is now success and any other value means failure. This value is stored in the log and you can use it to keep details about the error condition.
220
2213) New function StoreReferenceFile to _directly_ store a file (without opening it) to the reference storage.
222
2234) The memory usage of the preprocessor is monitored. If it exceeds 2 GB it is terminated.
224
2255) New function AliPreprocessor::ProcessDCS(). If you do not need to have DCS data in all cases, you can skip the processing by implemting this function and returning kFALSE under certain conditions. E.g. if there is a certain run type.
226If you always need DCS data (like before), you do not need to implement it.
227
2286) The run type has been added to the monitoring page
229
9827400b 230Revision 1.33 2007/04/03 13:56:01 acolla
231Grid Storage at the end of preprocessing. Added virtual method to disable DCS query according to the
232run type.
233
3301427a 234Revision 1.32 2007/02/28 10:41:56 acolla
235Run type field added in SHUTTLE framework. Run type is read from "run type" logbook and retrieved by
236AliPreprocessor::GetRunType() function.
237Added some ldap definition files.
238
d386d623 239Revision 1.30 2007/02/13 11:23:21 acolla
240Moved getters and setters of Shuttle's main OCDB/Reference, local
241OCDB/Reference, temp and log folders to AliShuttleInterface
242
9d733021 243Revision 1.27 2007/01/30 17:52:42 jgrosseo
244adding monalisa monitoring
245
e7f62f16 246Revision 1.26 2007/01/23 19:20:03 acolla
247Removed old ldif files, added TOF, MCH ldif files. Added some options in
248AliShuttleConfig::Print. Added in Ali Shuttle: SetShuttleTempDir and
249SetShuttleLogDir
250
36c99a6a 251Revision 1.25 2007/01/15 19:13:52 acolla
252Moved some AliInfo to AliDebug in SendMail function
253
fc5a4708 254Revision 1.21 2006/12/07 08:51:26 jgrosseo
255update (alberto):
256table, db names in ldap configuration
257added GRP preprocessor
258DCS data can also be retrieved by data point
259
2c15234c 260Revision 1.20 2006/11/16 16:16:48 jgrosseo
261introducing strict run ordering flag
262removed giving preprocessor name to preprocessor, they have to know their name themselves ;-)
263
be48e3ea 264Revision 1.19 2006/11/06 14:23:04 jgrosseo
265major update (Alberto)
266o) reading of run parameters from the logbook
267o) online offline naming conversion
268o) standalone DCSclient package
269
eba76848 270Revision 1.18 2006/10/20 15:22:59 jgrosseo
271o) Adding time out to the execution of the preprocessors: The Shuttle forks and the parent process monitors the child
272o) Merging Collect, CollectAll, CollectNew function
273o) Removing implementation of empty copy constructors (declaration still there!)
274
cb343cfd 275Revision 1.17 2006/10/05 16:20:55 jgrosseo
276adapting to new CDB classes
277
6ec0e06c 278Revision 1.16 2006/10/05 15:46:26 jgrosseo
279applying to the new interface
280
481441a2 281Revision 1.15 2006/10/02 16:38:39 jgrosseo
282update (alberto):
283fixed memory leaks
284storing of objects that failed to be stored to the grid before
285interfacing of shuttle status table in daq system
286
2bb7b766 287Revision 1.14 2006/08/29 09:16:05 jgrosseo
288small update
289
85a80aa9 290Revision 1.13 2006/08/15 10:50:00 jgrosseo
291effc++ corrections (alberto)
292
4f0ab988 293Revision 1.12 2006/08/08 14:19:29 jgrosseo
294Update to shuttle classes (Alberto)
295
296- Possibility to set the full object's path in the Preprocessor's and
297Shuttle's Store functions
298- Possibility to extend the object's run validity in the same classes
299("startValidity" and "validityInfinite" parameters)
300- Implementation of the StoreReferenceData function to store reference
301data in a dedicated CDB storage.
302
84090f85 303Revision 1.11 2006/07/21 07:37:20 jgrosseo
304last run is stored after each run
305
7bfb2090 306Revision 1.10 2006/07/20 09:54:40 jgrosseo
307introducing status management: The processing per subdetector is divided into several steps,
308after each step the status is stored on disk. If the system crashes in any of the steps the Shuttle
309can keep track of the number of failures and skips further processing after a certain threshold is
310exceeded. These thresholds can be configured in LDAP.
311
5164a766 312Revision 1.9 2006/07/19 10:09:55 jgrosseo
313new configuration, accesst to DAQ FES (Alberto)
314
57f50b3c 315Revision 1.8 2006/07/11 12:44:36 jgrosseo
316adding parameters for extended validity range of data produced by preprocessor
317
17111222 318Revision 1.7 2006/07/10 14:37:09 jgrosseo
319small fix + todo comment
320
e090413b 321Revision 1.6 2006/07/10 13:01:41 jgrosseo
322enhanced storing of last sucessfully processed run (alberto)
323
a7160fe9 324Revision 1.5 2006/07/04 14:59:57 jgrosseo
325revision of AliDCSValue: Removed wrapper classes, reduced storage size per value by factor 2
326
45a493ce 327Revision 1.4 2006/06/12 09:11:16 jgrosseo
328coding conventions (Alberto)
329
58bc3020 330Revision 1.3 2006/06/06 14:26:40 jgrosseo
331o) removed files that were moved to STEER
332o) shuttle updated to follow the new interface (Alberto)
333
b948db8d 334Revision 1.2 2006/03/07 07:52:34 hristov
335New version (B.Yordanov)
336
d477ad88 337Revision 1.6 2005/11/19 17:19:14 byordano
338RetrieveDATEEntries and RetrieveConditionsData added
339
340Revision 1.5 2005/11/19 11:09:27 byordano
341AliShuttle declaration added
342
343Revision 1.4 2005/11/17 17:47:34 byordano
344TList changed to TObjArray
345
346Revision 1.3 2005/11/17 14:43:23 byordano
347import to local CVS
348
349Revision 1.1.1.1 2005/10/28 07:33:58 hristov
350Initial import as subdirectory in AliRoot
351
73abe331 352Revision 1.2 2005/09/13 08:41:15 byordano
353default startTime endTime added
354
355Revision 1.4 2005/08/30 09:13:02 byordano
356some docs added
357
358Revision 1.3 2005/08/29 21:15:47 byordano
359some docs added
360
361*/
362
363//
364// This class is the main manager for AliShuttle.
365// It organizes the data retrieval from DCS and call the
b948db8d 366// interface methods of AliPreprocessor.
73abe331 367// For every detector in AliShuttleConfgi (see AliShuttleConfig),
368// data for its set of aliases is retrieved. If there is registered
b948db8d 369// AliPreprocessor for this detector then it will be used
370// accroding to the schema (see AliPreprocessor).
371// If there isn't registered AliPreprocessor than the retrieved
73abe331 372// data is stored automatically to the undelying AliCDBStorage.
373// For detSpec is used the alias name.
374//
375
376#include "AliShuttle.h"
377
378#include "AliCDBManager.h"
379#include "AliCDBStorage.h"
380#include "AliCDBId.h"
84090f85 381#include "AliCDBRunRange.h"
382#include "AliCDBPath.h"
5164a766 383#include "AliCDBEntry.h"
73abe331 384#include "AliShuttleConfig.h"
eba76848 385#include "DCSClient/AliDCSClient.h"
73abe331 386#include "AliLog.h"
b948db8d 387#include "AliPreprocessor.h"
5164a766 388#include "AliShuttleStatus.h"
2bb7b766 389#include "AliShuttleLogbookEntry.h"
73abe331 390
57f50b3c 391#include <TSystem.h>
58bc3020 392#include <TObject.h>
b948db8d 393#include <TString.h>
57f50b3c 394#include <TTimeStamp.h>
73abe331 395#include <TObjString.h>
57f50b3c 396#include <TSQLServer.h>
397#include <TSQLResult.h>
398#include <TSQLRow.h>
cb343cfd 399#include <TMutex.h>
9827400b 400#include <TSystemDirectory.h>
401#include <TSystemFile.h>
a986b218 402#include <TFile.h>
9827400b 403#include <TGrid.h>
404#include <TGridResult.h>
73abe331 405
e7f62f16 406#include <TMonaLisaWriter.h>
407
5164a766 408#include <fstream>
409
cb343cfd 410#include <sys/types.h>
411#include <sys/wait.h>
412
73abe331 413ClassImp(AliShuttle)
414
b948db8d 415//______________________________________________________________________________________________
416AliShuttle::AliShuttle(const AliShuttleConfig* config,
417 UInt_t timeout, Int_t retries):
4f0ab988 418fConfig(config),
419fTimeout(timeout), fRetries(retries),
420fPreprocessorMap(),
2bb7b766 421fLogbookEntry(0),
eba76848 422fCurrentDetector(),
85a80aa9 423fStatusEntry(0),
cb343cfd 424fMonitoringMutex(0),
eba76848 425fLastActionTime(0),
e7f62f16 426fLastAction(),
9827400b 427fMonaLisa(0),
428fTestMode(kNone),
ffa29e93 429fReadTestMode(kFALSE),
430fOutputRedirected(kFALSE)
73abe331 431{
432 //
433 // config: AliShuttleConfig used
73abe331 434 // timeout: timeout used for AliDCSClient connection
435 // retries: the number of retries in case of connection error.
436 //
437
57f50b3c 438 if (!fConfig->IsValid()) AliFatal("********** !!!!! Invalid configuration !!!!! **********");
be48e3ea 439 for(int iSys=0;iSys<4;iSys++) {
57f50b3c 440 fServer[iSys]=0;
be48e3ea 441 if (iSys < 3)
2c15234c 442 fFXSlist[iSys].SetOwner(kTRUE);
57f50b3c 443 }
2bb7b766 444 fPreprocessorMap.SetOwner(kTRUE);
be48e3ea 445
446 for (UInt_t iDet=0; iDet<NDetectors(); iDet++)
447 fFirstUnprocessed[iDet] = kFALSE;
448
cb343cfd 449 fMonitoringMutex = new TMutex();
58bc3020 450}
451
b948db8d 452//______________________________________________________________________________________________
57f50b3c 453AliShuttle::~AliShuttle()
58bc3020 454{
9827400b 455 //
456 // destructor
457 //
58bc3020 458
b948db8d 459 fPreprocessorMap.DeleteAll();
be48e3ea 460 for(int iSys=0;iSys<4;iSys++)
57f50b3c 461 if(fServer[iSys]) {
462 fServer[iSys]->Close();
463 delete fServer[iSys];
eba76848 464 fServer[iSys] = 0;
57f50b3c 465 }
2bb7b766 466
467 if (fStatusEntry){
468 delete fStatusEntry;
469 fStatusEntry = 0;
470 }
cb343cfd 471
472 if (fMonitoringMutex)
473 {
474 delete fMonitoringMutex;
475 fMonitoringMutex = 0;
476 }
73abe331 477}
478
b948db8d 479//______________________________________________________________________________________________
57f50b3c 480void AliShuttle::RegisterPreprocessor(AliPreprocessor* preprocessor)
58bc3020 481{
73abe331 482 //
b948db8d 483 // Registers new AliPreprocessor.
73abe331 484 // It uses GetName() for indentificator of the pre processor.
485 // The pre processor is registered it there isn't any other
486 // with the same identificator (GetName()).
487 //
488
eba76848 489 const char* detName = preprocessor->GetName();
490 if(GetDetPos(detName) < 0)
491 AliFatal(Form("********** !!!!! Invalid detector name: %s !!!!! **********", detName));
492
493 if (fPreprocessorMap.GetValue(detName)) {
494 AliWarning(Form("AliPreprocessor %s is already registered!", detName));
73abe331 495 return;
496 }
497
eba76848 498 fPreprocessorMap.Add(new TObjString(detName), preprocessor);
73abe331 499}
b948db8d 500//______________________________________________________________________________________________
3301427a 501Bool_t AliShuttle::Store(const AliCDBPath& path, TObject* object,
84090f85 502 AliCDBMetaData* metaData, Int_t validityStart, Bool_t validityInfinite)
73abe331 503{
9827400b 504 // Stores a CDB object in the storage for offline reconstruction. Objects that are not needed for
505 // offline reconstruction, but should be stored anyway (e.g. for debugging) should NOT be stored
506 // using this function. Use StoreReferenceData instead!
507 // It calls StoreLocally function which temporarily stores the data locally; when the preprocessor
508 // finishes the data are transferred to the main storage (Grid).
b948db8d 509
3301427a 510 return StoreLocally(fgkLocalCDB, path, object, metaData, validityStart, validityInfinite);
84090f85 511}
512
513//______________________________________________________________________________________________
3301427a 514Bool_t AliShuttle::StoreReferenceData(const AliCDBPath& path, TObject* object, AliCDBMetaData* metaData)
84090f85 515{
9827400b 516 // Stores a CDB object in the storage for reference data. This objects will not be available during
517 // offline reconstrunction. Use this function for reference data only!
518 // It calls StoreLocally function which temporarily stores the data locally; when the preprocessor
519 // finishes the data are transferred to the main storage (Grid).
85a80aa9 520
3301427a 521 return StoreLocally(fgkLocalRefStorage, path, object, metaData);
85a80aa9 522}
523
524//______________________________________________________________________________________________
3301427a 525Bool_t AliShuttle::StoreLocally(const TString& localUri,
85a80aa9 526 const AliCDBPath& path, TObject* object, AliCDBMetaData* metaData,
527 Int_t validityStart, Bool_t validityInfinite)
528{
9827400b 529 // Store object temporarily in local storage. Parameters are passed by Store and StoreReferenceData functions.
530 // when the preprocessor finishes the data are transferred to the main storage (Grid).
531 // The parameters are:
532 // 1) Uri of the backup storage (Local)
533 // 2) the object's path.
534 // 3) the object to be stored
535 // 4) the metaData to be associated with the object
536 // 5) the validity start run number w.r.t. the current run,
537 // if the data is valid only for this run leave the default 0
538 // 6) specifies if the calibration data is valid for infinity (this means until updated),
539 // typical for calibration runs, the default is kFALSE
540 //
541 // returns 0 if fail, 1 otherwise
84090f85 542
9827400b 543 if (fTestMode & kErrorStorage)
544 {
545 Log(fCurrentDetector, "StoreLocally - In TESTMODE - Simulating error while storing locally");
546 return kFALSE;
547 }
548
3301427a 549 const char* cdbType = (localUri == fgkLocalCDB) ? "CDB" : "Reference";
2bb7b766 550
85a80aa9 551 Int_t firstRun = GetCurrentRun() - validityStart;
84090f85 552 if(firstRun < 0) {
9827400b 553 AliWarning("First valid run happens to be less than 0! Setting it to 0.");
84090f85 554 firstRun=0;
555 }
556
557 Int_t lastRun = -1;
558 if(validityInfinite) {
559 lastRun = AliCDBRunRange::Infinity();
560 } else {
561 lastRun = GetCurrentRun();
562 }
563
3301427a 564 // Version is set to current run, it will be used later to transfer data to Grid
565 AliCDBId id(path, firstRun, lastRun, GetCurrentRun(), -1);
2bb7b766 566
567 if(! dynamic_cast<TObjString*> (metaData->GetProperty("RunUsed(TObjString)"))){
568 TObjString runUsed = Form("%d", GetCurrentRun());
9e080f92 569 metaData->SetProperty("RunUsed(TObjString)", runUsed.Clone());
2bb7b766 570 }
84090f85 571
3301427a 572 Bool_t result = kFALSE;
84090f85 573
3301427a 574 if (!(AliCDBManager::Instance()->GetStorage(localUri))) {
575 Log("SHUTTLE", Form("StoreLocally - Cannot activate local %s storage", cdbType));
84090f85 576 } else {
3301427a 577 result = AliCDBManager::Instance()->GetStorage(localUri)
84090f85 578 ->Put(object, id, metaData);
579 }
580
581 if(!result) {
582
9827400b 583 Log(fCurrentDetector, Form("StoreLocally - Can't store object <%s>!", id.ToString().Data()));
3301427a 584 }
2bb7b766 585
3301427a 586 return result;
587}
84090f85 588
3301427a 589//______________________________________________________________________________________________
590Bool_t AliShuttle::StoreOCDB()
591{
9827400b 592 //
593 // Called when preprocessor ends successfully or when previous storage attempt failed (kStoreError status)
594 // Calls underlying StoreOCDB(const char*) function twice, for OCDB and Reference storage.
595 // Then calls StoreRefFilesToGrid to store reference files.
596 //
597
598 if (fTestMode & kErrorGrid)
599 {
600 Log("SHUTTLE", "StoreOCDB - In TESTMODE - Simulating error while storing in the Grid");
601 Log(fCurrentDetector, "StoreOCDB - In TESTMODE - Simulating error while storing in the Grid");
602 return kFALSE;
603 }
604
c88ad5db 605 Log("SHUTTLE","StoreOCDB - Storing OCDB data ...");
86aa42c3 606 Bool_t resultCDB = StoreOCDB(fgkMainCDB);
607
c88ad5db 608 Log("SHUTTLE","StoreOCDB - Storing reference data ...");
3301427a 609 Bool_t resultRef = StoreOCDB(fgkMainRefStorage);
9827400b 610
c88ad5db 611 Log("SHUTTLE","StoreOCDB - Storing reference files ...");
612 Bool_t resultRefFiles = CopyFilesToGrid("reference");
613
614 Bool_t resultMetadata = kTRUE;
615 if(fCurrentDetector == "GRP")
616 {
617 Log("StoreOCDB - SHUTTLE","Storing Run Metadata file ...");
618 resultMetadata = CopyFilesToGrid("metadata");
619 }
9827400b 620
c88ad5db 621 return resultCDB && resultRef && resultRefFiles && resultMetadata;
3301427a 622}
623
624//______________________________________________________________________________________________
625Bool_t AliShuttle::StoreOCDB(const TString& gridURI)
626{
627 //
628 // Called by StoreOCDB(), performs actual storage to the main OCDB and reference storages (Grid)
629 //
630
631 TObjArray* gridIds=0;
632
633 Bool_t result = kTRUE;
634
635 const char* type = 0;
636 TString localURI;
637 if(gridURI == fgkMainCDB) {
638 type = "OCDB";
639 localURI = fgkLocalCDB;
640 } else if(gridURI == fgkMainRefStorage) {
641 type = "reference";
642 localURI = fgkLocalRefStorage;
643 } else {
644 AliError(Form("Invalid storage URI: %s", gridURI.Data()));
645 return kFALSE;
646 }
647
648 AliCDBManager* man = AliCDBManager::Instance();
649
650 AliCDBStorage *gridSto = man->GetStorage(gridURI);
651 if(!gridSto) {
652 Log("SHUTTLE",
653 Form("StoreOCDB - cannot activate main %s storage", type));
654 return kFALSE;
655 }
656
657 gridIds = gridSto->GetQueryCDBList();
658
659 // get objects previously stored in local CDB
660 AliCDBStorage *localSto = man->GetStorage(localURI);
661 if(!localSto) {
662 Log("SHUTTLE",
663 Form("StoreOCDB - cannot activate local %s storage", type));
664 return kFALSE;
665 }
666 AliCDBPath aPath(GetOfflineDetName(fCurrentDetector.Data()),"*","*");
667 // Local objects were stored with current run as Grid version!
668 TList* localEntries = localSto->GetAll(aPath.GetPath(), GetCurrentRun(), GetCurrentRun());
669 localEntries->SetOwner(1);
670
671 // loop on local stored objects
672 TIter localIter(localEntries);
673 AliCDBEntry *aLocEntry = 0;
674 while((aLocEntry = dynamic_cast<AliCDBEntry*> (localIter.Next()))){
675 aLocEntry->SetOwner(1);
676 AliCDBId aLocId = aLocEntry->GetId();
677 aLocEntry->SetVersion(-1);
678 aLocEntry->SetSubVersion(-1);
679
680 // If local object is valid up to infinity we store it only if it is
681 // the first unprocessed run!
682 if (aLocId.GetLastRun() == AliCDBRunRange::Infinity() &&
683 !fFirstUnprocessed[GetDetPos(fCurrentDetector)])
684 {
685 Log("SHUTTLE", Form("StoreOCDB - %s: object %s has validity infinite but "
686 "there are previous unprocessed runs!",
687 fCurrentDetector.Data(), aLocId.GetPath().Data()));
4508ab8b 688 result = kFALSE;
3301427a 689 continue;
690 }
691
692 // loop on Grid valid Id's
693 Bool_t store = kTRUE;
694 TIter gridIter(gridIds);
695 AliCDBId* aGridId = 0;
696 while((aGridId = dynamic_cast<AliCDBId*> (gridIter.Next()))){
697 if(aGridId->GetPath() != aLocId.GetPath()) continue;
698 // skip all objects valid up to infinity
699 if(aGridId->GetLastRun() == AliCDBRunRange::Infinity()) continue;
700 // if we get here, it means there's already some more recent object stored on Grid!
701 store = kFALSE;
702 break;
703 }
704
705 // If we get here, the file can be stored!
706 Bool_t storeOk = gridSto->Put(aLocEntry);
707 if(!store || storeOk){
708
709 if (!store)
710 {
711 Log(fCurrentDetector.Data(),
712 Form("StoreOCDB - A more recent object already exists in %s storage: <%s>",
713 type, aGridId->ToString().Data()));
714 } else {
715 Log("SHUTTLE",
716 Form("StoreOCDB - Object <%s> successfully put into %s storage",
717 aLocId.ToString().Data(), type));
2d9019b4 718 Log(fCurrentDetector.Data(),
719 Form("StoreOCDB - Object <%s> successfully put into %s storage",
720 aLocId.ToString().Data(), type));
3301427a 721 }
84090f85 722
3301427a 723 // removing local filename...
724 TString filename;
725 localSto->IdToFilename(aLocId, filename);
c88ad5db 726 Log("SHUTTLE", Form("StoreOCDB - Removing local file %s", filename.Data()));
3301427a 727 RemoveFile(filename.Data());
728 continue;
729 } else {
730 Log("SHUTTLE",
731 Form("StoreOCDB - Grid %s storage of object <%s> failed",
732 type, aLocId.ToString().Data()));
2d9019b4 733 Log(fCurrentDetector.Data(),
734 Form("StoreOCDB - Grid %s storage of object <%s> failed",
735 type, aLocId.ToString().Data()));
3301427a 736 result = kFALSE;
b948db8d 737 }
738 }
3301427a 739 localEntries->Clear();
2bb7b766 740
b948db8d 741 return result;
3301427a 742}
743
546242fb 744//______________________________________________________________________________________________
745Bool_t AliShuttle::CleanReferenceStorage(const char* detector)
746{
2d9019b4 747 // clears the directory used to store reference files of a given subdetector
546242fb 748
749 AliCDBManager* man = AliCDBManager::Instance();
750 AliCDBStorage* sto = man->GetStorage(fgkLocalRefStorage);
2d9019b4 751 TString localBaseFolder = sto->GetBaseFolder();
752
753 TString targetDir = GetRefFilePrefix(localBaseFolder.Data(), detector);
754
d524ade6 755 Log("SHUTTLE", Form("CleanReferenceStorage - Cleaning %s", targetDir.Data()));
2d9019b4 756
757 TString begin;
758 begin.Form("%d_", GetCurrentRun());
759
760 TSystemDirectory* baseDir = new TSystemDirectory("/", targetDir);
761 if (!baseDir)
762 return kTRUE;
763
764 TList* dirList = baseDir->GetListOfFiles();
765 delete baseDir;
766
767 if (!dirList) return kTRUE;
768
769 if (dirList->GetEntries() < 3)
770 {
771 delete dirList;
772 return kTRUE;
773 }
774
775 Int_t nDirs = 0, nDel = 0;
776 TIter dirIter(dirList);
777 TSystemFile* entry = 0;
546242fb 778
2d9019b4 779 Bool_t success = kTRUE;
546242fb 780
2d9019b4 781 while ((entry = dynamic_cast<TSystemFile*> (dirIter.Next())))
782 {
783 if (entry->IsDirectory())
784 continue;
785
786 TString fileName(entry->GetName());
787 if (!fileName.BeginsWith(begin))
788 continue;
789
790 nDirs++;
791
792 // delete file
793 Int_t result = gSystem->Unlink(fileName.Data());
794
795 if (result)
796 {
d524ade6 797 Log("SHUTTLE", Form("CleanReferenceStorage - Could not delete file %s!", fileName.Data()));
2d9019b4 798 success = kFALSE;
799 } else {
800 nDel++;
801 }
802 }
803
804 if(nDirs > 0)
805 Log("SHUTTLE", Form("CleanReferenceStorage - %d (over %d) reference files in folder %s were deleted.",
806 nDel, nDirs, targetDir.Data()));
807
808
809 delete dirList;
810 return success;
811
812
813
814
815
546242fb 816
817 Int_t result = gSystem->GetPathInfo(targetDir, 0, (Long64_t*) 0, 0, 0);
818 if (result == 0)
819 {
820 // delete directory
d524ade6 821 result = gSystem->Exec(Form("rm -rf %s", targetDir.Data()));
546242fb 822 if (result != 0)
823 {
d524ade6 824 Log("SHUTTLE", Form("CleanReferenceStorage - Could not clean directory %s", targetDir.Data()));
546242fb 825 return kFALSE;
826 }
827 }
828
829 result = gSystem->mkdir(targetDir, kTRUE);
830 if (result != 0)
831 {
c88ad5db 832 Log("SHUTTLE", Form("CleanReferenceStorage - Error creating base directory %s", targetDir.Data()));
546242fb 833 return kFALSE;
834 }
835
836 return kTRUE;
837}
838
9827400b 839//______________________________________________________________________________________________
840Bool_t AliShuttle::StoreReferenceFile(const char* detector, const char* localFile, const char* gridFileName)
841{
842 //
3c2a21c8 843 // Stores reference file directly (without opening it). This function stores the file locally.
9827400b 844 //
3c2a21c8 845 // The file is stored under the following location:
846 // <base folder of local reference storage>/<DET>/<RUN#>_<gridFileName>
847 // where <gridFileName> is the second parameter given to the function
848 //
9827400b 849
850 if (fTestMode & kErrorStorage)
851 {
852 Log(fCurrentDetector, "StoreReferenceFile - In TESTMODE - Simulating error while storing locally");
853 return kFALSE;
854 }
855
856 AliCDBManager* man = AliCDBManager::Instance();
857 AliCDBStorage* sto = man->GetStorage(fgkLocalRefStorage);
858
859 TString localBaseFolder = sto->GetBaseFolder();
860
d524ade6 861 TString target = GetRefFilePrefix(localBaseFolder.Data(), detector);
862 target.Append(Form("/%d_%s", GetCurrentRun(), gridFileName));
9827400b 863
d524ade6 864 return CopyFileLocally(localFile, target);
c88ad5db 865}
866
867//______________________________________________________________________________________________
868Bool_t AliShuttle::StoreRunMetadataFile(const char* localFile, const char* gridFileName)
869{
870 //
871 // Stores Run metadata file to the Grid, in the run folder
872 //
873 // Only GRP can call this function.
874
875 if (fTestMode & kErrorStorage)
876 {
877 Log(fCurrentDetector, "StoreRunMetaDataFile - In TESTMODE - Simulating error while storing locally");
878 return kFALSE;
879 }
880
881 AliCDBManager* man = AliCDBManager::Instance();
882 AliCDBStorage* sto = man->GetStorage(fgkLocalRefStorage);
883
884 TString localBaseFolder = sto->GetBaseFolder();
885
886 // Build Run level folder
955d2abc 887 // folder = /alice/data/year/lhcPeriod/runNb/raw
c88ad5db 888
c88ad5db 889
675f64cd 890 TString lhcPeriod = GetLHCPeriod();
c88ad5db 891 if (lhcPeriod.Length() == 0)
892 {
893 Log("SHUTTLE","StoreRunMetaDataFile - LHCPeriod not found in logbook!");
894 return 0;
895 }
a1c5723b 896
897 // TODO partitions with one detector only write data into LHCperiod_DET
2694f253 898 TString partition = GetRunParameter("detector");
a1c5723b 899
900 if (partition.Length() > 0 && partition != "ALICE")
901 {
902 lhcPeriod.Append(Form("_%s", partition.Data()));
903 Log(fCurrentDetector, Form("Run data tags merged file will be written in %s",
904 lhcPeriod.Data()));
905 }
8884d71e 906
955d2abc 907 TString target = Form("%s/GRP/RunMetadata/alice/data/%d/%s/%09d/raw/%s",
675f64cd 908 localBaseFolder.Data(), GetCurrentYear(),
d524ade6 909 lhcPeriod.Data(), GetCurrentRun(), gridFileName);
c88ad5db 910
d524ade6 911 return CopyFileLocally(localFile, target);
c88ad5db 912}
913
914//______________________________________________________________________________________________
d524ade6 915Bool_t AliShuttle::CopyFileLocally(const char* localFile, const TString& target)
c88ad5db 916{
917 //
918 // Stores file locally. Called by StoreReferenceFile and StoreRunMetadataFile
d524ade6 919 // Files are temporarily stored in the local reference storage. When the preprocessor
920 // finishes, the Shuttle calls CopyFilesToGrid to transfer the files to AliEn
921 // (in reference or run level folders)
c88ad5db 922 //
923
d524ade6 924 TString targetDir(target(0, target.Last('/')));
925
926 //try to open base dir folder, if it does not exist
2d9019b4 927 void* dir = gSystem->OpenDirectory(targetDir.Data());
928 if (dir == NULL) {
929 if (gSystem->mkdir(targetDir.Data(), kTRUE)) {
a1c5723b 930 Log("SHUTTLE", Form("CopyFileLocally - Can't open directory <%s>", targetDir.Data()));
2d9019b4 931 return kFALSE;
932 }
933
934 } else {
935 gSystem->FreeDirectory(dir);
936 }
9827400b 937
7d43a416 938 Int_t result = 0;
939
940 result = gSystem->GetPathInfo(localFile, 0, (Long64_t*) 0, 0, 0);
9827400b 941 if (result)
942 {
a1c5723b 943 Log("SHUTTLE", Form("CopyFileLocally - %s does not exist", localFile));
546242fb 944 return kFALSE;
9827400b 945 }
546242fb 946
7d43a416 947 result = gSystem->GetPathInfo(target, 0, (Long64_t*) 0, 0, 0);
948 if (!result)
949 {
a1c5723b 950 Log("SHUTTLE", Form("CopyFileLocally - target file %s already exist, removing...", target.Data()));
7d43a416 951 if (gSystem->Unlink(target.Data()))
952 {
a1c5723b 953 Log("SHUTTLE", Form("CopyFileLocally - Could not remove existing target file %s!", target.Data()));
7d43a416 954 return kFALSE;
955 }
956 }
957
9827400b 958 result = gSystem->CopyFile(localFile, target);
959
960 if (result == 0)
961 {
a1c5723b 962 Log("SHUTTLE", Form("CopyFileLocally - File %s stored locally to %s", localFile, target.Data()));
9827400b 963 return kTRUE;
964 }
965 else
966 {
a1c5723b 967 Log("SHUTTLE", Form("CopyFileLocally - Could not store file %s to %s! Error code = %d",
546242fb 968 localFile, target.Data(), result));
9827400b 969 return kFALSE;
970 }
c88ad5db 971
972
973
9827400b 974}
975
976//______________________________________________________________________________________________
c88ad5db 977Bool_t AliShuttle::CopyFilesToGrid(const char* type)
9827400b 978{
979 //
c88ad5db 980 // Transfers local files to the Grid. Local files can be reference files
981 // or run metadata file (from GRP only).
9827400b 982 //
c88ad5db 983 // According to the type (ref, metadata) the files are stored under the following location:
984 // ref --> <base folder of reference storage>/<DET>/<RUN#>_<gridFileName>
985 // metadata --> <run data folder>/<MetadataFileName>
86aa42c3 986 //
c88ad5db 987
9827400b 988 AliCDBManager* man = AliCDBManager::Instance();
989 AliCDBStorage* sto = man->GetStorage(fgkLocalRefStorage);
990 if (!sto)
991 return kFALSE;
992 TString localBaseFolder = sto->GetBaseFolder();
9827400b 993
c88ad5db 994 TString dir;
995 TString alienDir;
9827400b 996 TString begin;
9827400b 997
c88ad5db 998 if (strcmp(type, "reference") == 0)
999 {
1000 dir = GetRefFilePrefix(localBaseFolder.Data(), fCurrentDetector.Data());
1001 AliCDBStorage* gridSto = man->GetStorage(fgkMainRefStorage);
1002 if (!gridSto)
1003 return kFALSE;
1004 TString gridBaseFolder = gridSto->GetBaseFolder();
1005 alienDir = GetRefFilePrefix(gridBaseFolder.Data(), fCurrentDetector.Data());
1006 begin = Form("%d_", GetCurrentRun());
1007 }
1008 else if (strcmp(type, "metadata") == 0)
1009 {
c88ad5db 1010
675f64cd 1011 TString lhcPeriod = GetLHCPeriod();
c88ad5db 1012
1013 if (lhcPeriod.Length() == 0)
1014 {
1015 Log("SHUTTLE","CopyFilesToGrid - LHCPeriod not found in logbook!");
1016 return 0;
1017 }
1018
a1c5723b 1019 // TODO partitions with one detector only write data into LHCperiod_DET
2694f253 1020 TString partition = GetRunParameter("detector");
a1c5723b 1021
1022 if (partition.Length() > 0 && partition != "ALICE")
1023 {
1024 lhcPeriod.Append(Form("_%s", partition.Data()));
1025 }
1026
955d2abc 1027 dir = Form("%s/GRP/RunMetadata/alice/data/%d/%s/%09d/raw",
675f64cd 1028 localBaseFolder.Data(), GetCurrentYear(),
c88ad5db 1029 lhcPeriod.Data(), GetCurrentRun());
675f64cd 1030 alienDir = dir(dir.Index("/alice/data/"), dir.Length());
1031
c88ad5db 1032 begin = "";
1033 }
1034 else
1035 {
1036 Log("SHUTTLE", "CopyFilesToGrid - Unexpected: type label must be reference or metadata!");
1037 return kFALSE;
1038 }
1039
9827400b 1040 TSystemDirectory* baseDir = new TSystemDirectory("/", dir);
3d8bc902 1041 if (!baseDir)
1042 return kTRUE;
1043
2d9019b4 1044 TList* dirList = baseDir->GetListOfFiles();
1045 delete baseDir;
1046
1047 if (!dirList) return kTRUE;
1048
1049 if (dirList->GetEntries() < 3)
3d8bc902 1050 {
2d9019b4 1051 delete dirList;
9827400b 1052 return kTRUE;
3d8bc902 1053 }
2d9019b4 1054
546242fb 1055 if (!gGrid)
1056 {
c88ad5db 1057 Log("SHUTTLE", "CopyFilesToGrid - Connection to Grid failed: Cannot continue!");
2d9019b4 1058 delete dirList;
546242fb 1059 return kFALSE;
1060 }
1061
2d9019b4 1062 Int_t nDirs = 0, nTransfer = 0;
1063 TIter dirIter(dirList);
1064 TSystemFile* entry = 0;
1065
9827400b 1066 Bool_t success = kTRUE;
3d8bc902 1067 Bool_t first = kTRUE;
9827400b 1068
2d9019b4 1069 while ((entry = dynamic_cast<TSystemFile*> (dirIter.Next())))
1070 {
9827400b 1071 if (entry->IsDirectory())
1072 continue;
1073
1074 TString fileName(entry->GetName());
1075 if (!fileName.BeginsWith(begin))
1076 continue;
1077
2d9019b4 1078 nDirs++;
1079
3d8bc902 1080 if (first)
1081 {
1082 first = kFALSE;
c88ad5db 1083 // check that folder exists, otherwise create it
3d8bc902 1084 TGridResult* result = gGrid->Ls(alienDir.Data(), "a");
1085
1086 if (!result)
2d9019b4 1087 {
1088 delete dirList;
3d8bc902 1089 return kFALSE;
2d9019b4 1090 }
3d8bc902 1091
546242fb 1092 if (!result->GetFileName(1)) // TODO: It looks like element 0 is always 0!!
3d8bc902 1093 {
675f64cd 1094 // TODO It does not work currently! Bug in TAliEn::Mkdir
1095 // TODO Manually fixed in local root v5-16-00
c88ad5db 1096 if (!gGrid->Mkdir(alienDir.Data(),"-p",0))
3d8bc902 1097 {
c88ad5db 1098 Log("SHUTTLE", Form("CopyFilesToGrid - Cannot create directory %s",
3d8bc902 1099 alienDir.Data()));
2d9019b4 1100 delete dirList;
3d8bc902 1101 return kFALSE;
546242fb 1102 } else {
c88ad5db 1103 Log("SHUTTLE",Form("CopyFilesToGrid - Folder %s created", alienDir.Data()));
3d8bc902 1104 }
1105
546242fb 1106 } else {
c88ad5db 1107 Log("SHUTTLE",Form("CopyFilesToGrid - Folder %s found", alienDir.Data()));
3d8bc902 1108 }
1109 }
1110
9827400b 1111 TString fullLocalPath;
1112 fullLocalPath.Form("%s/%s", dir.Data(), fileName.Data());
1113
1114 TString fullGridPath;
1115 fullGridPath.Form("alien://%s/%s", alienDir.Data(), fileName.Data());
1116
a986b218 1117 Bool_t result = TFile::Cp(fullLocalPath, fullGridPath);
9827400b 1118
1119 if (result)
1120 {
c88ad5db 1121 Log("SHUTTLE", Form("CopyFilesToGrid - Copying local file %s to %s succeeded!",
1122 fullLocalPath.Data(), fullGridPath.Data()));
9827400b 1123 RemoveFile(fullLocalPath);
2d9019b4 1124 nTransfer++;
9827400b 1125 }
1126 else
1127 {
c88ad5db 1128 Log("SHUTTLE", Form("CopyFilesToGrid - Copying local file %s to %s FAILED!",
1129 fullLocalPath.Data(), fullGridPath.Data()));
9827400b 1130 success = kFALSE;
1131 }
1132 }
2d9019b4 1133
c88ad5db 1134 Log("SHUTTLE", Form("CopyFilesToGrid - %d (over %d) files in folder %s copied to Grid.",
1135 nTransfer, nDirs, dir.Data()));
2d9019b4 1136
1137
1138 delete dirList;
9827400b 1139 return success;
1140}
1141
2d9019b4 1142//______________________________________________________________________________________________
1143const char* AliShuttle::GetRefFilePrefix(const char* base, const char* detector)
1144{
1145 //
1146 // Get folder name of reference files
1147 //
1148
1149 TString offDetStr(GetOfflineDetName(detector));
1150 TString dir;
1151 if (offDetStr == "ITS" || offDetStr == "MUON" || offDetStr == "PHOS")
1152 {
1153 dir.Form("%s/%s/%s", base, offDetStr.Data(), detector);
1154 } else {
1155 dir.Form("%s/%s", base, offDetStr.Data());
1156 }
1157
1158 return dir.Data();
1159
1160
1161}
c88ad5db 1162
3301427a 1163//______________________________________________________________________________________________
1164void AliShuttle::CleanLocalStorage(const TString& uri)
1165{
9827400b 1166 //
1167 // Called in case the preprocessor is declared failed. Remove remaining objects from the local storages.
1168 //
3301427a 1169
1170 const char* type = 0;
1171 if(uri == fgkLocalCDB) {
1172 type = "OCDB";
1173 } else if(uri == fgkLocalRefStorage) {
546242fb 1174 type = "Reference";
3301427a 1175 } else {
1176 AliError(Form("Invalid storage URI: %s", uri.Data()));
1177 return;
1178 }
1179
1180 AliCDBManager* man = AliCDBManager::Instance();
b948db8d 1181
3301427a 1182 // open local storage
1183 AliCDBStorage *localSto = man->GetStorage(uri);
1184 if(!localSto) {
1185 Log("SHUTTLE",
1186 Form("CleanLocalStorage - cannot activate local %s storage", type));
1187 return;
1188 }
1189
1190 TString filename(Form("%s/%s/*/Run*_v%d_s*.root",
546242fb 1191 localSto->GetBaseFolder().Data(), GetOfflineDetName(fCurrentDetector.Data()), GetCurrentRun()));
3301427a 1192
c88ad5db 1193 AliDebug(2, Form("filename = %s", filename.Data()));
3301427a 1194
c88ad5db 1195 Log("SHUTTLE", Form("Removing remaining local files for run %d and detector %s ...",
3301427a 1196 GetCurrentRun(), fCurrentDetector.Data()));
1197
1198 RemoveFile(filename.Data());
1199
1200}
1201
1202//______________________________________________________________________________________________
1203void AliShuttle::RemoveFile(const char* filename)
1204{
9827400b 1205 //
1206 // removes local file
1207 //
3301427a 1208
1209 TString command(Form("rm -f %s", filename));
1210
1211 Int_t result = gSystem->Exec(command.Data());
1212 if(result != 0)
1213 {
1214 Log("SHUTTLE", Form("RemoveFile - %s: Cannot remove file %s!",
1215 fCurrentDetector.Data(), filename));
1216 }
73abe331 1217}
1218
b948db8d 1219//______________________________________________________________________________________________
5164a766 1220AliShuttleStatus* AliShuttle::ReadShuttleStatus()
1221{
9827400b 1222 //
1223 // Reads the AliShuttleStatus from the CDB
1224 //
5164a766 1225
2bb7b766 1226 if (fStatusEntry){
1227 delete fStatusEntry;
1228 fStatusEntry = 0;
1229 }
5164a766 1230
10a5a932 1231 fStatusEntry = AliCDBManager::Instance()->GetStorage(GetLocalCDB())
2bb7b766 1232 ->Get(Form("/SHUTTLE/STATUS/%s", fCurrentDetector.Data()), GetCurrentRun());
5164a766 1233
2bb7b766 1234 if (!fStatusEntry) return 0;
1235 fStatusEntry->SetOwner(1);
5164a766 1236
2bb7b766 1237 AliShuttleStatus* status = dynamic_cast<AliShuttleStatus*> (fStatusEntry->GetObject());
1238 if (!status) {
1239 AliError("Invalid object stored to CDB!");
1240 return 0;
1241 }
5164a766 1242
2bb7b766 1243 return status;
5164a766 1244}
1245
1246//______________________________________________________________________________________________
7bfb2090 1247Bool_t AliShuttle::WriteShuttleStatus(AliShuttleStatus* status)
5164a766 1248{
9827400b 1249 //
1250 // writes the status for one subdetector
1251 //
2bb7b766 1252
1253 if (fStatusEntry){
1254 delete fStatusEntry;
1255 fStatusEntry = 0;
1256 }
5164a766 1257
2bb7b766 1258 Int_t run = GetCurrentRun();
5164a766 1259
2bb7b766 1260 AliCDBId id(AliCDBPath("SHUTTLE", "STATUS", fCurrentDetector), run, run);
5164a766 1261
2bb7b766 1262 fStatusEntry = new AliCDBEntry(status, id, new AliCDBMetaData);
1263 fStatusEntry->SetOwner(1);
5164a766 1264
2bb7b766 1265 UInt_t result = AliCDBManager::Instance()->GetStorage(fgkLocalCDB)->Put(fStatusEntry);
7bfb2090 1266
2bb7b766 1267 if (!result) {
3301427a 1268 Log("SHUTTLE", Form("WriteShuttleStatus - Failed for %s, run %d",
1269 fCurrentDetector.Data(), run));
2bb7b766 1270 return kFALSE;
1271 }
e7f62f16 1272
1273 SendMLInfo();
7bfb2090 1274
2bb7b766 1275 return kTRUE;
5164a766 1276}
1277
1278//______________________________________________________________________________________________
1279void AliShuttle::UpdateShuttleStatus(AliShuttleStatus::Status newStatus, Bool_t increaseCount)
1280{
9827400b 1281 //
1282 // changes the AliShuttleStatus for the given detector and run to the given status
1283 //
5164a766 1284
2bb7b766 1285 if (!fStatusEntry){
1286 AliError("UNEXPECTED: fStatusEntry empty");
1287 return;
1288 }
5164a766 1289
2bb7b766 1290 AliShuttleStatus* status = dynamic_cast<AliShuttleStatus*> (fStatusEntry->GetObject());
5164a766 1291
2bb7b766 1292 if (!status){
c88ad5db 1293 Log("SHUTTLE", "UpdateShuttleStatus - UNEXPECTED: status could not be read from current CDB entry");
2bb7b766 1294 return;
1295 }
5164a766 1296
2c15234c 1297 TString actionStr = Form("UpdateShuttleStatus - %s: Changing state from %s to %s",
eba76848 1298 fCurrentDetector.Data(),
36c99a6a 1299 status->GetStatusName(),
eba76848 1300 status->GetStatusName(newStatus));
cb343cfd 1301 Log("SHUTTLE", actionStr);
1302 SetLastAction(actionStr);
5164a766 1303
2bb7b766 1304 status->SetStatus(newStatus);
1305 if (increaseCount) status->IncreaseCount();
5164a766 1306
2bb7b766 1307 AliCDBManager::Instance()->GetStorage(fgkLocalCDB)->Put(fStatusEntry);
e7f62f16 1308
1309 SendMLInfo();
5164a766 1310}
e7f62f16 1311
1312//______________________________________________________________________________________________
1313void AliShuttle::SendMLInfo()
1314{
1315 //
1316 // sends ML information about the current status of the current detector being processed
1317 //
1318
1319 AliShuttleStatus* status = dynamic_cast<AliShuttleStatus*> (fStatusEntry->GetObject());
1320
1321 if (!status){
3301427a 1322 Log("SHUTTLE", "SendMLInfo - UNEXPECTED: status could not be read from current CDB entry");
e7f62f16 1323 return;
1324 }
1325
1326 TMonaLisaText mlStatus(Form("%s_status", fCurrentDetector.Data()), status->GetStatusName());
1327 TMonaLisaValue mlRetryCount(Form("%s_count", fCurrentDetector.Data()), status->GetCount());
1328
1329 TList mlList;
1330 mlList.Add(&mlStatus);
1331 mlList.Add(&mlRetryCount);
1332
ee6f7523 1333 TString mlID;
1334 mlID.Form("%d", GetCurrentRun());
1335 fMonaLisa->SendParameters(&mlList, mlID);
e7f62f16 1336}
1337
5164a766 1338//______________________________________________________________________________________________
1339Bool_t AliShuttle::ContinueProcessing()
1340{
9827400b 1341 // this function reads the AliShuttleStatus information from CDB and
1342 // checks if the processing should be continued
1343 // if yes it returns kTRUE and updates the AliShuttleStatus with nextStatus
2bb7b766 1344
57c1a579 1345 if (!fConfig->HostProcessDetector(fCurrentDetector)) return kFALSE;
1346
1347 AliPreprocessor* aPreprocessor =
1348 dynamic_cast<AliPreprocessor*> (fPreprocessorMap.GetValue(fCurrentDetector));
1349 if (!aPreprocessor)
1350 {
c88ad5db 1351 Log("SHUTTLE", Form("ContinueProcessing - %s: no preprocessor registered", fCurrentDetector.Data()));
57c1a579 1352 return kFALSE;
1353 }
1354
2bb7b766 1355 AliShuttleLogbookEntry::Status entryStatus =
eba76848 1356 fLogbookEntry->GetDetectorStatus(fCurrentDetector);
2bb7b766 1357
1358 if(entryStatus != AliShuttleLogbookEntry::kUnprocessed) {
c88ad5db 1359 Log("SHUTTLE", Form("ContinueProcessing - %s is %s",
2bb7b766 1360 fCurrentDetector.Data(),
1361 fLogbookEntry->GetDetectorStatusName(entryStatus)));
1362 return kFALSE;
1363 }
1364
1365 // if we get here, according to Shuttle logbook subdetector is in UNPROCESSED state
be48e3ea 1366
1367 // check if current run is first unprocessed run for current detector
1368 if (fConfig->StrictRunOrder(fCurrentDetector) &&
1369 !fFirstUnprocessed[GetDetPos(fCurrentDetector)])
1370 {
86aa42c3 1371 if (fTestMode == kNone)
1372 {
c88ad5db 1373 Log("SHUTTLE", Form("ContinueProcessing - %s requires strict run ordering"
1374 " but this is not the first unprocessed run!"));
86aa42c3 1375 return kFALSE;
1376 }
1377 else
1378 {
c88ad5db 1379 Log("SHUTTLE", Form("ContinueProcessing - In TESTMODE - "
1380 "Although %s requires strict run ordering "
1381 "and this is not the first unprocessed run, "
1382 "the SHUTTLE continues"));
86aa42c3 1383 }
be48e3ea 1384 }
1385
2bb7b766 1386 AliShuttleStatus* status = ReadShuttleStatus();
1387 if (!status) {
1388 // first time
1389 Log("SHUTTLE", Form("ContinueProcessing - %s: Processing first time",
1390 fCurrentDetector.Data()));
1391 status = new AliShuttleStatus(AliShuttleStatus::kStarted);
1392 return WriteShuttleStatus(status);
1393 }
1394
1395 // The following two cases shouldn't happen if Shuttle Logbook was correctly updated.
1396 // If it happens it may mean Logbook updating failed... let's do it now!
1397 if (status->GetStatus() == AliShuttleStatus::kDone ||
1398 status->GetStatus() == AliShuttleStatus::kFailed){
1399 Log("SHUTTLE", Form("ContinueProcessing - %s is already %s. Updating Shuttle Logbook",
1400 fCurrentDetector.Data(),
1401 status->GetStatusName(status->GetStatus())));
1402 UpdateShuttleLogbook(fCurrentDetector.Data(),
1403 status->GetStatusName(status->GetStatus()));
1404 return kFALSE;
1405 }
1406
c42350c2 1407 if (status->GetStatus() == AliShuttleStatus::kStoreStarted || status->GetStatus() == AliShuttleStatus::kStoreError) {
2bb7b766 1408 Log("SHUTTLE",
c88ad5db 1409 Form("ContinueProcessing - %s: Grid storage of one or more "
1410 "objects failed. Trying again now",
2bb7b766 1411 fCurrentDetector.Data()));
9827400b 1412 UpdateShuttleStatus(AliShuttleStatus::kStoreStarted);
1413 if (StoreOCDB()){
c88ad5db 1414 Log("SHUTTLE", Form("ContinueProcessing - %s: all objects "
1415 "successfully stored into main storage",
3301427a 1416 fCurrentDetector.Data()));
a0aaf247 1417 UpdateShuttleStatus(AliShuttleStatus::kDone);
1418 UpdateShuttleLogbook(fCurrentDetector, "DONE");
2bb7b766 1419 } else {
1420 Log("SHUTTLE",
1421 Form("ContinueProcessing - %s: Grid storage failed again",
1422 fCurrentDetector.Data()));
9827400b 1423 UpdateShuttleStatus(AliShuttleStatus::kStoreError);
2bb7b766 1424 }
1425 return kFALSE;
1426 }
1427
1428 // if we get here, there is a restart
57c1a579 1429 Bool_t cont = kFALSE;
2bb7b766 1430
1431 // abort conditions
cb343cfd 1432 if (status->GetCount() >= fConfig->GetMaxRetries()) {
57c1a579 1433 Log("SHUTTLE", Form("ContinueProcessing - %s failed %d times in status %s - "
1434 "Updating Shuttle Logbook", fCurrentDetector.Data(),
2bb7b766 1435 status->GetCount(), status->GetStatusName()));
1436 UpdateShuttleLogbook(fCurrentDetector.Data(), "FAILED");
e7f62f16 1437 UpdateShuttleStatus(AliShuttleStatus::kFailed);
3301427a 1438
1439 // there may still be objects in local OCDB and reference storage
1440 // and FXS databases may be not updated: do it now!
9827400b 1441
1442 // TODO Currently disabled, we want to keep files in case of failure!
1443 // CleanLocalStorage(fgkLocalCDB);
1444 // CleanLocalStorage(fgkLocalRefStorage);
1445 // UpdateTableFailCase();
1446
1447 // Send mail to detector expert!
c88ad5db 1448 Log("SHUTTLE", Form("ContinueProcessing - Sending mail to %s expert...",
1449 fCurrentDetector.Data()));
9827400b 1450 if (!SendMail())
1451 Log("SHUTTLE", Form("ContinueProcessing - Could not send mail to %s expert",
1452 fCurrentDetector.Data()));
3301427a 1453
57c1a579 1454 } else {
1455 Log("SHUTTLE", Form("ContinueProcessing - %s: restarting. "
1456 "Aborted before with %s. Retry number %d.", fCurrentDetector.Data(),
1457 status->GetStatusName(), status->GetCount()));
9827400b 1458 Bool_t increaseCount = kTRUE;
c88ad5db 1459 if (status->GetStatus() == AliShuttleStatus::kDCSError ||
1460 status->GetStatus() == AliShuttleStatus::kDCSStarted)
1461 increaseCount = kFALSE;
675f64cd 1462
9827400b 1463 UpdateShuttleStatus(AliShuttleStatus::kStarted, increaseCount);
57c1a579 1464 cont = kTRUE;
2bb7b766 1465 }
1466
57c1a579 1467 return cont;
5164a766 1468}
1469
1470//______________________________________________________________________________________________
2bb7b766 1471Bool_t AliShuttle::Process(AliShuttleLogbookEntry* entry)
58bc3020 1472{
73abe331 1473 //
b948db8d 1474 // Makes data retrieval for all detectors in the configuration.
2bb7b766 1475 // entry: Shuttle logbook entry, contains run paramenters and status of detectors
1476 // (Unprocessed, Inactive, Failed or Done).
d477ad88 1477 // Returns kFALSE in case of error occured and kTRUE otherwise
73abe331 1478 //
1479
9827400b 1480 if (!entry) return kFALSE;
2bb7b766 1481
1482 fLogbookEntry = entry;
1483
c88ad5db 1484 Log("SHUTTLE", Form("\t\t\t^*^*^*^*^*^*^*^*^*^*^*^* run %d: START ^*^*^*^*^*^*^*^*^*^*^*^*",
9827400b 1485 GetCurrentRun()));
2bb7b766 1486
e7f62f16 1487 // Send the information to ML
1488 TMonaLisaText mlStatus("SHUTTLE_status", "Processing");
9827400b 1489 TMonaLisaText mlRunType("SHUTTLE_runtype", Form("%s (%s)", entry->GetRunType(), entry->GetRunParameter("log")));
e7f62f16 1490
1491 TList mlList;
1492 mlList.Add(&mlStatus);
9827400b 1493 mlList.Add(&mlRunType);
e7f62f16 1494
ee6f7523 1495 TString mlID;
1496 mlID.Form("%d", GetCurrentRun());
1497 fMonaLisa->SendParameters(&mlList, mlID);
3301427a 1498
9827400b 1499 if (fLogbookEntry->IsDone())
1500 {
1501 Log("SHUTTLE","Process - Shuttle is already DONE. Updating logbook");
1502 UpdateShuttleLogbook("shuttle_done");
1503 fLogbookEntry = 0;
1504 return kTRUE;
1505 }
1506
1507 // read test mode if flag is set
1508 if (fReadTestMode)
1509 {
3d8bc902 1510 fTestMode = kNone;
9827400b 1511 TString logEntry(entry->GetRunParameter("log"));
1512 //printf("log entry = %s\n", logEntry.Data());
1513 TString searchStr("Testmode: ");
1514 Int_t pos = logEntry.Index(searchStr.Data());
1515 //printf("%d\n", pos);
1516 if (pos >= 0)
1517 {
1518 TSubString subStr = logEntry(pos + searchStr.Length(), logEntry.Length());
1519 //printf("%s\n", subStr.String().Data());
1520 TString newStr(subStr.Data());
1521 TObjArray* token = newStr.Tokenize(' ');
1522 if (token)
1523 {
1524 //token->Print();
1525 TObjString* tmpStr = dynamic_cast<TObjString*> (token->First());
1526 if (tmpStr)
1527 {
1528 Int_t testMode = tmpStr->String().Atoi();
1529 if (testMode > 0)
1530 {
c88ad5db 1531 Log("SHUTTLE", Form("Process - Enabling test mode %d", testMode));
9827400b 1532 SetTestMode((TestMode) testMode);
1533 }
1534 }
1535 delete token;
1536 }
1537 }
1538 }
c88ad5db 1539
eba76848 1540 fLogbookEntry->Print("all");
57f50b3c 1541
1542 // Initialization
d477ad88 1543 Bool_t hasError = kFALSE;
5164a766 1544
675f64cd 1545 // Set the CDB and Reference folders according to the year and LHC period
1546 TString lhcPeriod(GetLHCPeriod());
1547 if (lhcPeriod.Length() == 0)
1548 {
7d4cf768 1549 Log("SHUTTLE","Process - LHCPeriod not found in logbook!");
1550 return 0;
675f64cd 1551 }
1552
1553 if (fgkMainCDB.Length() == 0)
1554 fgkMainCDB = Form("alien://folder=/alice/data/%d/%s/OCDB?user=alidaq?cacheFold=/tmp/OCDBCache",
1555 GetCurrentYear(), lhcPeriod.Data());
1556
1557 if (fgkMainRefStorage.Length() == 0)
1558 fgkMainRefStorage = Form("alien://folder=/alice/data/%d/%s/Reference?user=alidaq?cacheFold=/tmp/OCDBCache",
1559 GetCurrentYear(), lhcPeriod.Data());
1560
57f50b3c 1561 // Loop on detectors in the configuration
b948db8d 1562 TIter iter(fConfig->GetDetectors());
2bb7b766 1563 TObjString* aDetector = 0;
b948db8d 1564
4508ab8b 1565 Bool_t first = kTRUE;
1566
be48e3ea 1567 while ((aDetector = (TObjString*) iter.Next()))
1568 {
7bfb2090 1569 fCurrentDetector = aDetector->String();
5164a766 1570
9e080f92 1571 if (ContinueProcessing() == kFALSE) continue;
4508ab8b 1572
1573 if (first)
1574 {
1575 // only read QueryCDB when needed and only once
1576 AliCDBStorage *mainCDBSto = AliCDBManager::Instance()->GetStorage(fgkMainCDB);
1577 if(mainCDBSto) mainCDBSto->QueryCDB(GetCurrentRun());
1578 AliCDBStorage *mainRefSto = AliCDBManager::Instance()->GetStorage(fgkMainRefStorage);
1579 if(mainRefSto) mainRefSto->QueryCDB(GetCurrentRun());
1580 first = kFALSE;
1581 }
9e080f92 1582
c88ad5db 1583 Log("SHUTTLE", Form("\t\t\t****** run %d - %s: START ******",
2bb7b766 1584 GetCurrentRun(), aDetector->GetName()));
1585
9d733021 1586 for(Int_t iSys=0;iSys<3;iSys++) fFXSCalled[iSys]=kFALSE;
1587
c88ad5db 1588 Log(fCurrentDetector.Data(), "Process - Starting processing");
85a80aa9 1589
be48e3ea 1590 Int_t pid = fork();
1591
1592 if (pid < 0)
1593 {
c88ad5db 1594 Log("SHUTTLE", "Process - ERROR: Forking failed");
be48e3ea 1595 }
1596 else if (pid > 0)
1597 {
1598 // parent
c88ad5db 1599 Log("SHUTTLE", Form("Process - In parent process of %d - %s: Starting monitoring",
be48e3ea 1600 GetCurrentRun(), aDetector->GetName()));
1601
1602 Long_t begin = time(0);
1603
1604 int status; // to be used with waitpid, on purpose an int (not Int_t)!
1605 while (waitpid(pid, &status, WNOHANG) == 0)
1606 {
1607 Long_t expiredTime = time(0) - begin;
1608
1609 if (expiredTime > fConfig->GetPPTimeOut())
1610 {
9827400b 1611 TString tmp;
c88ad5db 1612 tmp.Form("Process - Process of %s time out. "
1613 "Run time: %d seconds. Killing...",
1614 fCurrentDetector.Data(), expiredTime);
9827400b 1615 Log("SHUTTLE", tmp);
1616 Log(fCurrentDetector, tmp);
be48e3ea 1617
1618 kill(pid, 9);
1619
3301427a 1620 UpdateShuttleStatus(AliShuttleStatus::kPPTimeOut);
be48e3ea 1621 hasError = kTRUE;
1622
1623 gSystem->Sleep(1000);
1624 }
1625 else
1626 {
be48e3ea 1627 gSystem->Sleep(1000);
9827400b 1628
1629 TString checkStr;
1630 checkStr.Form("ps -o vsize --pid %d | tail -n 1", pid);
1631 FILE* pipe = gSystem->OpenPipe(checkStr, "r");
1632 if (!pipe)
1633 {
c88ad5db 1634 Log("SHUTTLE", Form("Process - Error: "
1635 "Could not open pipe to %s", checkStr.Data()));
9827400b 1636 continue;
1637 }
1638
1639 char buffer[100];
1640 if (!fgets(buffer, 100, pipe))
1641 {
c88ad5db 1642 Log("SHUTTLE", "Process - Error: ps did not return anything");
9827400b 1643 gSystem->ClosePipe(pipe);
1644 continue;
1645 }
1646 gSystem->ClosePipe(pipe);
1647
1648 //Log("SHUTTLE", Form("ps returned %s", buffer));
1649
1650 Int_t mem = 0;
1651 if ((sscanf(buffer, "%d\n", &mem) != 1) || !mem)
1652 {
c88ad5db 1653 Log("SHUTTLE", "Process - Error: Could not parse output of ps");
9827400b 1654 continue;
1655 }
1656
1657 if (expiredTime % 60 == 0)
ee6f7523 1658 {
c88ad5db 1659 Log("SHUTTLE", Form("Process - %s: Checking process. "
1660 "Run time: %d seconds - Memory consumption: %d KB",
1661 fCurrentDetector.Data(), expiredTime, mem));
ee6f7523 1662 SendAlive();
1663 }
9827400b 1664
1665 if (mem > fConfig->GetPPMaxMem())
1666 {
1667 TString tmp;
c88ad5db 1668 tmp.Form("Process - Process exceeds maximum allowed memory "
1669 "(%d KB > %d KB). Killing...",
9827400b 1670 mem, fConfig->GetPPMaxMem());
1671 Log("SHUTTLE", tmp);
1672 Log(fCurrentDetector, tmp);
1673
1674 kill(pid, 9);
1675
1676 UpdateShuttleStatus(AliShuttleStatus::kPPOutOfMemory);
1677 hasError = kTRUE;
1678
1679 gSystem->Sleep(1000);
1680 }
be48e3ea 1681 }
1682 }
1683
c88ad5db 1684 Log("SHUTTLE", Form("Process - In parent process of %d - %s: Client has terminated.",
be48e3ea 1685 GetCurrentRun(), aDetector->GetName()));
1686
1687 if (WIFEXITED(status))
1688 {
1689 Int_t returnCode = WEXITSTATUS(status);
1690
c88ad5db 1691 Log("SHUTTLE", Form("Process - %s: the return code is %d", fCurrentDetector.Data(),
3301427a 1692 returnCode));
be48e3ea 1693
9827400b 1694 if (returnCode == 0) hasError = kTRUE;
be48e3ea 1695 }
1696 }
1697 else if (pid == 0)
1698 {
1699 // client
c88ad5db 1700 Log("SHUTTLE", Form("Process - In client process of %d - %s", GetCurrentRun(),
1701 aDetector->GetName()));
be48e3ea 1702
c88ad5db 1703 Log("SHUTTLE", Form("Process - Redirecting output to %s log",fCurrentDetector.Data()));
ffa29e93 1704
546242fb 1705 if ((freopen(GetLogFileName(fCurrentDetector), "a", stdout)) == 0)
ffa29e93 1706 {
c88ad5db 1707 Log("SHUTTLE", "Process - Could not freopen stdout");
ffa29e93 1708 }
1709 else
1710 {
1711 fOutputRedirected = kTRUE;
1712 if ((dup2(fileno(stdout), fileno(stderr))) < 0)
c88ad5db 1713 Log("SHUTTLE", "Process - Could not redirect stderr");
ffa29e93 1714
1715 }
1716
5bac2bde 1717 TString wd = gSystem->WorkingDirectory();
675f64cd 1718 TString tmpDir = Form("%s/%s_%d_process", GetShuttleTempDir(),
1719 fCurrentDetector.Data(), GetCurrentRun());
5bac2bde 1720
d524ade6 1721 Int_t result = gSystem->GetPathInfo(tmpDir.Data(), 0, (Long64_t*) 0, 0, 0);
1722 if (!result) // temp dir already exists!
1723 {
1724 Log(fCurrentDetector.Data(),
1725 Form("Process - %s dir already exists! Removing...", tmpDir.Data()));
1726 gSystem->Exec(Form("rm -rf %s",tmpDir.Data()));
675f64cd 1727 }
1728
1729 if (gSystem->mkdir(tmpDir.Data(), 1))
1730 {
1731 Log(fCurrentDetector.Data(), "Process - could not make temp directory!!");
1732 gSystem->Exit(1);
d524ade6 1733 }
1734
1735 if (!gSystem->ChangeDirectory(tmpDir.Data()))
1736 {
1737 Log(fCurrentDetector.Data(), "Process - could not change directory!!");
1738 gSystem->Exit(1);
1739 }
5bac2bde 1740
9827400b 1741 Bool_t success = ProcessCurrentDetector();
5bac2bde 1742
1743 gSystem->ChangeDirectory(wd.Data());
b1d18693 1744
9827400b 1745 if (success) // Preprocessor finished successfully!
1746 {
b1d18693 1747 // remove temporary folder
fbc112e3 1748 // temporary commented (JF)
1749 //gSystem->Exec(Form("rm -rf %s",tmpDir.Data()));
b1d18693 1750
3301427a 1751 // Update time_processed field in FXS DB
1752 if (UpdateTable() == kFALSE)
5bac2bde 1753 Log("SHUTTLE", Form("Process - %s: Could not update FXS databases!",
1754 fCurrentDetector.Data()));
3301427a 1755
1756 // Transfer the data from local storage to main storage (Grid)
1757 UpdateShuttleStatus(AliShuttleStatus::kStoreStarted);
1758 if (StoreOCDB() == kFALSE)
1759 {
c88ad5db 1760 Log("SHUTTLE",
1761 Form("\t\t\t****** run %d - %s: STORAGE ERROR ******",
3301427a 1762 GetCurrentRun(), aDetector->GetName()));
1763 UpdateShuttleStatus(AliShuttleStatus::kStoreError);
9827400b 1764 success = kFALSE;
3301427a 1765 } else {
c88ad5db 1766 Log("SHUTTLE",
1767 Form("\t\t\t****** run %d - %s: DONE ******",
3301427a 1768 GetCurrentRun(), aDetector->GetName()));
1769 UpdateShuttleStatus(AliShuttleStatus::kDone);
9827400b 1770 UpdateShuttleLogbook(fCurrentDetector, "DONE");
3301427a 1771 }
c88ad5db 1772 } else
1773 {
1774 Log("SHUTTLE",
1775 Form("\t\t\t****** run %d - %s: PP ERROR ******",
1776 GetCurrentRun(), aDetector->GetName()));
be48e3ea 1777 }
1778
4b95672b 1779 for (UInt_t iSys=0; iSys<3; iSys++)
1780 {
1781 if (fFXSCalled[iSys]) fFXSlist[iSys].Clear();
1782 }
1783
c88ad5db 1784 Log("SHUTTLE", Form("Process - Client process of %d - %s is exiting now with %d.",
9827400b 1785 GetCurrentRun(), aDetector->GetName(), success));
be48e3ea 1786
1787 // the client exits here
9827400b 1788 gSystem->Exit(success);
be48e3ea 1789
1790 AliError("We should never get here!!!");
1791 }
7bfb2090 1792 }
5164a766 1793
c88ad5db 1794 Log("SHUTTLE", Form("\t\t\t^*^*^*^*^*^*^*^*^*^*^*^* run %d: FINISH ^*^*^*^*^*^*^*^*^*^*^*^*",
2bb7b766 1795 GetCurrentRun()));
1796
1797 //check if shuttle is done for this run, if so update logbook
1798 TObjArray checkEntryArray;
1799 checkEntryArray.SetOwner(1);
9e080f92 1800 TString whereClause = Form("where run=%d", GetCurrentRun());
b0e53b15 1801 if (!QueryShuttleLogbook(whereClause.Data(), checkEntryArray) ||
1802 checkEntryArray.GetEntries() == 0) {
9e080f92 1803 Log("SHUTTLE", Form("Process - Warning: Cannot check status of run %d on Shuttle logbook!",
1804 GetCurrentRun()));
1805 return hasError == kFALSE;
1806 }
b948db8d 1807
9e080f92 1808 AliShuttleLogbookEntry* checkEntry = dynamic_cast<AliShuttleLogbookEntry*>
1809 (checkEntryArray.At(0));
2bb7b766 1810
9e080f92 1811 if (checkEntry)
1812 {
1813 if (checkEntry->IsDone())
be48e3ea 1814 {
9e080f92 1815 Log("SHUTTLE","Process - Shuttle is DONE. Updating logbook");
1816 UpdateShuttleLogbook("shuttle_done");
1817 }
1818 else
1819 {
1820 for (UInt_t iDet=0; iDet<NDetectors(); iDet++)
be48e3ea 1821 {
9e080f92 1822 if (checkEntry->GetDetectorStatus(iDet) == AliShuttleLogbookEntry::kUnprocessed)
be48e3ea 1823 {
9e080f92 1824 AliDebug(2, Form("Run %d: setting %s as \"not first time unprocessed\"",
1825 checkEntry->GetRun(), GetDetName(iDet)));
1826 fFirstUnprocessed[iDet] = kFALSE;
be48e3ea 1827 }
1828 }
2bb7b766 1829 }
1830 }
1831
1832 fLogbookEntry = 0;
85a80aa9 1833
a7160fe9 1834 return hasError == kFALSE;
73abe331 1835}
1836
b948db8d 1837//______________________________________________________________________________________________
9827400b 1838Bool_t AliShuttle::ProcessCurrentDetector()
73abe331 1839{
1840 //
2bb7b766 1841 // Makes data retrieval just for a specific detector (fCurrentDetector).
73abe331 1842 // Threre should be a configuration for this detector.
73abe331 1843
1d172743 1844 Log("SHUTTLE", Form("ProcessCurrentDetector - Retrieving values for %s, run %d",
1845 fCurrentDetector.Data(), GetCurrentRun()));
73abe331 1846
d524ade6 1847 TString wd = gSystem->WorkingDirectory();
1848
2d9019b4 1849 if (!CleanReferenceStorage(fCurrentDetector.Data()))
546242fb 1850 return kFALSE;
d524ade6 1851
1852 gSystem->ChangeDirectory(wd.Data());
1853
1d172743 1854 TMap* dcsMap = new TMap();
3301427a 1855
1856 // call preprocessor
1857 AliPreprocessor* aPreprocessor =
1858 dynamic_cast<AliPreprocessor*> (fPreprocessorMap.GetValue(fCurrentDetector));
1859
1860 aPreprocessor->Initialize(GetCurrentRun(), GetCurrentStartTime(), GetCurrentEndTime());
1861
1862 Bool_t processDCS = aPreprocessor->ProcessDCS();
d477ad88 1863
651fdaab 1864 if (!processDCS)
1865 {
1d172743 1866 Log(fCurrentDetector, "ProcessCurrentDetector -"
1867 " The preprocessor requested to skip the retrieval of DCS values");
651fdaab 1868 }
8b739301 1869 else if (fTestMode & kSkipDCS)
2c15234c 1870 {
1d172743 1871 Log(fCurrentDetector, "ProcessCurrentDetector - In TESTMODE: Skipping DCS processing");
9827400b 1872 }
1873 else if (fTestMode & kErrorDCS)
1874 {
1d172743 1875 Log(fCurrentDetector, "ProcessCurrentDetector - In TESTMODE: Simulating DCS error");
3d8bc902 1876 UpdateShuttleStatus(AliShuttleStatus::kDCSStarted);
9827400b 1877 UpdateShuttleStatus(AliShuttleStatus::kDCSError);
1d172743 1878 delete dcsMap;
9827400b 1879 return kFALSE;
2c15234c 1880 } else {
3301427a 1881
1882 UpdateShuttleStatus(AliShuttleStatus::kDCSStarted);
1883
1d172743 1884 // Query DCS archive
1885 Int_t nServers = fConfig->GetNServers(fCurrentDetector);
a038aa70 1886
1d172743 1887 for (int iServ=0; iServ<nServers; iServ++)
2c15234c 1888 {
1d172743 1889
1890 TString host(fConfig->GetDCSHost(fCurrentDetector, iServ));
1891 Int_t port = fConfig->GetDCSPort(fCurrentDetector, iServ);
542b6cc8 1892 Int_t multiSplit = fConfig->GetMultiSplit(fCurrentDetector, iServ);
1893
1790d4b7 1894 Log(fCurrentDetector, Form("ProcessCurrentDetector -"
1895 " Querying DCS Amanda server %s:%d (%d of %d)",
1896 host.Data(), port, iServ+1, nServers));
1d172743 1897
1898 TMap* aliasMap = 0;
1899 TMap* dpMap = 0;
1900
1901 if (fConfig->GetDCSAliases(fCurrentDetector, iServ)->GetEntries() > 0)
2c15234c 1902 {
1d172743 1903 aliasMap = GetValueSet(host, port,
542b6cc8 1904 fConfig->GetDCSAliases(fCurrentDetector, iServ),
1905 kAlias, multiSplit);
1d172743 1906 if (!aliasMap)
1907 {
1908 Log(fCurrentDetector,
1909 Form("ProcessCurrentDetector -"
675f64cd 1910 " Error retrieving DCS aliases from server %s."
1911 " Sending mail to DCS experts!", host.Data()));
1d172743 1912 UpdateShuttleStatus(AliShuttleStatus::kDCSError);
675f64cd 1913
b5f1c82e 1914 //if (!SendMailToDCS())
1915 // Log("SHUTTLE", Form("ProcessCurrentDetector - Could not send mail to DCS experts!"));
675f64cd 1916
a038aa70 1917 delete dcsMap;
1d172743 1918 return kFALSE;
1919 }
2c15234c 1920 }
a038aa70 1921
1d172743 1922 if (fConfig->GetDCSDataPoints(fCurrentDetector, iServ)->GetEntries() > 0)
a038aa70 1923 {
1d172743 1924 dpMap = GetValueSet(host, port,
542b6cc8 1925 fConfig->GetDCSDataPoints(fCurrentDetector, iServ),
1926 kDP, multiSplit);
1d172743 1927 if (!dpMap)
1928 {
1929 Log(fCurrentDetector,
1930 Form("ProcessCurrentDetector -"
675f64cd 1931 " Error retrieving DCS data points from server %s."
1932 " Sending mail to DCS experts!", host.Data()));
1d172743 1933 UpdateShuttleStatus(AliShuttleStatus::kDCSError);
675f64cd 1934
b5f1c82e 1935 //if (!SendMailToDCS())
1936 // Log("SHUTTLE", Form("ProcessCurrentDetector - Could not send mail to DCS experts!"));
675f64cd 1937
1d172743 1938 if (aliasMap) delete aliasMap;
1939 delete dcsMap;
1940 return kFALSE;
1941 }
a038aa70 1942 }
1d172743 1943
1944 // merge aliasMap and dpMap into dcsMap
1945 if(aliasMap) {
1946 TIter iter(aliasMap);
a038aa70 1947 TObjString* key = 0;
1948 while ((key = (TObjString*) iter.Next()))
1d172743 1949 dcsMap->Add(key, aliasMap->GetValue(key->String()));
1950
1951 aliasMap->SetOwner(kFALSE);
1952 delete aliasMap;
1953 }
1954
1955 if(dpMap) {
1956 TIter iter(dpMap);
1957 TObjString* key = 0;
1958 while ((key = (TObjString*) iter.Next()))
1959 dcsMap->Add(key, dpMap->GetValue(key->String()));
1960
1961 dpMap->SetOwner(kFALSE);
1962 delete dpMap;
a038aa70 1963 }
73abe331 1964 }
1965 }
dc25836b 1966
b1d18693 1967 // save map into file, to help debugging in case of preprocessor error
fbc112e3 1968 TFile* f = TFile::Open("DCSMap.root","recreate");
b1d18693 1969 f->cd();
1970 dcsMap->Write("DCSMap", TObject::kSingleKey);
1971 f->Close();
fbc112e3 1972 delete f;
b1d18693 1973
2bb7b766 1974 // DCS Archive DB processing successful. Call Preprocessor!
85a80aa9 1975 UpdateShuttleStatus(AliShuttleStatus::kPPStarted);
a7160fe9 1976
a038aa70 1977 UInt_t returnValue = aPreprocessor->Process(dcsMap);
b948db8d 1978
3301427a 1979 if (returnValue > 0) // Preprocessor error!
1980 {
c88ad5db 1981 Log(fCurrentDetector, Form("ProcessCurrentDetector - "
1982 "Preprocessor failed. Process returned %d.", returnValue));
cb343cfd 1983 UpdateShuttleStatus(AliShuttleStatus::kPPError);
a038aa70 1984 dcsMap->DeleteAll();
1985 delete dcsMap;
9827400b 1986 return kFALSE;
1987 }
1988
1989 // preprocessor ok!
1990 UpdateShuttleStatus(AliShuttleStatus::kPPDone);
1991 Log(fCurrentDetector, Form("ProcessCurrentDetector - %s preprocessor returned success",
1992 fCurrentDetector.Data()));
b948db8d 1993
a038aa70 1994 dcsMap->DeleteAll();
1995 delete dcsMap;
b948db8d 1996
9827400b 1997 return kTRUE;
2bb7b766 1998}
1999
fbc112e3 2000//______________________________________________________________________________________________
2001void AliShuttle::CountOpenRuns()
2002{
2003 // Query DAQ's Shuttle logbook and sends the number of open runs to ML
2004
2005 // check connection, in case connect
2006 if (!Connect(3))
2007 return;
2008
2009 TString sqlQuery;
2010 sqlQuery = Form("select count(*) from %s where shuttle_done=0", fConfig->GetShuttlelbTable());
2011
2012 TSQLResult* aResult = fServer[3]->Query(sqlQuery);
2013 if (!aResult) {
2014 AliError(Form("Can't execute query <%s>!", sqlQuery.Data()));
2015 return;
2016 }
2017
2018 AliDebug(2,Form("Query = %s", sqlQuery.Data()));
2019
2020 if (aResult->GetRowCount() == 0) {
2021 AliError(Form("No result for query %s received", sqlQuery.Data()));
2022 return;
2023 }
2024
2025 if (aResult->GetFieldCount() != 1) {
2026 AliError(Form("Invalid field count for query %s received", sqlQuery.Data()));
2027 return;
2028 }
2029
2030 TSQLRow* aRow = aResult->Next();
2031 if (!aRow) {
2032 AliError(Form("Could not receive result of query %s", sqlQuery.Data()));
2033 return;
2034 }
2035
2036 TString result(aRow->GetField(0), aRow->GetFieldLength(0));
2037 Int_t count = result.Atoi();
2038
2039 Log("SHUTTLE", Form("%d unprocessed runs", count));
2040
2041 delete aRow;
2042 delete aResult;
2043
2044 TMonaLisaValue mlStatus("SHUTTLE_openruns", count);
2045
2046 TList mlList;
2047 mlList.Add(&mlStatus);
2048
2049 fMonaLisa->SendParameters(&mlList, "__PROCESSINGINFO__");
2050}
2051
2bb7b766 2052//______________________________________________________________________________________________
2053Bool_t AliShuttle::QueryShuttleLogbook(const char* whereClause,
2054 TObjArray& entries)
2055{
9827400b 2056 // Query DAQ's Shuttle logbook and fills detector status object.
2057 // Call QueryRunParameters to query DAQ logbook for run parameters.
2058 //
2bb7b766 2059
fc5a4708 2060 entries.SetOwner(1);
2061
2bb7b766 2062 // check connection, in case connect
fbc112e3 2063 if (!Connect(3)) return kFALSE;
2bb7b766 2064
2065 TString sqlQuery;
441b0e9c 2066 sqlQuery = Form("select * from %s %s order by run", fConfig->GetShuttlelbTable(), whereClause);
2bb7b766 2067
be48e3ea 2068 TSQLResult* aResult = fServer[3]->Query(sqlQuery);
2bb7b766 2069 if (!aResult) {
2070 AliError(Form("Can't execute query <%s>!", sqlQuery.Data()));
2071 return kFALSE;
2072 }
2073
fc5a4708 2074 AliDebug(2,Form("Query = %s", sqlQuery.Data()));
2075
2bb7b766 2076 if(aResult->GetRowCount() == 0) {
c88ad5db 2077 Log("SHUTTLE", "No entries in Shuttle Logbook match request");
9827400b 2078 delete aResult;
2079 return kTRUE;
2bb7b766 2080 }
2081
2082 // TODO Check field count!
db99d43e 2083 const UInt_t nCols = 23;
2bb7b766 2084 if (aResult->GetFieldCount() != (Int_t) nCols) {
c88ad5db 2085 Log("SHUTTLE", "Invalid SQL result field number!");
2bb7b766 2086 delete aResult;
2087 return kFALSE;
2088 }
2089
2bb7b766 2090 TSQLRow* aRow;
2091 while ((aRow = aResult->Next())) {
2092 TString runString(aRow->GetField(0), aRow->GetFieldLength(0));
2093 Int_t run = runString.Atoi();
2094
eba76848 2095 AliShuttleLogbookEntry *entry = QueryRunParameters(run);
2096 if (!entry)
2097 continue;
2bb7b766 2098
2099 // loop on detectors
eba76848 2100 for(UInt_t ii = 0; ii < nCols; ii++)
2101 entry->SetDetectorStatus(aResult->GetFieldName(ii), aRow->GetField(ii));
2bb7b766 2102
eba76848 2103 entries.AddLast(entry);
2bb7b766 2104 delete aRow;
2105 }
2106
2bb7b766 2107 delete aResult;
2108 return kTRUE;
2109}
2110
2111//______________________________________________________________________________________________
eba76848 2112AliShuttleLogbookEntry* AliShuttle::QueryRunParameters(Int_t run)
2bb7b766 2113{
eba76848 2114 //
2115 // Retrieve run parameters written in the DAQ logbook and sets them into AliShuttleLogbookEntry object
2116 //
2bb7b766 2117
2118 // check connection, in case connect
be48e3ea 2119 if (!Connect(3))
eba76848 2120 return 0;
2bb7b766 2121
2122 TString sqlQuery;
2c15234c 2123 sqlQuery.Form("select * from %s where run=%d", fConfig->GetDAQlbTable(), run);
2bb7b766 2124
be48e3ea 2125 TSQLResult* aResult = fServer[3]->Query(sqlQuery);
2bb7b766 2126 if (!aResult) {
c88ad5db 2127 Log("SHUTTLE", Form("Can't execute query <%s>!", sqlQuery.Data()));
eba76848 2128 return 0;
2bb7b766 2129 }
2130
eba76848 2131 if (aResult->GetRowCount() == 0) {
2bb7b766 2132 Log("SHUTTLE", Form("QueryRunParameters - No entry in DAQ Logbook for run %d. Skipping", run));
2133 delete aResult;
eba76848 2134 return 0;
2bb7b766 2135 }
2136
eba76848 2137 if (aResult->GetRowCount() > 1) {
c88ad5db 2138 Log("SHUTTLE", Form("QueryRunParameters - UNEXPECTED: "
2139 "more than one entry in DAQ Logbook for run %d!", run));
2bb7b766 2140 delete aResult;
eba76848 2141 return 0;
2bb7b766 2142 }
2143
eba76848 2144 TSQLRow* aRow = aResult->Next();
2145 if (!aRow)
2146 {
c88ad5db 2147 Log("SHUTTLE", Form("QueryRunParameters - Could not retrieve row for run %d. Skipping", run));
eba76848 2148 delete aResult;
2149 return 0;
2150 }
2bb7b766 2151
eba76848 2152 AliShuttleLogbookEntry* entry = new AliShuttleLogbookEntry(run);
2bb7b766 2153
eba76848 2154 for (Int_t ii = 0; ii < aResult->GetFieldCount(); ii++)
2155 entry->SetRunParameter(aResult->GetFieldName(ii), aRow->GetField(ii));
2bb7b766 2156
eba76848 2157 UInt_t startTime = entry->GetStartTime();
2158 UInt_t endTime = entry->GetEndTime();
2159
1abfbb60 2160// if (!startTime || !endTime || startTime > endTime)
2161// {
2162// Log("SHUTTLE",
2163// Form("QueryRunParameters - Invalid parameters for Run %d: startTime = %d, endTime = %d. Skipping!",
2164// run, startTime, endTime));
2165//
2166// Log("SHUTTLE", Form("Marking SHUTTLE done for run %d", run));
2167// fLogbookEntry = entry;
2168// if (!UpdateShuttleLogbook("shuttle_done"))
2169// {
2170// AliError(Form("Could not update logbook for run %d !", run));
2171// }
2172// fLogbookEntry = 0;
2173//
2174// delete entry;
2175// delete aRow;
2176// delete aResult;
2177// return 0;
2178// }
2179
2180 if (!startTime)
b0e53b15 2181 {
2182 Log("SHUTTLE",
1abfbb60 2183 Form("QueryRunParameters - Invalid parameters for Run %d: "
2184 "startTime = %d, endTime = %d. Skipping!",
2185 run, startTime, endTime));
b0e53b15 2186
2187 Log("SHUTTLE", Form("Marking SHUTTLE done for run %d", run));
2188 fLogbookEntry = entry;
ee6f7523 2189 if (!UpdateShuttleLogbook("shuttle_ignored"))
b0e53b15 2190 {
2191 AliError(Form("Could not update logbook for run %d !", run));
2192 }
2193 fLogbookEntry = 0;
2194
2195 delete entry;
2196 delete aRow;
2197 delete aResult;
2198 return 0;
2199 }
2200
1abfbb60 2201 if (startTime && !endTime)
2202 {
2203 // TODO Here we don't mark SHUTTLE done, because this may mean
2204 //the run is still ongoing!!
2205 Log("SHUTTLE",
2206 Form("QueryRunParameters - Invalid parameters for Run %d: "
2207 "startTime = %d, endTime = %d. Skipping (Shuttle won't be marked as DONE)!",
2208 run, startTime, endTime));
2209
2210 //Log("SHUTTLE", Form("Marking SHUTTLE done for run %d", run));
2211 //fLogbookEntry = entry;
2212 //if (!UpdateShuttleLogbook("shuttle_done"))
2213 //{
2214 // AliError(Form("Could not update logbook for run %d !", run));
2215 //}
2216 //fLogbookEntry = 0;
2217
2218 delete entry;
2219 delete aRow;
2220 delete aResult;
2221 return 0;
2222 }
2223
2224 if (startTime && endTime && (startTime > endTime))
2225 {
2226 Log("SHUTTLE",
2227 Form("QueryRunParameters - Invalid parameters for Run %d: "
2228 "startTime = %d, endTime = %d. Skipping!",
2229 run, startTime, endTime));
2230
2231 Log("SHUTTLE", Form("Marking SHUTTLE done for run %d", run));
2232 fLogbookEntry = entry;
ee6f7523 2233 if (!UpdateShuttleLogbook("shuttle_ignored"))
1abfbb60 2234 {
2235 AliError(Form("Could not update logbook for run %d !", run));
2236 }
2237 fLogbookEntry = 0;
2238
2239 delete entry;
2240 delete aRow;
2241 delete aResult;
2242 return 0;
2243 }
2244
b0e53b15 2245 TString totEventsStr = entry->GetRunParameter("totalEvents");
2246 Int_t totEvents = totEventsStr.Atoi();
2247 if (totEvents < 1)
2248 {
eba76848 2249 Log("SHUTTLE",
b0e53b15 2250 Form("QueryRunParameters - Run %d has 0 events - Skipping!", run));
2251
2252 Log("SHUTTLE", Form("Marking SHUTTLE done for run %d", run));
2253 fLogbookEntry = entry;
97e3c167 2254 if (!UpdateShuttleLogbook("shuttle_ignored"))
b0e53b15 2255 {
2256 AliError(Form("Could not update logbook for run %d !", run));
2257 }
2258 fLogbookEntry = 0;
2259
eba76848 2260 delete entry;
2bb7b766 2261 delete aRow;
eba76848 2262 delete aResult;
2263 return 0;
2bb7b766 2264 }
2265
eba76848 2266 delete aRow;
2bb7b766 2267 delete aResult;
eba76848 2268
2269 return entry;
2bb7b766 2270}
2271
a038aa70 2272//______________________________________________________________________________________________
2273TMap* AliShuttle::GetValueSet(const char* host, Int_t port, const TSeqCollection* entries,
542b6cc8 2274 DCSType type, Int_t multiSplit)
a038aa70 2275{
2276 // Retrieve all "entry" data points from the DCS server
2277 // host, port: TSocket connection parameters
2278 // entries: list of name of the alias or data point
2279 // type: kAlias or kDP
2280 // returns TMap of values, 0 when failure
542b6cc8 2281
2282 AliDCSClient client(host, port, fTimeout, fRetries, multiSplit);
b41b252a 2283
a038aa70 2284 TMap* result = 0;
b41b252a 2285 if (type == kAlias)
a038aa70 2286 {
b41b252a 2287 result = client.GetAliasValues(entries, GetCurrentStartTime(),
2288 GetCurrentEndTime());
2289 }
2290 else if (type == kDP)
2291 {
2292 result = client.GetDPValues(entries, GetCurrentStartTime(),
2293 GetCurrentEndTime());
2294 }
a038aa70 2295
b41b252a 2296 if (result == 0)
2297 {
2298 Log(fCurrentDetector.Data(), Form("GetValueSet - Can't get entries! Reason: %s",
1790d4b7 2299 client.GetErrorString(client.GetResultErrorCode())));
2300 if (client.GetResultErrorCode() == AliDCSClient::fgkServerError)
2301 Log(fCurrentDetector.Data(), Form("GetValueSet - Server error code: %s",
2302 client.GetServerError().Data()));
a038aa70 2303
b41b252a 2304 return 0;
a038aa70 2305 }
b41b252a 2306
a038aa70 2307 return result;
2308}
b41b252a 2309
b948db8d 2310//______________________________________________________________________________________________
57f50b3c 2311const char* AliShuttle::GetFile(Int_t system, const char* detector,
2312 const char* id, const char* source)
b948db8d 2313{
9827400b 2314 // Get calibration file from file exchange servers
2315 // First queris the FXS database for the file name, using the run, detector, id and source info
2316 // then calls RetrieveFile(filename) for actual copy to local disk
2317 // run: current run being processed (given by Logbook entry fLogbookEntry)
2318 // detector: the Preprocessor name
2319 // id: provided as a parameter by the Preprocessor
2320 // source: provided by the Preprocessor through GetFileSources function
2321
2322 // check if test mode should simulate a FXS error
2323 if (fTestMode & kErrorFXSFiles)
2324 {
2325 Log(detector, Form("GetFile - In TESTMODE - Simulating error while connecting to %s FXS", GetSystemName(system)));
2326 return 0;
2327 }
2328
57f50b3c 2329 // check connection, in case connect
9d733021 2330 if (!Connect(system))
eba76848 2331 {
9d733021 2332 Log(detector, Form("GetFile - Couldn't connect to %s FXS database", GetSystemName(system)));
57f50b3c 2333 return 0;
2334 }
2335
2336 // Query preparation
9d733021 2337 TString sourceName(source);
d386d623 2338 Int_t nFields = 3;
2339 TString sqlQueryStart = Form("select filePath,size,fileChecksum from %s where",
2340 fConfig->GetFXSdbTable(system));
2341 TString whereClause = Form("run=%d and detector=\"%s\" and fileId=\"%s\"",
2342 GetCurrentRun(), detector, id);
2343
9d733021 2344 if (system == kDAQ)
2345 {
d386d623 2346 whereClause += Form(" and DAQsource=\"%s\"", source);
57f50b3c 2347 }
9d733021 2348 else if (system == kDCS)
eba76848 2349 {
9d733021 2350 sourceName="none";
57f50b3c 2351 }
9d733021 2352 else if (system == kHLT)
9e080f92 2353 {
d386d623 2354 whereClause += Form(" and DDLnumbers=\"%s\"", source);
9d733021 2355 nFields = 3;
9e080f92 2356 }
2357
9e080f92 2358 TString sqlQuery = Form("%s %s", sqlQueryStart.Data(), whereClause.Data());
2359
2360 AliDebug(2, Form("SQL query: \n%s",sqlQuery.Data()));
2361
2362 // Query execution
2363 TSQLResult* aResult = 0;
9d733021 2364 aResult = dynamic_cast<TSQLResult*> (fServer[system]->Query(sqlQuery));
9e080f92 2365 if (!aResult) {
9d733021 2366 Log(detector, Form("GetFileName - Can't execute SQL query to %s database for: id = %s, source = %s",
2367 GetSystemName(system), id, sourceName.Data()));
9e080f92 2368 return 0;
2369 }
2370
2371 if(aResult->GetRowCount() == 0)
2372 {
2373 Log(detector,
9d733021 2374 Form("GetFileName - No entry in %s FXS db for: id = %s, source = %s",
2375 GetSystemName(system), id, sourceName.Data()));
9e080f92 2376 delete aResult;
2377 return 0;
2378 }
2bb7b766 2379
9e080f92 2380 if (aResult->GetRowCount() > 1) {
2381 Log(detector,
9d733021 2382 Form("GetFileName - More than one entry in %s FXS db for: id = %s, source = %s",
2383 GetSystemName(system), id, sourceName.Data()));
9e080f92 2384 delete aResult;
2385 return 0;
2386 }
2387
9d733021 2388 if (aResult->GetFieldCount() != nFields) {
9e080f92 2389 Log(detector,
9d733021 2390 Form("GetFileName - Wrong field count in %s FXS db for: id = %s, source = %s",
2391 GetSystemName(system), id, sourceName.Data()));
9e080f92 2392 delete aResult;
2393 return 0;
2394 }
2395
2396 TSQLRow* aRow = dynamic_cast<TSQLRow*> (aResult->Next());
2397
2398 if (!aRow){
9d733021 2399 Log(detector, Form("GetFileName - Empty set result in %s FXS db from query: id = %s, source = %s",
2400 GetSystemName(system), id, sourceName.Data()));
9e080f92 2401 delete aResult;
2402 return 0;
2403 }
2404
2405 TString filePath(aRow->GetField(0), aRow->GetFieldLength(0));
2406 TString fileSize(aRow->GetField(1), aRow->GetFieldLength(1));
d386d623 2407 TString fileChecksum(aRow->GetField(2), aRow->GetFieldLength(2));
9e080f92 2408
2409 delete aResult;
2410 delete aRow;
2411
d386d623 2412 AliDebug(2, Form("filePath = %s; size = %s, fileChecksum = %s",
2413 filePath.Data(), fileSize.Data(), fileChecksum.Data()));
9e080f92 2414
9e080f92 2415 // retrieved file is renamed to make it unique
675f64cd 2416 TString localFileName = Form("%s/%s_%d_process/%s_%s_%d_%s_%s.shuttle",
2417 GetShuttleTempDir(), detector, GetCurrentRun(),
d524ade6 2418 GetSystemName(system), detector, GetCurrentRun(),
2419 id, sourceName.Data());
9d733021 2420
9e080f92 2421
9d733021 2422 // file retrieval from FXS
4b95672b 2423 UInt_t nRetries = 0;
2424 UInt_t maxRetries = 3;
2425 Bool_t result = kFALSE;
2426
2427 // copy!! if successful TSystem::Exec returns 0
2428 while(nRetries++ < maxRetries) {
2429 AliDebug(2, Form("Trying to copy file. Retry # %d", nRetries));
2430 result = RetrieveFile(system, filePath.Data(), localFileName.Data());
2431 if(!result)
2432 {
2433 Log(detector, Form("GetFileName - Copy of file %s from %s FXS failed",
9d733021 2434 filePath.Data(), GetSystemName(system)));
4b95672b 2435 continue;
4f0749a8 2436 }
9e080f92 2437
d386d623 2438 if (fileChecksum.Length()>0)
4b95672b 2439 {
2440 // compare md5sum of local file with the one stored in the FXS DB
d524ade6 2441 Int_t md5Comp = gSystem->Exec(Form("md5sum %s |grep %s 2>&1 > /dev/null",
2442 localFileName.Data(), fileChecksum.Data()));
9e080f92 2443
4b95672b 2444 if (md5Comp != 0)
2445 {
2446 Log(detector, Form("GetFileName - md5sum of file %s does not match with local copy!",
2447 filePath.Data()));
2448 result = kFALSE;
2449 continue;
2450 }
d386d623 2451 } else {
2452 Log(fCurrentDetector, Form("GetFile - md5sum of file %s not set in %s database, skipping comparison",
2453 filePath.Data(), GetSystemName(system)));
9d733021 2454 }
4b95672b 2455 if (result) break;
9e080f92 2456 }
2457
4b95672b 2458 if(!result) return 0;
2459
9d733021 2460 fFXSCalled[system]=kTRUE;
2461 TObjString *fileParams = new TObjString(Form("%s#!?!#%s", id, sourceName.Data()));
2462 fFXSlist[system].Add(fileParams);
9e080f92 2463
675f64cd 2464 static TString staticLocalFileName;
2465 staticLocalFileName.Form("%s", localFileName.Data());
2466
c88ad5db 2467 Log(fCurrentDetector, Form("GetFile - Retrieved file with id %s and "
2468 "source %s from %s to %s", id, source,
d524ade6 2469 GetSystemName(system), localFileName.Data()));
675f64cd 2470
d524ade6 2471 return staticLocalFileName.Data();
2bb7b766 2472}
2473
2474//______________________________________________________________________________________________
9d733021 2475Bool_t AliShuttle::RetrieveFile(UInt_t system, const char* fxsFileName, const char* localFileName)
9e080f92 2476{
9827400b 2477 //
2478 // Copies file from FXS to local Shuttle machine
2479 //
2bb7b766 2480
9e080f92 2481 // check temp directory: trying to cd to temp; if it does not exist, create it
d524ade6 2482 AliDebug(2, Form("Copy file %s from %s FXS into %s",
2483 GetSystemName(system), fxsFileName, localFileName));
2484
2485 TString tmpDir(localFileName);
2486
2487 tmpDir = tmpDir(0,tmpDir.Last('/'));
9e080f92 2488
d524ade6 2489 Int_t noDir = gSystem->GetPathInfo(tmpDir.Data(), 0, (Long64_t*) 0, 0, 0);
2490 if (noDir) // temp dir does not exists!
2491 {
2492 if (gSystem->mkdir(tmpDir.Data(), 1))
2493 {
2494 Log(fCurrentDetector.Data(), "RetrieveFile - could not make temp directory!!");
9e080f92 2495 return kFALSE;
2496 }
9e080f92 2497 }
2498
9d733021 2499 TString baseFXSFolder;
2500 if (system == kDAQ)
2501 {
2502 baseFXSFolder = "FES/";
2503 }
2504 else if (system == kDCS)
2505 {
2506 baseFXSFolder = "";
2507 }
2508 else if (system == kHLT)
2509 {
42fde080 2510 baseFXSFolder = "/opt/FXS/";
9d733021 2511 }
2512
2513
d524ade6 2514 TString command = Form("scp -oPort=%d -2 %s@%s:%s%s %s",
9d733021 2515 fConfig->GetFXSPort(system),
2516 fConfig->GetFXSUser(system),
2517 fConfig->GetFXSHost(system),
2518 baseFXSFolder.Data(),
2519 fxsFileName,
9e080f92 2520 localFileName);
2521
2522 AliDebug(2, Form("%s",command.Data()));
2523
4b95672b 2524 Bool_t result = (gSystem->Exec(command.Data()) == 0);
9e080f92 2525
4b95672b 2526 return result;
9e080f92 2527}
2528
2529//______________________________________________________________________________________________
9d733021 2530TList* AliShuttle::GetFileSources(Int_t system, const char* detector, const char* id)
2531{
9827400b 2532 //
2533 // Get sources producing the condition file Id from file exchange servers
4a33bdd9 2534 // if id is NULL all sources are returned (distinct)
9827400b 2535 //
1bcd28db 2536
8884d71e 2537 if (id)
2538 {
2539 Log(detector, Form("GetFileSources - Querying %s FXS for files with id %s produced by %s", GetSystemName(system), id, detector));
2540 } else {
2541 Log(detector, Form("GetFileSources - Querying %s FXS for files produced by %s", GetSystemName(system), detector));
2542 }
9827400b 2543
2544 // check if test mode should simulate a FXS error
2545 if (fTestMode & kErrorFXSSources)
2546 {
2547 Log(detector, Form("GetFileSources - In TESTMODE - Simulating error while connecting to %s FXS", GetSystemName(system)));
2548 return 0;
2549 }
2550
9d733021 2551 if (system == kDCS)
2552 {
c88ad5db 2553 Log(detector, "GetFileSources - WARNING: DCS system has only one source of data!");
6297b37d 2554 TList *list = new TList();
2555 list->SetOwner(1);
2556 list->Add(new TObjString(" "));
2557 return list;
9d733021 2558 }
9e080f92 2559
2560 // check connection, in case connect
9d733021 2561 if (!Connect(system))
2562 {
4a33bdd9 2563 Log(detector, Form("GetFileSources - Couldn't connect to %s FXS database", GetSystemName(system)));
9d733021 2564 return NULL;
9e080f92 2565 }
2566
9d733021 2567 TString sourceName = 0;
2568 if (system == kDAQ)
2569 {
2570 sourceName = "DAQsource";
2571 } else if (system == kHLT)
2572 {
2573 sourceName = "DDLnumbers";
2574 }
2575
4a33bdd9 2576 TString sqlQueryStart = Form("select distinct %s from %s where", sourceName.Data(), fConfig->GetFXSdbTable(system));
2577 TString whereClause = Form("run=%d and detector=\"%s\"",
2578 GetCurrentRun(), detector);
2579 if (id)
2580 whereClause += Form(" and fileId=\"%s\"", id);
9e080f92 2581 TString sqlQuery = Form("%s %s", sqlQueryStart.Data(), whereClause.Data());
2582
2583 AliDebug(2, Form("SQL query: \n%s",sqlQuery.Data()));
2584
2585 // Query execution
2586 TSQLResult* aResult;
9d733021 2587 aResult = fServer[system]->Query(sqlQuery);
9e080f92 2588 if (!aResult) {
9d733021 2589 Log(detector, Form("GetFileSources - Can't execute SQL query to %s database for id: %s",
2590 GetSystemName(system), id));
9e080f92 2591 return 0;
2592 }
2593
86aa42c3 2594 TList *list = new TList();
2595 list->SetOwner(1);
2596
9d733021 2597 if (aResult->GetRowCount() == 0)
2598 {
9e080f92 2599 Log(detector,
9d733021 2600 Form("GetFileSources - No entry in %s FXS table for id: %s", GetSystemName(system), id));
9e080f92 2601 delete aResult;
86aa42c3 2602 return list;
9e080f92 2603 }
2604
1bcd28db 2605 Log(detector, Form("GetFileSources - Found %d sources", aResult->GetRowCount()));
9e080f92 2606
1bcd28db 2607 TSQLRow* aRow;
9d733021 2608 while ((aRow = aResult->Next()))
2609 {
9e080f92 2610
9d733021 2611 TString source(aRow->GetField(0), aRow->GetFieldLength(0));
2612 AliDebug(2, Form("%s = %s", sourceName.Data(), source.Data()));
2613 list->Add(new TObjString(source));
9e080f92 2614 delete aRow;
2615 }
9d733021 2616
9e080f92 2617 delete aResult;
2618
2619 return list;
2bb7b766 2620}
2621
4a33bdd9 2622//______________________________________________________________________________________________
2623TList* AliShuttle::GetFileIDs(Int_t system, const char* detector, const char* source)
2624{
2625 //
2626 // Get all ids of condition files produced by a given source from file exchange servers
2627 //
2628
1bcd28db 2629 Log(detector, Form("GetFileIDs - Retrieving ids with source %s with %s", source, GetSystemName(system)));
2630
4a33bdd9 2631 // check if test mode should simulate a FXS error
2632 if (fTestMode & kErrorFXSSources)
2633 {
2634 Log(detector, Form("GetFileIDs - In TESTMODE - Simulating error while connecting to %s FXS", GetSystemName(system)));
2635 return 0;
2636 }
2637
2638 // check connection, in case connect
2639 if (!Connect(system))
2640 {
2641 Log(detector, Form("GetFileIDs - Couldn't connect to %s FXS database", GetSystemName(system)));
2642 return NULL;
2643 }
2644
2645 TString sourceName = 0;
2646 if (system == kDAQ)
2647 {
2648 sourceName = "DAQsource";
2649 } else if (system == kHLT)
2650 {
2651 sourceName = "DDLnumbers";
2652 }
2653
2654 TString sqlQueryStart = Form("select fileId from %s where", fConfig->GetFXSdbTable(system));
2655 TString whereClause = Form("run=%d and detector=\"%s\"",
2656 GetCurrentRun(), detector);
2657 if (sourceName.Length() > 0 && source)
2658 whereClause += Form(" and %s=\"%s\"", sourceName.Data(), source);
2659 TString sqlQuery = Form("%s %s", sqlQueryStart.Data(), whereClause.Data());
2660
2661 AliDebug(2, Form("SQL query: \n%s",sqlQuery.Data()));
2662
2663 // Query execution
2664 TSQLResult* aResult;
2665 aResult = fServer[system]->Query(sqlQuery);
2666 if (!aResult) {
2667 Log(detector, Form("GetFileIDs - Can't execute SQL query to %s database for source: %s",
2668 GetSystemName(system), source));
2669 return 0;
2670 }
2671
2672 TList *list = new TList();
2673 list->SetOwner(1);
2674
2675 if (aResult->GetRowCount() == 0)
2676 {
2677 Log(detector,
2678 Form("GetFileIDs - No entry in %s FXS table for source: %s", GetSystemName(system), source));
2679 delete aResult;
2680 return list;
2681 }
2682
1bcd28db 2683 Log(detector, Form("GetFileIDs - Found %d ids", aResult->GetRowCount()));
2684
4a33bdd9 2685 TSQLRow* aRow;
2686
2687 while ((aRow = aResult->Next()))
2688 {
2689
2690 TString id(aRow->GetField(0), aRow->GetFieldLength(0));
2691 AliDebug(2, Form("fileId = %s", id.Data()));
2692 list->Add(new TObjString(id));
2693 delete aRow;
2694 }
2695
2696 delete aResult;
2697
2698 return list;
2699}
2700
2bb7b766 2701//______________________________________________________________________________________________
9d733021 2702Bool_t AliShuttle::Connect(Int_t system)
2bb7b766 2703{
9827400b 2704 // Connect to MySQL Server of the system's FXS MySQL databases
2705 // DAQ Logbook, Shuttle Logbook and DAQ FXS db are on the same host
2706 //
57f50b3c 2707
9d733021 2708 // check connection: if already connected return
2709 if(fServer[system] && fServer[system]->IsConnected()) return kTRUE;
57f50b3c 2710
9d733021 2711 TString dbHost, dbUser, dbPass, dbName;
57f50b3c 2712
9d733021 2713 if (system < 3) // FXS db servers
2714 {
2715 dbHost = Form("mysql://%s:%d", fConfig->GetFXSdbHost(system), fConfig->GetFXSdbPort(system));
2716 dbUser = fConfig->GetFXSdbUser(system);
2717 dbPass = fConfig->GetFXSdbPass(system);
2718 dbName = fConfig->GetFXSdbName(system);
2719 } else { // Run & Shuttle logbook servers
2720 // TODO Will the Shuttle logbook server be the same as the Run logbook server ???
2721 dbHost = Form("mysql://%s:%d", fConfig->GetDAQlbHost(), fConfig->GetDAQlbPort());
2722 dbUser = fConfig->GetDAQlbUser();
2723 dbPass = fConfig->GetDAQlbPass();
2724 dbName = fConfig->GetDAQlbDB();
2725 }
57f50b3c 2726
9d733021 2727 fServer[system] = TSQLServer::Connect(dbHost.Data(), dbUser.Data(), dbPass.Data());
2728 if (!fServer[system] || !fServer[system]->IsConnected()) {
2729 if(system < 3)
2730 {
2731 AliError(Form("Can't establish connection to FXS database for %s",
2732 AliShuttleInterface::GetSystemName(system)));
2733 } else {
2734 AliError("Can't establish connection to Run logbook.");
57f50b3c 2735 }
9d733021 2736 if(fServer[system]) delete fServer[system];
2737 return kFALSE;
2bb7b766 2738 }
57f50b3c 2739
9d733021 2740 // Get tables
2741 TSQLResult* aResult=0;
2742 switch(system){
2743 case kDAQ:
2744 aResult = fServer[kDAQ]->GetTables(dbName.Data());
2745 break;
2746 case kDCS:
2747 aResult = fServer[kDCS]->GetTables(dbName.Data());
2748 break;
2749 case kHLT:
2750 aResult = fServer[kHLT]->GetTables(dbName.Data());
2751 break;
2752 default:
2753 aResult = fServer[3]->GetTables(dbName.Data());
2754 break;
2755 }
2756
2757 delete aResult;
2bb7b766 2758 return kTRUE;
2759}
57f50b3c 2760
9e080f92 2761//______________________________________________________________________________________________
9d733021 2762Bool_t AliShuttle::UpdateTable()
9e080f92 2763{
9827400b 2764 //
2765 // Update FXS table filling time_processed field in all rows corresponding to current run and detector
2766 //
9e080f92 2767
9d733021 2768 Bool_t result = kTRUE;
9e080f92 2769
9d733021 2770 for (UInt_t system=0; system<3; system++)
2771 {
2772 if(!fFXSCalled[system]) continue;
9e080f92 2773
9d733021 2774 // check connection, in case connect
2775 if (!Connect(system))
2776 {
2777 Log(fCurrentDetector, Form("UpdateTable - Couldn't connect to %s FXS database", GetSystemName(system)));
2778 result = kFALSE;
2779 continue;
9e080f92 2780 }
9e080f92 2781
9d733021 2782 TTimeStamp now; // now
2783
2784 // Loop on FXS list entries
2785 TIter iter(&fFXSlist[system]);
2786 TObjString *aFXSentry=0;
2787 while ((aFXSentry = dynamic_cast<TObjString*> (iter.Next())))
2788 {
2789 TString aFXSentrystr = aFXSentry->String();
2790 TObjArray *aFXSarray = aFXSentrystr.Tokenize("#!?!#");
2791 if (!aFXSarray || aFXSarray->GetEntries() != 2 )
2792 {
2793 Log(fCurrentDetector, Form("UpdateTable - error updating %s FXS entry. Check string: <%s>",
2794 GetSystemName(system), aFXSentrystr.Data()));
2795 if(aFXSarray) delete aFXSarray;
2796 result = kFALSE;
2797 continue;
2798 }
2799 const char* fileId = ((TObjString*) aFXSarray->At(0))->GetName();
2800 const char* source = ((TObjString*) aFXSarray->At(1))->GetName();
2801
2802 TString whereClause;
2803 if (system == kDAQ)
2804 {
2805 whereClause = Form("where run=%d and detector=\"%s\" and fileId=\"%s\" and DAQsource=\"%s\";",
2806 GetCurrentRun(), fCurrentDetector.Data(), fileId, source);
2807 }
2808 else if (system == kDCS)
2809 {
2810 whereClause = Form("where run=%d and detector=\"%s\" and fileId=\"%s\";",
2811 GetCurrentRun(), fCurrentDetector.Data(), fileId);
2812 }
2813 else if (system == kHLT)
2814 {
2815 whereClause = Form("where run=%d and detector=\"%s\" and fileId=\"%s\" and DDLnumbers=\"%s\";",
2816 GetCurrentRun(), fCurrentDetector.Data(), fileId, source);
2817 }
2818
2819 delete aFXSarray;
9e080f92 2820
9d733021 2821 TString sqlQuery = Form("update %s set time_processed=%d %s", fConfig->GetFXSdbTable(system),
2822 now.GetSec(), whereClause.Data());
9e080f92 2823
9d733021 2824 AliDebug(2, Form("SQL query: \n%s",sqlQuery.Data()));
9e080f92 2825
9d733021 2826 // Query execution
2827 TSQLResult* aResult;
2828 aResult = dynamic_cast<TSQLResult*> (fServer[system]->Query(sqlQuery));
2829 if (!aResult)
2830 {
2831 Log(fCurrentDetector, Form("UpdateTable - %s db: can't execute SQL query <%s>",
2832 GetSystemName(system), sqlQuery.Data()));
2833 result = kFALSE;
2834 continue;
2835 }
2836 delete aResult;
9e080f92 2837 }
9e080f92 2838 }
2839
9d733021 2840 return result;
9e080f92 2841}
57f50b3c 2842
3301427a 2843//______________________________________________________________________________________________
2844Bool_t AliShuttle::UpdateTableFailCase()
2845{
9827400b 2846 // Update FXS table filling time_processed field in all rows corresponding to current run and detector
2847 // this is called in case the preprocessor is declared failed for the current run, because
2848 // the fields are updated only in case of success
3301427a 2849
2850 Bool_t result = kTRUE;
2851
2852 for (UInt_t system=0; system<3; system++)
2853 {
2854 // check connection, in case connect
2855 if (!Connect(system))
2856 {
2857 Log(fCurrentDetector, Form("UpdateTableFailCase - Couldn't connect to %s FXS database",
2858 GetSystemName(system)));
2859 result = kFALSE;
2860 continue;
2861 }
2862
2863 TTimeStamp now; // now
2864
2865 // Loop on FXS list entries
2866
2867 TString whereClause = Form("where run=%d and detector=\"%s\";",
2868 GetCurrentRun(), fCurrentDetector.Data());
2869
2870
2871 TString sqlQuery = Form("update %s set time_processed=%d %s", fConfig->GetFXSdbTable(system),
2872 now.GetSec(), whereClause.Data());
2873
2874 AliDebug(2, Form("SQL query: \n%s",sqlQuery.Data()));
2875
2876 // Query execution
2877 TSQLResult* aResult;
2878 aResult = dynamic_cast<TSQLResult*> (fServer[system]->Query(sqlQuery));
2879 if (!aResult)
2880 {
2881 Log(fCurrentDetector, Form("UpdateTableFailCase - %s db: can't execute SQL query <%s>",
2882 GetSystemName(system), sqlQuery.Data()));
2883 result = kFALSE;
2884 continue;
2885 }
2886 delete aResult;
2887 }
2888
2889 return result;
2890}
2891
2bb7b766 2892//______________________________________________________________________________________________
2893Bool_t AliShuttle::UpdateShuttleLogbook(const char* detector, const char* status)
2894{
e7f62f16 2895 //
2896 // Update Shuttle logbook filling detector or shuttle_done column
2897 // ex. of usage: UpdateShuttleLogbook("PHOS", "DONE") or UpdateShuttleLogbook("shuttle_done")
2898 //
57f50b3c 2899
2bb7b766 2900 // check connection, in case connect
be48e3ea 2901 if(!Connect(3)){
2bb7b766 2902 Log("SHUTTLE", "UpdateShuttleLogbook - Couldn't connect to DAQ Logbook.");
2903 return kFALSE;
57f50b3c 2904 }
2905
2bb7b766 2906 TString detName(detector);
2907 TString setClause;
ee6f7523 2908 if (detName == "shuttle_done" || detName == "shuttle_ignored")
e7f62f16 2909 {
2bb7b766 2910 setClause = "set shuttle_done=1";
e7f62f16 2911
ee6f7523 2912 if (detName == "shuttle_done")
b0e53b15 2913 {
2914 // Send the information to ML
2915 TMonaLisaText mlStatus("SHUTTLE_status", "Done");
e7f62f16 2916
b0e53b15 2917 TList mlList;
2918 mlList.Add(&mlStatus);
2919
ee6f7523 2920 TString mlID;
2921 mlID.Form("%d", GetCurrentRun());
2922 fMonaLisa->SendParameters(&mlList, mlID);
b0e53b15 2923 }
2bb7b766 2924 } else {
2bb7b766 2925 TString statusStr(status);
2926 if(statusStr.Contains("done", TString::kIgnoreCase) ||
2927 statusStr.Contains("failed", TString::kIgnoreCase)){
eba76848 2928 setClause = Form("set %s=\"%s\"", detector, status);
2bb7b766 2929 } else {
2930 Log("SHUTTLE",
2931 Form("UpdateShuttleLogbook - Invalid status <%s> for detector %s",
2932 status, detector));
2933 return kFALSE;
2934 }
2935 }
57f50b3c 2936
2bb7b766 2937 TString whereClause = Form("where run=%d", GetCurrentRun());
2938
441b0e9c 2939 TString sqlQuery = Form("update %s %s %s",
2940 fConfig->GetShuttlelbTable(), setClause.Data(), whereClause.Data());
57f50b3c 2941
2bb7b766 2942 AliDebug(2, Form("SQL query: \n%s",sqlQuery.Data()));
2943
2944 // Query execution
2945 TSQLResult* aResult;
be48e3ea 2946 aResult = dynamic_cast<TSQLResult*> (fServer[3]->Query(sqlQuery));
2bb7b766 2947 if (!aResult) {
2948 Log("SHUTTLE", Form("UpdateShuttleLogbook - Can't execute query <%s>", sqlQuery.Data()));
2949 return kFALSE;
57f50b3c 2950 }
2bb7b766 2951 delete aResult;
57f50b3c 2952
2953 return kTRUE;
2954}
2955
2956//______________________________________________________________________________________________
2bb7b766 2957Int_t AliShuttle::GetCurrentRun() const
2958{
9827400b 2959 //
2960 // Get current run from logbook entry
2961 //
57f50b3c 2962
2bb7b766 2963 return fLogbookEntry ? fLogbookEntry->GetRun() : -1;
57f50b3c 2964}
2965
2966//______________________________________________________________________________________________
2bb7b766 2967UInt_t AliShuttle::GetCurrentStartTime() const
2968{
9827400b 2969 //
2970 // get current start time
2971 //
57f50b3c 2972
2bb7b766 2973 return fLogbookEntry ? fLogbookEntry->GetStartTime() : 0;
57f50b3c 2974}
2975
2976//______________________________________________________________________________________________
2bb7b766 2977UInt_t AliShuttle::GetCurrentEndTime() const
2978{
9827400b 2979 //
2980 // get current end time from logbook entry
2981 //
57f50b3c 2982
2bb7b766 2983 return fLogbookEntry ? fLogbookEntry->GetEndTime() : 0;
57f50b3c 2984}
2985
675f64cd 2986//______________________________________________________________________________________________
2987UInt_t AliShuttle::GetCurrentYear() const
2988{
2989 //
2990 // Get current year from logbook entry
2991 //
2992
2993 if (!fLogbookEntry) return 0;
2994
2995 TTimeStamp startTime(GetCurrentStartTime());
2996 TString year = Form("%d",startTime.GetDate());
2997 year = year(0,4);
2998
2999 return year.Atoi();
3000}
3001
3002//______________________________________________________________________________________________
3003const char* AliShuttle::GetLHCPeriod() const
3004{
3005 //
3006 // Get current LHC period from logbook entry
3007 //
3008
3009 if (!fLogbookEntry) return 0;
3010
3011 return fLogbookEntry->GetRunParameter("LHCperiod");
3012}
3013
b948db8d 3014//______________________________________________________________________________________________
3015void AliShuttle::Log(const char* detector, const char* message)
3016{
9827400b 3017 //
3018 // Fill log string with a message
3019 //
b948db8d 3020
7d4cf768 3021 TString logRunDir = GetShuttleLogDir();
3022 if (GetCurrentRun() >=0)
3023 logRunDir += Form("/%d", GetCurrentRun());
3024
3025 void* dir = gSystem->OpenDirectory(logRunDir.Data());
84090f85 3026 if (dir == NULL) {
7d4cf768 3027 if (gSystem->mkdir(logRunDir.Data(), kTRUE)) {
36c99a6a 3028 AliError(Form("Can't open directory <%s>", GetShuttleLogDir()));
84090f85 3029 return;
3030 }
b948db8d 3031
84090f85 3032 } else {
3033 gSystem->FreeDirectory(dir);
3034 }
b948db8d 3035
cb343cfd 3036 TString toLog = Form("%s (%d): %s - ", TTimeStamp(time(0)).AsString("s"), getpid(), detector);
e7f62f16 3037 if (GetCurrentRun() >= 0)
3038 toLog += Form("run %d - ", GetCurrentRun());
2bb7b766 3039 toLog += Form("%s", message);
3040
84090f85 3041 AliInfo(toLog.Data());
ffa29e93 3042
3043 // if we redirect the log output already to the file, leave here
3044 if (fOutputRedirected && strcmp(detector, "SHUTTLE") != 0)
3045 return;
b948db8d 3046
ffa29e93 3047 TString fileName = GetLogFileName(detector);
e7f62f16 3048
84090f85 3049 gSystem->ExpandPathName(fileName);
3050
3051 ofstream logFile;
3052 logFile.open(fileName, ofstream::out | ofstream::app);
3053
3054 if (!logFile.is_open()) {
3055 AliError(Form("Could not open file %s", fileName.Data()));
3056 return;
3057 }
7bfb2090 3058
84090f85 3059 logFile << toLog.Data() << "\n";
b948db8d 3060
84090f85 3061 logFile.close();
b948db8d 3062}
2bb7b766 3063
ffa29e93 3064//______________________________________________________________________________________________
3065TString AliShuttle::GetLogFileName(const char* detector) const
3066{
3067 //
3068 // returns the name of the log file for a given sub detector
3069 //
3070
3071 TString fileName;
3072
3073 if (GetCurrentRun() >= 0)
7d4cf768 3074 {
3075 fileName.Form("%s/%d/%s_%d.log", GetShuttleLogDir(), GetCurrentRun(),
3076 detector, GetCurrentRun());
3077 } else {
ffa29e93 3078 fileName.Form("%s/%s.log", GetShuttleLogDir(), detector);
7d4cf768 3079 }
ffa29e93 3080
3081 return fileName;
3082}
3083
ee6f7523 3084//______________________________________________________________________________________________
3085void AliShuttle::SendAlive()
3086{
3087 // sends alive message to ML
3088
3089 TMonaLisaText mlStatus("SHUTTLE_status", "Alive");
3090
3091 TList mlList;
3092 mlList.Add(&mlStatus);
3093
3094 fMonaLisa->SendParameters(&mlList, "__PROCESSINGINFO__");
3095}
3096
2bb7b766 3097//______________________________________________________________________________________________
3098Bool_t AliShuttle::Collect(Int_t run)
3099{
9827400b 3100 //
3101 // Collects conditions data for all UNPROCESSED run written to DAQ LogBook in case of run = -1 (default)
3102 // If a dedicated run is given this run is processed
3103 //
3104 // In operational mode, this is the Shuttle function triggered by the EOR signal.
3105 //
2bb7b766 3106
eba76848 3107 if (run == -1)
3108 Log("SHUTTLE","Collect - Shuttle called. Collecting conditions data for unprocessed runs");
3109 else
3110 Log("SHUTTLE", Form("Collect - Shuttle called. Collecting conditions data for run %d", run));
cb343cfd 3111
3112 SetLastAction("Starting");
2bb7b766 3113
ee6f7523 3114 // create ML instance
3115 if (!fMonaLisa)
3116 fMonaLisa = new TMonaLisaWriter(fConfig->GetMonitorHost(), fConfig->GetMonitorTable());
3117
ee6f7523 3118 SendAlive();
fbc112e3 3119 CountOpenRuns();
ee6f7523 3120
2bb7b766 3121 TString whereClause("where shuttle_done=0");
eba76848 3122 if (run != -1)
3123 whereClause += Form(" and run=%d", run);
2bb7b766 3124
3125 TObjArray shuttleLogbookEntries;
be48e3ea 3126 if (!QueryShuttleLogbook(whereClause, shuttleLogbookEntries))
3127 {
cb343cfd 3128 Log("SHUTTLE", "Collect - Can't retrieve entries from Shuttle logbook");
2bb7b766 3129 return kFALSE;
3130 }
3131
9e080f92 3132 if (shuttleLogbookEntries.GetEntries() == 0)
3133 {
3134 if (run == -1)
3135 Log("SHUTTLE","Collect - Found no UNPROCESSED runs in Shuttle logbook");
3136 else
3137 Log("SHUTTLE", Form("Collect - Run %d is already DONE "
3138 "or it does not exist in Shuttle logbook", run));
3139 return kTRUE;
3140 }
3141
be48e3ea 3142 for (UInt_t iDet=0; iDet<NDetectors(); iDet++)
3143 fFirstUnprocessed[iDet] = kTRUE;
3144
fc5a4708 3145 if (run != -1)
be48e3ea 3146 {
3147 // query Shuttle logbook for earlier runs, check if some detectors are unprocessed,
3148 // flag them into fFirstUnprocessed array
3149 TString whereClause(Form("where shuttle_done=0 and run < %d", run));
3150 TObjArray tmpLogbookEntries;
3151 if (!QueryShuttleLogbook(whereClause, tmpLogbookEntries))
3152 {
3153 Log("SHUTTLE", "Collect - Can't retrieve entries from Shuttle logbook");
3154 return kFALSE;
3155 }
3156
3157 TIter iter(&tmpLogbookEntries);
3158 AliShuttleLogbookEntry* anEntry = 0;
3159 while ((anEntry = dynamic_cast<AliShuttleLogbookEntry*> (iter.Next())))
3160 {
3161 for (UInt_t iDet=0; iDet<NDetectors(); iDet++)
3162 {
3163 if (anEntry->GetDetectorStatus(iDet) == AliShuttleLogbookEntry::kUnprocessed)
3164 {
3165 AliDebug(2, Form("Run %d: setting %s as \"not first time unprocessed\"",
3166 anEntry->GetRun(), GetDetName(iDet)));
3167 fFirstUnprocessed[iDet] = kFALSE;
3168 }
3169 }
3170
3171 }
3172
3173 }
3174
3175 if (!RetrieveConditionsData(shuttleLogbookEntries))
3176 {
cb343cfd 3177 Log("SHUTTLE", "Collect - Process of at least one run failed");
a3ca69fe 3178 CountOpenRuns();
2bb7b766 3179 return kFALSE;
3180 }
3181
36c99a6a 3182 Log("SHUTTLE", "Collect - Requested run(s) successfully processed");
a3ca69fe 3183 CountOpenRuns();
eba76848 3184 return kTRUE;
2bb7b766 3185}
3186
2bb7b766 3187//______________________________________________________________________________________________
3188Bool_t AliShuttle::RetrieveConditionsData(const TObjArray& dateEntries)
3189{
9827400b 3190 //
3191 // Retrieve conditions data for all runs that aren't processed yet
3192 //
2bb7b766 3193
3194 Bool_t hasError = kFALSE;
3195
3196 TIter iter(&dateEntries);
3197 AliShuttleLogbookEntry* anEntry;
3198
3199 while ((anEntry = (AliShuttleLogbookEntry*) iter.Next())){
3200 if (!Process(anEntry)){
3201 hasError = kTRUE;
3202 }
4b95672b 3203
3204 // clean SHUTTLE temp directory
d524ade6 3205 //TString filename = Form("%s/*.shuttle", GetShuttleTempDir());
3206 //RemoveFile(filename.Data());
2bb7b766 3207 }
3208
3209 return hasError == kFALSE;
3210}
cb343cfd 3211
3212//______________________________________________________________________________________________
3213ULong_t AliShuttle::GetTimeOfLastAction() const
3214{
9827400b 3215 //
3216 // Gets time of last action
3217 //
3218
cb343cfd 3219 ULong_t tmp;
36c99a6a 3220
cb343cfd 3221 fMonitoringMutex->Lock();
be48e3ea 3222
cb343cfd 3223 tmp = fLastActionTime;
36c99a6a 3224
cb343cfd 3225 fMonitoringMutex->UnLock();
36c99a6a 3226
cb343cfd 3227 return tmp;
3228}
3229
3230//______________________________________________________________________________________________
3231const TString AliShuttle::GetLastAction() const
3232{
9827400b 3233 //
cb343cfd 3234 // returns a string description of the last action
9827400b 3235 //
cb343cfd 3236
3237 TString tmp;
36c99a6a 3238
cb343cfd 3239 fMonitoringMutex->Lock();
3240
3241 tmp = fLastAction;
3242
3243 fMonitoringMutex->UnLock();
3244
36c99a6a 3245 return tmp;
cb343cfd 3246}
3247
3248//______________________________________________________________________________________________
3249void AliShuttle::SetLastAction(const char* action)
3250{
9827400b 3251 //
cb343cfd 3252 // updates the monitoring variables
9827400b 3253 //
36c99a6a 3254
cb343cfd 3255 fMonitoringMutex->Lock();
36c99a6a 3256
cb343cfd 3257 fLastAction = action;
3258 fLastActionTime = time(0);
3259
3260 fMonitoringMutex->UnLock();
3261}
eba76848 3262
3263//______________________________________________________________________________________________
3264const char* AliShuttle::GetRunParameter(const char* param)
3265{
9827400b 3266 //
3267 // returns run parameter read from DAQ logbook
3268 //
eba76848 3269
3270 if(!fLogbookEntry) {
3271 AliError("No logbook entry!");
3272 return 0;
3273 }
3274
3275 return fLogbookEntry->GetRunParameter(param);
3276}
57c1a579 3277
d386d623 3278//______________________________________________________________________________________________
9827400b 3279AliCDBEntry* AliShuttle::GetFromOCDB(const char* detector, const AliCDBPath& path)
d386d623 3280{
9827400b 3281 //
3282 // returns object from OCDB valid for current run
3283 //
d386d623 3284
9827400b 3285 if (fTestMode & kErrorOCDB)
3286 {
3287 Log(detector, "GetFromOCDB - In TESTMODE - Simulating error with OCDB");
3288 return 0;
3289 }
3290
d386d623 3291 AliCDBStorage *sto = AliCDBManager::Instance()->GetStorage(fgkMainCDB);
3292 if (!sto)
3293 {
9827400b 3294 Log(detector, "GetFromOCDB - Cannot activate main OCDB for query!");
d386d623 3295 return 0;
3296 }
3297
3298 return dynamic_cast<AliCDBEntry*> (sto->Get(path, GetCurrentRun()));
3299}
3300
57c1a579 3301//______________________________________________________________________________________________
3302Bool_t AliShuttle::SendMail()
3303{
9827400b 3304 //
3305 // sends a mail to the subdetector expert in case of preprocessor error
3306 //
3307
3308 if (fTestMode != kNone)
3309 return kTRUE;
57c1a579 3310
57c1a579 3311 TString to="";
3312 TIter iterExperts(fConfig->GetResponsibles(fCurrentDetector));
3313 TObjString *anExpert=0;
3314 while ((anExpert = (TObjString*) iterExperts.Next()))
3315 {
3316 to += Form("%s,", anExpert->GetName());
3317 }
4508ab8b 3318 if (to.Length() > 0)
3319 to.Remove(to.Length()-1);
909732f7 3320 AliDebug(2, Form("to: %s",to.Data()));
57c1a579 3321
86aa42c3 3322 if (to.IsNull()) {
c88ad5db 3323 Log("SHUTTLE", "List of detector responsibles not yet set!");
36c99a6a 3324 return kFALSE;
3325 }
3326
fbc112e3 3327 void* dir = gSystem->OpenDirectory(GetShuttleLogDir());
3328 if (dir == NULL)
3329 {
3330 if (gSystem->mkdir(GetShuttleLogDir(), kTRUE))
3331 {
3332 Log("SHUTTLE", Form("SendMail - Can't open directory <%s>", GetShuttleLogDir()));
3333 return kFALSE;
3334 }
3335
3336 } else {
3337 gSystem->FreeDirectory(dir);
3338 }
3339
4508ab8b 3340 TString bodyFileName;
3341 bodyFileName.Form("%s/mail.body", GetShuttleLogDir());
3342 gSystem->ExpandPathName(bodyFileName);
3343
3344 ofstream mailBody;
3345 mailBody.open(bodyFileName, ofstream::out);
3346
3347 if (!mailBody.is_open())
3348 {
3349 Log("SHUTTLE", Form("Could not open mail body file %s", bodyFileName.Data()));
3350 return kFALSE;
3351 }
3352
57c1a579 3353 TString cc="alberto.colla@cern.ch";
3354
6a1146c4 3355 TString subject = Form("%s Shuttle preprocessor FAILED in run %d (run type = %s)!",
3356 fCurrentDetector.Data(), GetCurrentRun(), GetRunType());
909732f7 3357 AliDebug(2, Form("subject: %s", subject.Data()));
57c1a579 3358
3359 TString body = Form("Dear %s expert(s), \n\n", fCurrentDetector.Data());
3360 body += Form("SHUTTLE just detected that your preprocessor "
6a1146c4 3361 "failed processing run %d (run type = %s)!!\n\n",
3362 GetCurrentRun(), GetRunType());
7d4cf768 3363 body += Form("Please check %s status on the SHUTTLE monitoring page: \n\n",
3364 fCurrentDetector.Data());
b0e53b15 3365 if (fConfig->GetRunMode() == AliShuttleConfig::kTest)
3366 {
3367 body += Form("\thttp://pcalimonitor.cern.ch:8889/shuttle.jsp?time=168 \n\n");
3368 } else {
4508ab8b 3369 body += Form("\thttp://pcalimonitor.cern.ch/shuttle.jsp?instance=PROD&time=168 \n\n");
b0e53b15 3370 }
3371
7d4cf768 3372
3373 TString logFolder = "logs";
3374 if (fConfig->GetRunMode() == AliShuttleConfig::kProd)
3375 logFolder += "_PROD";
3376
3377
546242fb 3378 body += Form("Find the %s log for the current run on \n\n"
7d4cf768 3379 "\thttp://pcalishuttle01.cern.ch:8880/%s/%d/%s_%d.log \n\n",
3380 fCurrentDetector.Data(), logFolder.Data(), GetCurrentRun(),
3381 fCurrentDetector.Data(), GetCurrentRun());
3382 body += Form("The last 10 lines of %s log file are following:\n\n", fCurrentDetector.Data());
57c1a579 3383
909732f7 3384 AliDebug(2, Form("Body begin: %s", body.Data()));
57c1a579 3385
3386 mailBody << body.Data();
3387 mailBody.close();
3388 mailBody.open(bodyFileName, ofstream::out | ofstream::app);
3389
7d4cf768 3390 TString logFileName = Form("%s/%d/%s_%d.log", GetShuttleLogDir(),
3391 GetCurrentRun(), fCurrentDetector.Data(), GetCurrentRun());
57c1a579 3392 TString tailCommand = Form("tail -n 10 %s >> %s", logFileName.Data(), bodyFileName.Data());
3393 if (gSystem->Exec(tailCommand.Data()))
3394 {
3395 mailBody << Form("%s log file not found ...\n\n", fCurrentDetector.Data());
3396 }
3397
3398 TString endBody = Form("------------------------------------------------------\n\n");
675f64cd 3399 endBody += Form("In case of problems please contact the SHUTTLE core team.\n\n");
3400 endBody += "Please do not answer this message directly, it is automatically generated.\n\n";
3401 endBody += "Greetings,\n\n \t\t\tthe SHUTTLE\n";
3402
3403 AliDebug(2, Form("Body end: %s", endBody.Data()));
3404
3405 mailBody << endBody.Data();
3406
3407 mailBody.close();
3408
3409 // send mail!
3410 TString mailCommand = Form("mail -s \"%s\" -c %s %s < %s",
3411 subject.Data(),
3412 cc.Data(),
3413 to.Data(),
3414 bodyFileName.Data());
3415 AliDebug(2, Form("mail command: %s", mailCommand.Data()));
3416
3417 Bool_t result = gSystem->Exec(mailCommand.Data());
3418
3419 return result == 0;
3420}
3421
3422//______________________________________________________________________________________________
3423Bool_t AliShuttle::SendMailToDCS()
3424{
3425 //
3426 // sends a mail to the DCS experts in case of DCS error
3427 //
3428
3429 if (fTestMode != kNone)
3430 return kTRUE;
3431
3432 void* dir = gSystem->OpenDirectory(GetShuttleLogDir());
3433 if (dir == NULL)
3434 {
3435 if (gSystem->mkdir(GetShuttleLogDir(), kTRUE))
3436 {
3437 Log("SHUTTLE", Form("SendMailToDCS - Can't open directory <%s>", GetShuttleLogDir()));
3438 return kFALSE;
3439 }
3440
3441 } else {
3442 gSystem->FreeDirectory(dir);
3443 }
3444
3445 TString bodyFileName;
3446 bodyFileName.Form("%s/mail.body", GetShuttleLogDir());
3447 gSystem->ExpandPathName(bodyFileName);
3448
3449 ofstream mailBody;
3450 mailBody.open(bodyFileName, ofstream::out);
3451
3452 if (!mailBody.is_open())
3453 {
3454 Log("SHUTTLE", Form("SendMailToDCS - Could not open mail body file %s", bodyFileName.Data()));
3455 return kFALSE;
3456 }
3457
3458 TString to="Vladimir.Fekete@cern.ch, Svetozar.Kapusta@cern.ch";
3459 //TString to="alberto.colla@cern.ch";
3460 AliDebug(2, Form("to: %s",to.Data()));
3461
3462 if (to.IsNull()) {
3463 Log("SHUTTLE", "List of detector responsibles not yet set!");
3464 return kFALSE;
3465 }
3466
3467 TString cc="alberto.colla@cern.ch";
3468
3469 TString subject = Form("Retrieval of data points for %s FAILED in run %d !",
3470 fCurrentDetector.Data(), GetCurrentRun());
3471 AliDebug(2, Form("subject: %s", subject.Data()));
3472
3473 TString body = Form("Dear DCS experts, \n\n");
3474 body += Form("SHUTTLE couldn\'t retrieve the data points for detector %s "
3475 "in run %d!!\n\n", fCurrentDetector.Data(), GetCurrentRun());
7d4cf768 3476 body += Form("Please check %s status on the SHUTTLE monitoring page: \n\n",
3477 fCurrentDetector.Data());
b0e53b15 3478 if (fConfig->GetRunMode() == AliShuttleConfig::kTest)
3479 {
3480 body += Form("\thttp://pcalimonitor.cern.ch:8889/shuttle.jsp?time=168 \n\n");
3481 } else {
3482 body += Form("\thttp://pcalimonitor.cern.ch/shuttle.jsp?instance=PROD?time=168 \n\n");
3483 }
7d4cf768 3484
3485 TString logFolder = "logs";
3486 if (fConfig->GetRunMode() == AliShuttleConfig::kProd)
3487 logFolder += "_PROD";
3488
3489
675f64cd 3490 body += Form("Find the %s log for the current run on \n\n"
7d4cf768 3491 "\thttp://pcalishuttle01.cern.ch:8880/%s/%d/%s_%d.log \n\n",
3492 fCurrentDetector.Data(), logFolder.Data(), GetCurrentRun(),
3493 fCurrentDetector.Data(), GetCurrentRun());
3494 body += Form("The last 10 lines of %s log file are following:\n\n", fCurrentDetector.Data());
675f64cd 3495
3496 AliDebug(2, Form("Body begin: %s", body.Data()));
3497
3498 mailBody << body.Data();
3499 mailBody.close();
3500 mailBody.open(bodyFileName, ofstream::out | ofstream::app);
3501
7d4cf768 3502 TString logFileName = Form("%s/%d/%s_%d.log", GetShuttleLogDir(), GetCurrentRun(),
3503 fCurrentDetector.Data(), GetCurrentRun());
675f64cd 3504 TString tailCommand = Form("tail -n 10 %s >> %s", logFileName.Data(), bodyFileName.Data());
3505 if (gSystem->Exec(tailCommand.Data()))
3506 {
3507 mailBody << Form("%s log file not found ...\n\n", fCurrentDetector.Data());
3508 }
3509
3510 TString endBody = Form("------------------------------------------------------\n\n");
36c99a6a 3511 endBody += Form("In case of problems please contact the SHUTTLE core team.\n\n");
3512 endBody += "Please do not answer this message directly, it is automatically generated.\n\n";
546242fb 3513 endBody += "Greetings,\n\n \t\t\tthe SHUTTLE\n";
57c1a579 3514
909732f7 3515 AliDebug(2, Form("Body end: %s", endBody.Data()));
57c1a579 3516
3517 mailBody << endBody.Data();
3518
3519 mailBody.close();
3520
3521 // send mail!
3522 TString mailCommand = Form("mail -s \"%s\" -c %s %s < %s",
3523 subject.Data(),
3524 cc.Data(),
3525 to.Data(),
3526 bodyFileName.Data());
909732f7 3527 AliDebug(2, Form("mail command: %s", mailCommand.Data()));
57c1a579 3528
3529 Bool_t result = gSystem->Exec(mailCommand.Data());
3530
3531 return result == 0;
3532}
d386d623 3533
441b0e9c 3534//______________________________________________________________________________________________
9827400b 3535const char* AliShuttle::GetRunType()
441b0e9c 3536{
9827400b 3537 //
3538 // returns run type read from "run type" logbook
3539 //
441b0e9c 3540
3541 if(!fLogbookEntry) {
3542 AliError("No logbook entry!");
3543 return 0;
3544 }
3545
9827400b 3546 return fLogbookEntry->GetRunType();
441b0e9c 3547}
3548
4859271b 3549//______________________________________________________________________________________________
3550Bool_t AliShuttle::GetHLTStatus()
3551{
3552 // Return HLT status (ON=1 OFF=0)
3553 // Converts the HLT status from the status string read in the run logbook (not just a bool)
3554
3555 if(!fLogbookEntry) {
3556 AliError("No logbook entry!");
3557 return 0;
3558 }
3559
3560 // TODO implement when HLTStatus is inserted in run logbook
3561 //TString hltStatus = fLogbookEntry->GetRunParameter("HLTStatus");
3562 //if(hltStatus == "OFF") {return kFALSE};
3563
3564 return kTRUE;
3565}
3566
d386d623 3567//______________________________________________________________________________________________
3568void AliShuttle::SetShuttleTempDir(const char* tmpDir)
3569{
9827400b 3570 //
3571 // sets Shuttle temp directory
3572 //
d386d623 3573
3574 fgkShuttleTempDir = gSystem->ExpandPathName(tmpDir);
3575}
3576
3577//______________________________________________________________________________________________
3578void AliShuttle::SetShuttleLogDir(const char* logDir)
3579{
9827400b 3580 //
3581 // sets Shuttle log directory
3582 //
d386d623 3583
3584 fgkShuttleLogDir = gSystem->ExpandPathName(logDir);
3585}