]> git.uio.no Git - u/mrichter/AliRoot.git/blame - SHUTTLE/AliShuttle.cxx
git://git.uio.no / u / mrichter / AliRoot.git / blame
? search:
summary | shortlog | log | commit | commitdiff | tree
blob | blame (incremental) | history | HEAD
Updated GRP preprocessor (to be validated with Panos)
[u/mrichter/AliRoot.git] / SHUTTLE / AliShuttle.cxx
CommitLineData
73abe331 1/**************************************************************************
2 * Copyright(c) 1998-1999, ALICE Experiment at CERN, All rights reserved. *
3 * *
4 * Author: The ALICE Off-line Project. *
5 * Contributors are mentioned in the code where appropriate. *
6 * *
7 * Permission to use, copy, modify and distribute this software and its *
8 * documentation strictly for non-commercial purposes is hereby granted *
9 * without fee, provided that the above copyright notice appears in all *
10 * copies and that both the copyright notice and this permission notice *
11 * appear in the supporting documentation. The authors make no claims *
12 * about the suitability of this software for any purpose. It is *
13 * provided "as is" without express or implied warranty. *
14 **************************************************************************/
15
16/*
17$Log$
c88ad5db 18Revision 1.59 2007/10/05 12:40:55 acolla
19
20Result error code added to AliDCSClient data members (it was "lost" with the new implementation of TMap* GetAliasValues and GetDPValues).
21
1790d4b7 22Revision 1.58 2007/09/28 15:27:40 acolla
23
24AliDCSClient "multiSplit" option added in the DCS configuration
25in AliDCSMessage: variable MAX_BODY_SIZE set to 500000
26
542b6cc8 27Revision 1.57 2007/09/27 16:53:13 acolla
28Detectors can have more than one AMANDA server. SHUTTLE queries the servers sequentially,
29merges the dcs aliases/DPs in one TMap and sends it to the preprocessor.
30
1d172743 31Revision 1.56 2007/09/14 16:46:14 jgrosseo
321) Connect and Close are called before and after each query, so one can
33keep the same AliDCSClient object.
342) The splitting of a query is moved to GetDPValues/GetAliasValues.
353) Splitting interval can be specified in constructor
36
b41b252a 37Revision 1.55 2007/08/06 12:26:40 acolla
38Function Bool_t GetHLTStatus added to preprocessor. It returns the status of HLT
39read from the run logbook.
40
4859271b 41Revision 1.54 2007/07/12 09:51:25 jgrosseo
42removed duplicated log message in GetFile
43
4f0749a8 44Revision 1.53 2007/07/12 09:26:28 jgrosseo
45updating hlt fxs base path
46
42fde080 47Revision 1.52 2007/07/12 08:06:45 jgrosseo
48adding log messages in getfile... functions
49adding not implemented copy constructor in alishuttleconfigholder
50
1bcd28db 51Revision 1.51 2007/07/03 17:24:52 acolla
52root moved to v5-16-00. TFileMerger->Cp moved to TFile::Cp.
53
a986b218 54Revision 1.50 2007/07/02 17:19:32 acolla
55preprocessor is run in a temp directory that is removed when process is finished.
56
5bac2bde 57Revision 1.49 2007/06/29 10:45:06 acolla
58Number of columns in MySql Shuttle logbook increased by one (HLT added)
59
db99d43e 60Revision 1.48 2007/06/21 13:06:19 acolla
61GetFileSources returns dummy list with 1 source if system=DCS (better than
62returning error as it was)
63
6297b37d 64Revision 1.47 2007/06/19 17:28:56 acolla
65HLT updated; missing map bug removed.
66
dc25836b 67Revision 1.46 2007/06/09 13:01:09 jgrosseo
68Switching to retrieval of several DCS DPs at a time (multiDPrequest)
69
a038aa70 70Revision 1.45 2007/05/30 06:35:20 jgrosseo
71Adding functionality to the Shuttle/TestShuttle:
72o) Function to retrieve list of sources from a given system (GetFileSources with id=0)
73o) Function to retrieve list of IDs for a given source (GetFileIDs)
74These functions are needed for dealing with the tag files that are saved for the GRP preprocessor
75Example code has been added to the TestProcessor in TestShuttle
76
4a33bdd9 77Revision 1.44 2007/05/11 16:09:32 acolla
78Reference files for ITS, MUON and PHOS are now stored in OfflineDetName/OnlineDetName/run_...
79example: ITS/SPD/100_filename.root
80
2d9019b4 81Revision 1.43 2007/05/10 09:59:51 acolla
82Various bug fixes in StoreRefFilesToGrid; Cleaning of reference storage before processing detector (CleanReferenceStorage)
83
546242fb 84Revision 1.42 2007/05/03 08:01:39 jgrosseo
85typo in last commit :-(
86
8b739301 87Revision 1.41 2007/05/03 08:00:48 jgrosseo
88fixing log message when pp want to skip dcs value retrieval
89
651fdaab 90Revision 1.40 2007/04/27 07:06:48 jgrosseo
91GetFileSources returns empty list in case of no files, but successful query
92No mails sent in testmode
93
86aa42c3 94Revision 1.39 2007/04/17 12:43:57 acolla
95Correction in StoreOCDB; change of text in mail to detector expert
96
26758fce 97Revision 1.38 2007/04/12 08:26:18 jgrosseo
98updated comment
99
3c2a21c8 100Revision 1.37 2007/04/10 16:53:14 jgrosseo
101redirecting sub detector stdout, stderr to sub detector log file
102
3d8bc902 103Revision 1.35 2007/04/04 16:26:38 acolla
1041. Re-organization of function calls in TestPreprocessor to make it more meaningful.
1052. Added missing dependency in test preprocessors.
1063. in AliShuttle.cxx: processing time and memory consumption info on a single line.
107
886d60e6 108Revision 1.34 2007/04/04 10:33:36 jgrosseo
1091) Storing of files to the Grid is now done _after_ your preprocessors succeeded. This is transparent, which means that you can still use the same functions (Store, StoreReferenceData) to store files to the Grid. However, the Shuttle first stores them locally and transfers them after the preprocessor finished. The return code of these two functions has changed from UInt_t to Bool_t which gives you the success of the storing.
110In case of an error with the Grid, the Shuttle will retry the storing later, the preprocessor does not need to be run again.
111
1122) The meaning of the return code of the preprocessor has changed. 0 is now success and any other value means failure. This value is stored in the log and you can use it to keep details about the error condition.
113
1143) New function StoreReferenceFile to _directly_ store a file (without opening it) to the reference storage.
115
1164) The memory usage of the preprocessor is monitored. If it exceeds 2 GB it is terminated.
117
1185) New function AliPreprocessor::ProcessDCS(). If you do not need to have DCS data in all cases, you can skip the processing by implemting this function and returning kFALSE under certain conditions. E.g. if there is a certain run type.
119If you always need DCS data (like before), you do not need to implement it.
120
1216) The run type has been added to the monitoring page
122
9827400b 123Revision 1.33 2007/04/03 13:56:01 acolla
124Grid Storage at the end of preprocessing. Added virtual method to disable DCS query according to the
125run type.
126
3301427a 127Revision 1.32 2007/02/28 10:41:56 acolla
128Run type field added in SHUTTLE framework. Run type is read from "run type" logbook and retrieved by
129AliPreprocessor::GetRunType() function.
130Added some ldap definition files.
131
d386d623 132Revision 1.30 2007/02/13 11:23:21 acolla
133Moved getters and setters of Shuttle's main OCDB/Reference, local
134OCDB/Reference, temp and log folders to AliShuttleInterface
135
9d733021 136Revision 1.27 2007/01/30 17:52:42 jgrosseo
137adding monalisa monitoring
138
e7f62f16 139Revision 1.26 2007/01/23 19:20:03 acolla
140Removed old ldif files, added TOF, MCH ldif files. Added some options in
141AliShuttleConfig::Print. Added in Ali Shuttle: SetShuttleTempDir and
142SetShuttleLogDir
143
36c99a6a 144Revision 1.25 2007/01/15 19:13:52 acolla
145Moved some AliInfo to AliDebug in SendMail function
146
fc5a4708 147Revision 1.21 2006/12/07 08:51:26 jgrosseo
148update (alberto):
149table, db names in ldap configuration
150added GRP preprocessor
151DCS data can also be retrieved by data point
152
2c15234c 153Revision 1.20 2006/11/16 16:16:48 jgrosseo
154introducing strict run ordering flag
155removed giving preprocessor name to preprocessor, they have to know their name themselves ;-)
156
be48e3ea 157Revision 1.19 2006/11/06 14:23:04 jgrosseo
158major update (Alberto)
159o) reading of run parameters from the logbook
160o) online offline naming conversion
161o) standalone DCSclient package
162
eba76848 163Revision 1.18 2006/10/20 15:22:59 jgrosseo
164o) Adding time out to the execution of the preprocessors: The Shuttle forks and the parent process monitors the child
165o) Merging Collect, CollectAll, CollectNew function
166o) Removing implementation of empty copy constructors (declaration still there!)
167
cb343cfd 168Revision 1.17 2006/10/05 16:20:55 jgrosseo
169adapting to new CDB classes
170
6ec0e06c 171Revision 1.16 2006/10/05 15:46:26 jgrosseo
172applying to the new interface
173
481441a2 174Revision 1.15 2006/10/02 16:38:39 jgrosseo
175update (alberto):
176fixed memory leaks
177storing of objects that failed to be stored to the grid before
178interfacing of shuttle status table in daq system
179
2bb7b766 180Revision 1.14 2006/08/29 09:16:05 jgrosseo
181small update
182
85a80aa9 183Revision 1.13 2006/08/15 10:50:00 jgrosseo
184effc++ corrections (alberto)
185
4f0ab988 186Revision 1.12 2006/08/08 14:19:29 jgrosseo
187Update to shuttle classes (Alberto)
188
189- Possibility to set the full object's path in the Preprocessor's and
190Shuttle's Store functions
191- Possibility to extend the object's run validity in the same classes
192("startValidity" and "validityInfinite" parameters)
193- Implementation of the StoreReferenceData function to store reference
194data in a dedicated CDB storage.
195
84090f85 196Revision 1.11 2006/07/21 07:37:20 jgrosseo
197last run is stored after each run
198
7bfb2090 199Revision 1.10 2006/07/20 09:54:40 jgrosseo
200introducing status management: The processing per subdetector is divided into several steps,
201after each step the status is stored on disk. If the system crashes in any of the steps the Shuttle
202can keep track of the number of failures and skips further processing after a certain threshold is
203exceeded. These thresholds can be configured in LDAP.
204
5164a766 205Revision 1.9 2006/07/19 10:09:55 jgrosseo
206new configuration, accesst to DAQ FES (Alberto)
207
57f50b3c 208Revision 1.8 2006/07/11 12:44:36 jgrosseo
209adding parameters for extended validity range of data produced by preprocessor
210
17111222 211Revision 1.7 2006/07/10 14:37:09 jgrosseo
212small fix + todo comment
213
e090413b 214Revision 1.6 2006/07/10 13:01:41 jgrosseo
215enhanced storing of last sucessfully processed run (alberto)
216
a7160fe9 217Revision 1.5 2006/07/04 14:59:57 jgrosseo
218revision of AliDCSValue: Removed wrapper classes, reduced storage size per value by factor 2
219
45a493ce 220Revision 1.4 2006/06/12 09:11:16 jgrosseo
221coding conventions (Alberto)
222
58bc3020 223Revision 1.3 2006/06/06 14:26:40 jgrosseo
224o) removed files that were moved to STEER
225o) shuttle updated to follow the new interface (Alberto)
226
b948db8d 227Revision 1.2 2006/03/07 07:52:34 hristov
228New version (B.Yordanov)
229
d477ad88 230Revision 1.6 2005/11/19 17:19:14 byordano
231RetrieveDATEEntries and RetrieveConditionsData added
232
233Revision 1.5 2005/11/19 11:09:27 byordano
234AliShuttle declaration added
235
236Revision 1.4 2005/11/17 17:47:34 byordano
237TList changed to TObjArray
238
239Revision 1.3 2005/11/17 14:43:23 byordano
240import to local CVS
241
242Revision 1.1.1.1 2005/10/28 07:33:58 hristov
243Initial import as subdirectory in AliRoot
244
73abe331 245Revision 1.2 2005/09/13 08:41:15 byordano
246default startTime endTime added
247
248Revision 1.4 2005/08/30 09:13:02 byordano
249some docs added
250
251Revision 1.3 2005/08/29 21:15:47 byordano
252some docs added
253
254*/
255
256//
257// This class is the main manager for AliShuttle.
258// It organizes the data retrieval from DCS and call the
b948db8d 259// interface methods of AliPreprocessor.
73abe331 260// For every detector in AliShuttleConfgi (see AliShuttleConfig),
261// data for its set of aliases is retrieved. If there is registered
b948db8d 262// AliPreprocessor for this detector then it will be used
263// accroding to the schema (see AliPreprocessor).
264// If there isn't registered AliPreprocessor than the retrieved
73abe331 265// data is stored automatically to the undelying AliCDBStorage.
266// For detSpec is used the alias name.
267//
268
269#include "AliShuttle.h"
270
271#include "AliCDBManager.h"
272#include "AliCDBStorage.h"
273#include "AliCDBId.h"
84090f85 274#include "AliCDBRunRange.h"
275#include "AliCDBPath.h"
5164a766 276#include "AliCDBEntry.h"
73abe331 277#include "AliShuttleConfig.h"
eba76848 278#include "DCSClient/AliDCSClient.h"
73abe331 279#include "AliLog.h"
b948db8d 280#include "AliPreprocessor.h"
5164a766 281#include "AliShuttleStatus.h"
2bb7b766 282#include "AliShuttleLogbookEntry.h"
73abe331 283
57f50b3c 284#include <TSystem.h>
58bc3020 285#include <TObject.h>
b948db8d 286#include <TString.h>
57f50b3c 287#include <TTimeStamp.h>
73abe331 288#include <TObjString.h>
57f50b3c 289#include <TSQLServer.h>
290#include <TSQLResult.h>
291#include <TSQLRow.h>
cb343cfd 292#include <TMutex.h>
9827400b 293#include <TSystemDirectory.h>
294#include <TSystemFile.h>
a986b218 295#include <TFile.h>
9827400b 296#include <TGrid.h>
297#include <TGridResult.h>
73abe331 298
e7f62f16 299#include <TMonaLisaWriter.h>
300
5164a766 301#include <fstream>
302
cb343cfd 303#include <sys/types.h>
304#include <sys/wait.h>
305
73abe331 306ClassImp(AliShuttle)
307
b948db8d 308//______________________________________________________________________________________________
309AliShuttle::AliShuttle(const AliShuttleConfig* config,
310 UInt_t timeout, Int_t retries):
4f0ab988 311fConfig(config),
312fTimeout(timeout), fRetries(retries),
313fPreprocessorMap(),
2bb7b766 314fLogbookEntry(0),
eba76848 315fCurrentDetector(),
85a80aa9 316fStatusEntry(0),
cb343cfd 317fMonitoringMutex(0),
eba76848 318fLastActionTime(0),
e7f62f16 319fLastAction(),
9827400b 320fMonaLisa(0),
321fTestMode(kNone),
ffa29e93 322fReadTestMode(kFALSE),
323fOutputRedirected(kFALSE)
73abe331 324{
325 //
326 // config: AliShuttleConfig used
73abe331 327 // timeout: timeout used for AliDCSClient connection
328 // retries: the number of retries in case of connection error.
329 //
330
57f50b3c 331 if (!fConfig->IsValid()) AliFatal("********** !!!!! Invalid configuration !!!!! **********");
be48e3ea 332 for(int iSys=0;iSys<4;iSys++) {
57f50b3c 333 fServer[iSys]=0;
be48e3ea 334 if (iSys < 3)
2c15234c 335 fFXSlist[iSys].SetOwner(kTRUE);
57f50b3c 336 }
2bb7b766 337 fPreprocessorMap.SetOwner(kTRUE);
be48e3ea 338
339 for (UInt_t iDet=0; iDet<NDetectors(); iDet++)
340 fFirstUnprocessed[iDet] = kFALSE;
341
cb343cfd 342 fMonitoringMutex = new TMutex();
58bc3020 343}
344
b948db8d 345//______________________________________________________________________________________________
57f50b3c 346AliShuttle::~AliShuttle()
58bc3020 347{
9827400b 348 //
349 // destructor
350 //
58bc3020 351
b948db8d 352 fPreprocessorMap.DeleteAll();
be48e3ea 353 for(int iSys=0;iSys<4;iSys++)
57f50b3c 354 if(fServer[iSys]) {
355 fServer[iSys]->Close();
356 delete fServer[iSys];
eba76848 357 fServer[iSys] = 0;
57f50b3c 358 }
2bb7b766 359
360 if (fStatusEntry){
361 delete fStatusEntry;
362 fStatusEntry = 0;
363 }
cb343cfd 364
365 if (fMonitoringMutex)
366 {
367 delete fMonitoringMutex;
368 fMonitoringMutex = 0;
369 }
73abe331 370}
371
b948db8d 372//______________________________________________________________________________________________
57f50b3c 373void AliShuttle::RegisterPreprocessor(AliPreprocessor* preprocessor)
58bc3020 374{
73abe331 375 //
b948db8d 376 // Registers new AliPreprocessor.
73abe331 377 // It uses GetName() for indentificator of the pre processor.
378 // The pre processor is registered it there isn't any other
379 // with the same identificator (GetName()).
380 //
381
eba76848 382 const char* detName = preprocessor->GetName();
383 if(GetDetPos(detName) < 0)
384 AliFatal(Form("********** !!!!! Invalid detector name: %s !!!!! **********", detName));
385
386 if (fPreprocessorMap.GetValue(detName)) {
387 AliWarning(Form("AliPreprocessor %s is already registered!", detName));
73abe331 388 return;
389 }
390
eba76848 391 fPreprocessorMap.Add(new TObjString(detName), preprocessor);
73abe331 392}
b948db8d 393//______________________________________________________________________________________________
3301427a 394Bool_t AliShuttle::Store(const AliCDBPath& path, TObject* object,
84090f85 395 AliCDBMetaData* metaData, Int_t validityStart, Bool_t validityInfinite)
73abe331 396{
9827400b 397 // Stores a CDB object in the storage for offline reconstruction. Objects that are not needed for
398 // offline reconstruction, but should be stored anyway (e.g. for debugging) should NOT be stored
399 // using this function. Use StoreReferenceData instead!
400 // It calls StoreLocally function which temporarily stores the data locally; when the preprocessor
401 // finishes the data are transferred to the main storage (Grid).
b948db8d 402
3301427a 403 return StoreLocally(fgkLocalCDB, path, object, metaData, validityStart, validityInfinite);
84090f85 404}
405
406//______________________________________________________________________________________________
3301427a 407Bool_t AliShuttle::StoreReferenceData(const AliCDBPath& path, TObject* object, AliCDBMetaData* metaData)
84090f85 408{
9827400b 409 // Stores a CDB object in the storage for reference data. This objects will not be available during
410 // offline reconstrunction. Use this function for reference data only!
411 // It calls StoreLocally function which temporarily stores the data locally; when the preprocessor
412 // finishes the data are transferred to the main storage (Grid).
85a80aa9 413
3301427a 414 return StoreLocally(fgkLocalRefStorage, path, object, metaData);
85a80aa9 415}
416
417//______________________________________________________________________________________________
3301427a 418Bool_t AliShuttle::StoreLocally(const TString& localUri,
85a80aa9 419 const AliCDBPath& path, TObject* object, AliCDBMetaData* metaData,
420 Int_t validityStart, Bool_t validityInfinite)
421{
9827400b 422 // Store object temporarily in local storage. Parameters are passed by Store and StoreReferenceData functions.
423 // when the preprocessor finishes the data are transferred to the main storage (Grid).
424 // The parameters are:
425 // 1) Uri of the backup storage (Local)
426 // 2) the object's path.
427 // 3) the object to be stored
428 // 4) the metaData to be associated with the object
429 // 5) the validity start run number w.r.t. the current run,
430 // if the data is valid only for this run leave the default 0
431 // 6) specifies if the calibration data is valid for infinity (this means until updated),
432 // typical for calibration runs, the default is kFALSE
433 //
434 // returns 0 if fail, 1 otherwise
84090f85 435
9827400b 436 if (fTestMode & kErrorStorage)
437 {
438 Log(fCurrentDetector, "StoreLocally - In TESTMODE - Simulating error while storing locally");
439 return kFALSE;
440 }
441
3301427a 442 const char* cdbType = (localUri == fgkLocalCDB) ? "CDB" : "Reference";
2bb7b766 443
85a80aa9 444 Int_t firstRun = GetCurrentRun() - validityStart;
84090f85 445 if(firstRun < 0) {
9827400b 446 AliWarning("First valid run happens to be less than 0! Setting it to 0.");
84090f85 447 firstRun=0;
448 }
449
450 Int_t lastRun = -1;
451 if(validityInfinite) {
452 lastRun = AliCDBRunRange::Infinity();
453 } else {
454 lastRun = GetCurrentRun();
455 }
456
3301427a 457 // Version is set to current run, it will be used later to transfer data to Grid
458 AliCDBId id(path, firstRun, lastRun, GetCurrentRun(), -1);
2bb7b766 459
460 if(! dynamic_cast<TObjString*> (metaData->GetProperty("RunUsed(TObjString)"))){
461 TObjString runUsed = Form("%d", GetCurrentRun());
9e080f92 462 metaData->SetProperty("RunUsed(TObjString)", runUsed.Clone());
2bb7b766 463 }
84090f85 464
3301427a 465 Bool_t result = kFALSE;
84090f85 466
3301427a 467 if (!(AliCDBManager::Instance()->GetStorage(localUri))) {
468 Log("SHUTTLE", Form("StoreLocally - Cannot activate local %s storage", cdbType));
84090f85 469 } else {
3301427a 470 result = AliCDBManager::Instance()->GetStorage(localUri)
84090f85 471 ->Put(object, id, metaData);
472 }
473
474 if(!result) {
475
9827400b 476 Log(fCurrentDetector, Form("StoreLocally - Can't store object <%s>!", id.ToString().Data()));
3301427a 477 }
2bb7b766 478
3301427a 479 return result;
480}
84090f85 481
3301427a 482//______________________________________________________________________________________________
483Bool_t AliShuttle::StoreOCDB()
484{
9827400b 485 //
486 // Called when preprocessor ends successfully or when previous storage attempt failed (kStoreError status)
487 // Calls underlying StoreOCDB(const char*) function twice, for OCDB and Reference storage.
488 // Then calls StoreRefFilesToGrid to store reference files.
489 //
490
491 if (fTestMode & kErrorGrid)
492 {
493 Log("SHUTTLE", "StoreOCDB - In TESTMODE - Simulating error while storing in the Grid");
494 Log(fCurrentDetector, "StoreOCDB - In TESTMODE - Simulating error while storing in the Grid");
495 return kFALSE;
496 }
497
c88ad5db 498 Log("SHUTTLE","StoreOCDB - Storing OCDB data ...");
86aa42c3 499 Bool_t resultCDB = StoreOCDB(fgkMainCDB);
500
c88ad5db 501 Log("SHUTTLE","StoreOCDB - Storing reference data ...");
3301427a 502 Bool_t resultRef = StoreOCDB(fgkMainRefStorage);
9827400b 503
c88ad5db 504 Log("SHUTTLE","StoreOCDB - Storing reference files ...");
505 Bool_t resultRefFiles = CopyFilesToGrid("reference");
506
507 Bool_t resultMetadata = kTRUE;
508 if(fCurrentDetector == "GRP")
509 {
510 Log("StoreOCDB - SHUTTLE","Storing Run Metadata file ...");
511 resultMetadata = CopyFilesToGrid("metadata");
512 }
9827400b 513
c88ad5db 514 return resultCDB && resultRef && resultRefFiles && resultMetadata;
3301427a 515}
516
517//______________________________________________________________________________________________
518Bool_t AliShuttle::StoreOCDB(const TString& gridURI)
519{
520 //
521 // Called by StoreOCDB(), performs actual storage to the main OCDB and reference storages (Grid)
522 //
523
524 TObjArray* gridIds=0;
525
526 Bool_t result = kTRUE;
527
528 const char* type = 0;
529 TString localURI;
530 if(gridURI == fgkMainCDB) {
531 type = "OCDB";
532 localURI = fgkLocalCDB;
533 } else if(gridURI == fgkMainRefStorage) {
534 type = "reference";
535 localURI = fgkLocalRefStorage;
536 } else {
537 AliError(Form("Invalid storage URI: %s", gridURI.Data()));
538 return kFALSE;
539 }
540
541 AliCDBManager* man = AliCDBManager::Instance();
542
543 AliCDBStorage *gridSto = man->GetStorage(gridURI);
544 if(!gridSto) {
545 Log("SHUTTLE",
546 Form("StoreOCDB - cannot activate main %s storage", type));
547 return kFALSE;
548 }
549
550 gridIds = gridSto->GetQueryCDBList();
551
552 // get objects previously stored in local CDB
553 AliCDBStorage *localSto = man->GetStorage(localURI);
554 if(!localSto) {
555 Log("SHUTTLE",
556 Form("StoreOCDB - cannot activate local %s storage", type));
557 return kFALSE;
558 }
559 AliCDBPath aPath(GetOfflineDetName(fCurrentDetector.Data()),"*","*");
560 // Local objects were stored with current run as Grid version!
561 TList* localEntries = localSto->GetAll(aPath.GetPath(), GetCurrentRun(), GetCurrentRun());
562 localEntries->SetOwner(1);
563
564 // loop on local stored objects
565 TIter localIter(localEntries);
566 AliCDBEntry *aLocEntry = 0;
567 while((aLocEntry = dynamic_cast<AliCDBEntry*> (localIter.Next()))){
568 aLocEntry->SetOwner(1);
569 AliCDBId aLocId = aLocEntry->GetId();
570 aLocEntry->SetVersion(-1);
571 aLocEntry->SetSubVersion(-1);
572
573 // If local object is valid up to infinity we store it only if it is
574 // the first unprocessed run!
575 if (aLocId.GetLastRun() == AliCDBRunRange::Infinity() &&
576 !fFirstUnprocessed[GetDetPos(fCurrentDetector)])
577 {
578 Log("SHUTTLE", Form("StoreOCDB - %s: object %s has validity infinite but "
579 "there are previous unprocessed runs!",
580 fCurrentDetector.Data(), aLocId.GetPath().Data()));
581 continue;
582 }
583
584 // loop on Grid valid Id's
585 Bool_t store = kTRUE;
586 TIter gridIter(gridIds);
587 AliCDBId* aGridId = 0;
588 while((aGridId = dynamic_cast<AliCDBId*> (gridIter.Next()))){
589 if(aGridId->GetPath() != aLocId.GetPath()) continue;
590 // skip all objects valid up to infinity
591 if(aGridId->GetLastRun() == AliCDBRunRange::Infinity()) continue;
592 // if we get here, it means there's already some more recent object stored on Grid!
593 store = kFALSE;
594 break;
595 }
596
597 // If we get here, the file can be stored!
598 Bool_t storeOk = gridSto->Put(aLocEntry);
599 if(!store || storeOk){
600
601 if (!store)
602 {
603 Log(fCurrentDetector.Data(),
604 Form("StoreOCDB - A more recent object already exists in %s storage: <%s>",
605 type, aGridId->ToString().Data()));
606 } else {
607 Log("SHUTTLE",
608 Form("StoreOCDB - Object <%s> successfully put into %s storage",
609 aLocId.ToString().Data(), type));
2d9019b4 610 Log(fCurrentDetector.Data(),
611 Form("StoreOCDB - Object <%s> successfully put into %s storage",
612 aLocId.ToString().Data(), type));
3301427a 613 }
84090f85 614
3301427a 615 // removing local filename...
616 TString filename;
617 localSto->IdToFilename(aLocId, filename);
c88ad5db 618 Log("SHUTTLE", Form("StoreOCDB - Removing local file %s", filename.Data()));
3301427a 619 RemoveFile(filename.Data());
620 continue;
621 } else {
622 Log("SHUTTLE",
623 Form("StoreOCDB - Grid %s storage of object <%s> failed",
624 type, aLocId.ToString().Data()));
2d9019b4 625 Log(fCurrentDetector.Data(),
626 Form("StoreOCDB - Grid %s storage of object <%s> failed",
627 type, aLocId.ToString().Data()));
3301427a 628 result = kFALSE;
b948db8d 629 }
630 }
3301427a 631 localEntries->Clear();
2bb7b766 632
b948db8d 633 return result;
3301427a 634}
635
546242fb 636//______________________________________________________________________________________________
637Bool_t AliShuttle::CleanReferenceStorage(const char* detector)
638{
2d9019b4 639 // clears the directory used to store reference files of a given subdetector
546242fb 640
641 AliCDBManager* man = AliCDBManager::Instance();
642 AliCDBStorage* sto = man->GetStorage(fgkLocalRefStorage);
2d9019b4 643 TString localBaseFolder = sto->GetBaseFolder();
644
645 TString targetDir = GetRefFilePrefix(localBaseFolder.Data(), detector);
646
647 Log("SHUTTLE", Form("Cleaning %s", targetDir.Data()));
648
649 TString begin;
650 begin.Form("%d_", GetCurrentRun());
651
652 TSystemDirectory* baseDir = new TSystemDirectory("/", targetDir);
653 if (!baseDir)
654 return kTRUE;
655
656 TList* dirList = baseDir->GetListOfFiles();
657 delete baseDir;
658
659 if (!dirList) return kTRUE;
660
661 if (dirList->GetEntries() < 3)
662 {
663 delete dirList;
664 return kTRUE;
665 }
666
667 Int_t nDirs = 0, nDel = 0;
668 TIter dirIter(dirList);
669 TSystemFile* entry = 0;
546242fb 670
2d9019b4 671 Bool_t success = kTRUE;
546242fb 672
2d9019b4 673 while ((entry = dynamic_cast<TSystemFile*> (dirIter.Next())))
674 {
675 if (entry->IsDirectory())
676 continue;
677
678 TString fileName(entry->GetName());
679 if (!fileName.BeginsWith(begin))
680 continue;
681
682 nDirs++;
683
684 // delete file
685 Int_t result = gSystem->Unlink(fileName.Data());
686
687 if (result)
688 {
689 Log("SHUTTLE", Form("Could not delete file %s!", fileName.Data()));
690 success = kFALSE;
691 } else {
692 nDel++;
693 }
694 }
695
696 if(nDirs > 0)
697 Log("SHUTTLE", Form("CleanReferenceStorage - %d (over %d) reference files in folder %s were deleted.",
698 nDel, nDirs, targetDir.Data()));
699
700
701 delete dirList;
702 return success;
703
704
705
706
707
546242fb 708
709 Int_t result = gSystem->GetPathInfo(targetDir, 0, (Long64_t*) 0, 0, 0);
710 if (result == 0)
711 {
712 // delete directory
713 result = gSystem->Exec(Form("rm -r %s", targetDir.Data()));
714 if (result != 0)
715 {
c88ad5db 716 Log("SHUTTLE", Form("CleanReferenceStorage - Could not clear directory %s", targetDir.Data()));
546242fb 717 return kFALSE;
718 }
719 }
720
721 result = gSystem->mkdir(targetDir, kTRUE);
722 if (result != 0)
723 {
c88ad5db 724 Log("SHUTTLE", Form("CleanReferenceStorage - Error creating base directory %s", targetDir.Data()));
546242fb 725 return kFALSE;
726 }
727
728 return kTRUE;
729}
730
9827400b 731//______________________________________________________________________________________________
732Bool_t AliShuttle::StoreReferenceFile(const char* detector, const char* localFile, const char* gridFileName)
733{
734 //
3c2a21c8 735 // Stores reference file directly (without opening it). This function stores the file locally.
9827400b 736 //
3c2a21c8 737 // The file is stored under the following location:
738 // <base folder of local reference storage>/<DET>/<RUN#>_<gridFileName>
739 // where <gridFileName> is the second parameter given to the function
740 //
9827400b 741
742 if (fTestMode & kErrorStorage)
743 {
744 Log(fCurrentDetector, "StoreReferenceFile - In TESTMODE - Simulating error while storing locally");
745 return kFALSE;
746 }
747
748 AliCDBManager* man = AliCDBManager::Instance();
749 AliCDBStorage* sto = man->GetStorage(fgkLocalRefStorage);
750
751 TString localBaseFolder = sto->GetBaseFolder();
752
2d9019b4 753 TString targetDir = GetRefFilePrefix(localBaseFolder.Data(), detector);
9827400b 754
c88ad5db 755 return CopyFileLocally(targetDir, localFile, gridFileName);
756}
757
758//______________________________________________________________________________________________
759Bool_t AliShuttle::StoreRunMetadataFile(const char* localFile, const char* gridFileName)
760{
761 //
762 // Stores Run metadata file to the Grid, in the run folder
763 //
764 // Only GRP can call this function.
765
766 if (fTestMode & kErrorStorage)
767 {
768 Log(fCurrentDetector, "StoreRunMetaDataFile - In TESTMODE - Simulating error while storing locally");
769 return kFALSE;
770 }
771
772 AliCDBManager* man = AliCDBManager::Instance();
773 AliCDBStorage* sto = man->GetStorage(fgkLocalRefStorage);
774
775 TString localBaseFolder = sto->GetBaseFolder();
776
777 // Build Run level folder
778 // folder = /alice/data/year/lhcPeriod/runNb/Raw
779
780 TTimeStamp startTime(GetCurrentStartTime());
781
782 TString year = Form("%d",startTime.GetDate());
783 year = year(0,4);
784
785 TString lhcPeriod = GetRunParameter("LHCperiod");
786
787 if (lhcPeriod.Length() == 0)
788 {
789 Log("SHUTTLE","StoreRunMetaDataFile - LHCPeriod not found in logbook!");
790 return 0;
791 }
792
793 // TODO: currently SHUTTLE cannot write in /alice/data/ !!!!!
794 //TString targetDir = Form("%s/GRP/RunMetadata/alice/data/%s/%s/%d/Raw",
795 // localBaseFolder.Data(), year.Data(),
796 // lhcPeriod.Data(), GetCurrentRun());
797
798 TString targetDir = Form("%s/GRP/RunMetadata/alice/simulation/%s/%s/%d/Raw",
799 localBaseFolder.Data(), year.Data(),
800 lhcPeriod.Data(), GetCurrentRun());
801
802 return CopyFileLocally(targetDir, localFile, gridFileName);
803}
804
805//______________________________________________________________________________________________
806Bool_t AliShuttle::CopyFileLocally(TString& targetDir, const char* localFile, const char* gridFileName)
807{
808 //
809 // Stores file locally. Called by StoreReferenceFile and StoreRunMetadataFile
810 //
811
812 //try to open folder, if it does not exist
2d9019b4 813 void* dir = gSystem->OpenDirectory(targetDir.Data());
814 if (dir == NULL) {
815 if (gSystem->mkdir(targetDir.Data(), kTRUE)) {
c88ad5db 816 Log("SHUTTLE", Form("StoreFileLocally - Can't open directory <%s>", targetDir.Data()));
2d9019b4 817 return kFALSE;
818 }
819
820 } else {
821 gSystem->FreeDirectory(dir);
822 }
823
c88ad5db 824 TString target = Form("%s/%s", targetDir.Data(), gridFileName);
9827400b 825
546242fb 826 Int_t result = gSystem->GetPathInfo(localFile, 0, (Long64_t*) 0, 0, 0);
9827400b 827 if (result)
828 {
c88ad5db 829 Log("SHUTTLE", Form("StoreFileLocally - %s does not exist", localFile));
546242fb 830 return kFALSE;
9827400b 831 }
546242fb 832
9827400b 833 result = gSystem->CopyFile(localFile, target);
834
835 if (result == 0)
836 {
c88ad5db 837 Log("SHUTTLE", Form("StoreFileLocally - File %s stored locally to %s", localFile, target.Data()));
9827400b 838 return kTRUE;
839 }
840 else
841 {
c88ad5db 842 Log("SHUTTLE", Form("StoreFileLocally - Could not store file %s to %s!. Error code = %d",
546242fb 843 localFile, target.Data(), result));
9827400b 844 return kFALSE;
845 }
c88ad5db 846
847
848
9827400b 849}
850
851//______________________________________________________________________________________________
c88ad5db 852Bool_t AliShuttle::CopyFilesToGrid(const char* type)
9827400b 853{
854 //
c88ad5db 855 // Transfers local files to the Grid. Local files can be reference files
856 // or run metadata file (from GRP only).
9827400b 857 //
c88ad5db 858 // According to the type (ref, metadata) the files are stored under the following location:
859 // ref --> <base folder of reference storage>/<DET>/<RUN#>_<gridFileName>
860 // metadata --> <run data folder>/<MetadataFileName>
86aa42c3 861 //
c88ad5db 862
9827400b 863 AliCDBManager* man = AliCDBManager::Instance();
864 AliCDBStorage* sto = man->GetStorage(fgkLocalRefStorage);
865 if (!sto)
866 return kFALSE;
867 TString localBaseFolder = sto->GetBaseFolder();
9827400b 868
c88ad5db 869 TString dir;
870 TString alienDir;
9827400b 871 TString begin;
9827400b 872
c88ad5db 873 if (strcmp(type, "reference") == 0)
874 {
875 dir = GetRefFilePrefix(localBaseFolder.Data(), fCurrentDetector.Data());
876 AliCDBStorage* gridSto = man->GetStorage(fgkMainRefStorage);
877 if (!gridSto)
878 return kFALSE;
879 TString gridBaseFolder = gridSto->GetBaseFolder();
880 alienDir = GetRefFilePrefix(gridBaseFolder.Data(), fCurrentDetector.Data());
881 begin = Form("%d_", GetCurrentRun());
882 }
883 else if (strcmp(type, "metadata") == 0)
884 {
885 TTimeStamp startTime(GetCurrentStartTime());
886
887 TString year = Form("%d",startTime.GetDate());
888 year = year(0,4);
889
890 TString lhcPeriod = GetRunParameter("LHCperiod");
891
892 if (lhcPeriod.Length() == 0)
893 {
894 Log("SHUTTLE","CopyFilesToGrid - LHCPeriod not found in logbook!");
895 return 0;
896 }
897
898 // TODO: currently SHUTTLE cannot write in /alice/data/ !!!!!
899 //dir = Form("%s/GRP/RunMetadata/alice/data/%s/%s/%d/Raw",
900 // localBaseFolder.Data(), year.Data(),
901 // lhcPeriod.Data(), GetCurrentRun());
902 //alienDir = dir(dir.Index("/alice/data/"), dir.Length());
903
904 dir = Form("%s/GRP/RunMetadata/alice/simulation/%s/%s/%d/Raw",
905 localBaseFolder.Data(), year.Data(),
906 lhcPeriod.Data(), GetCurrentRun());
907 alienDir = dir(dir.Index("/alice/simulation/"), dir.Length());
908 begin = "";
909 }
910 else
911 {
912 Log("SHUTTLE", "CopyFilesToGrid - Unexpected: type label must be reference or metadata!");
913 return kFALSE;
914 }
915
9827400b 916 TSystemDirectory* baseDir = new TSystemDirectory("/", dir);
3d8bc902 917 if (!baseDir)
918 return kTRUE;
919
2d9019b4 920 TList* dirList = baseDir->GetListOfFiles();
921 delete baseDir;
922
923 if (!dirList) return kTRUE;
924
925 if (dirList->GetEntries() < 3)
3d8bc902 926 {
2d9019b4 927 delete dirList;
9827400b 928 return kTRUE;
3d8bc902 929 }
2d9019b4 930
546242fb 931 if (!gGrid)
932 {
c88ad5db 933 Log("SHUTTLE", "CopyFilesToGrid - Connection to Grid failed: Cannot continue!");
2d9019b4 934 delete dirList;
546242fb 935 return kFALSE;
936 }
937
2d9019b4 938 Int_t nDirs = 0, nTransfer = 0;
939 TIter dirIter(dirList);
940 TSystemFile* entry = 0;
941
9827400b 942 Bool_t success = kTRUE;
3d8bc902 943 Bool_t first = kTRUE;
9827400b 944
2d9019b4 945 while ((entry = dynamic_cast<TSystemFile*> (dirIter.Next())))
946 {
9827400b 947 if (entry->IsDirectory())
948 continue;
949
950 TString fileName(entry->GetName());
951 if (!fileName.BeginsWith(begin))
952 continue;
953
2d9019b4 954 nDirs++;
955
3d8bc902 956 if (first)
957 {
958 first = kFALSE;
c88ad5db 959 // check that folder exists, otherwise create it
3d8bc902 960 TGridResult* result = gGrid->Ls(alienDir.Data(), "a");
961
962 if (!result)
2d9019b4 963 {
964 delete dirList;
3d8bc902 965 return kFALSE;
2d9019b4 966 }
3d8bc902 967
546242fb 968 if (!result->GetFileName(1)) // TODO: It looks like element 0 is always 0!!
3d8bc902 969 {
c88ad5db 970 if (!gGrid->Mkdir(alienDir.Data(),"-p",0))
3d8bc902 971 {
c88ad5db 972 Log("SHUTTLE", Form("CopyFilesToGrid - Cannot create directory %s",
3d8bc902 973 alienDir.Data()));
2d9019b4 974 delete dirList;
3d8bc902 975 return kFALSE;
546242fb 976 } else {
c88ad5db 977 Log("SHUTTLE",Form("CopyFilesToGrid - Folder %s created", alienDir.Data()));
3d8bc902 978 }
979
546242fb 980 } else {
c88ad5db 981 Log("SHUTTLE",Form("CopyFilesToGrid - Folder %s found", alienDir.Data()));
3d8bc902 982 }
983 }
984
9827400b 985 TString fullLocalPath;
986 fullLocalPath.Form("%s/%s", dir.Data(), fileName.Data());
987
988 TString fullGridPath;
989 fullGridPath.Form("alien://%s/%s", alienDir.Data(), fileName.Data());
990
a986b218 991 Bool_t result = TFile::Cp(fullLocalPath, fullGridPath);
9827400b 992
993 if (result)
994 {
c88ad5db 995 Log("SHUTTLE", Form("CopyFilesToGrid - Copying local file %s to %s succeeded!",
996 fullLocalPath.Data(), fullGridPath.Data()));
9827400b 997 RemoveFile(fullLocalPath);
2d9019b4 998 nTransfer++;
9827400b 999 }
1000 else
1001 {
c88ad5db 1002 Log("SHUTTLE", Form("CopyFilesToGrid - Copying local file %s to %s FAILED!",
1003 fullLocalPath.Data(), fullGridPath.Data()));
9827400b 1004 success = kFALSE;
1005 }
1006 }
2d9019b4 1007
c88ad5db 1008 Log("SHUTTLE", Form("CopyFilesToGrid - %d (over %d) files in folder %s copied to Grid.",
1009 nTransfer, nDirs, dir.Data()));
2d9019b4 1010
1011
1012 delete dirList;
9827400b 1013 return success;
1014}
1015
2d9019b4 1016//______________________________________________________________________________________________
1017const char* AliShuttle::GetRefFilePrefix(const char* base, const char* detector)
1018{
1019 //
1020 // Get folder name of reference files
1021 //
1022
1023 TString offDetStr(GetOfflineDetName(detector));
1024 TString dir;
1025 if (offDetStr == "ITS" || offDetStr == "MUON" || offDetStr == "PHOS")
1026 {
1027 dir.Form("%s/%s/%s", base, offDetStr.Data(), detector);
1028 } else {
1029 dir.Form("%s/%s", base, offDetStr.Data());
1030 }
1031
1032 return dir.Data();
1033
1034
1035}
c88ad5db 1036
3301427a 1037//______________________________________________________________________________________________
1038void AliShuttle::CleanLocalStorage(const TString& uri)
1039{
9827400b 1040 //
1041 // Called in case the preprocessor is declared failed. Remove remaining objects from the local storages.
1042 //
3301427a 1043
1044 const char* type = 0;
1045 if(uri == fgkLocalCDB) {
1046 type = "OCDB";
1047 } else if(uri == fgkLocalRefStorage) {
546242fb 1048 type = "Reference";
3301427a 1049 } else {
1050 AliError(Form("Invalid storage URI: %s", uri.Data()));
1051 return;
1052 }
1053
1054 AliCDBManager* man = AliCDBManager::Instance();
b948db8d 1055
3301427a 1056 // open local storage
1057 AliCDBStorage *localSto = man->GetStorage(uri);
1058 if(!localSto) {
1059 Log("SHUTTLE",
1060 Form("CleanLocalStorage - cannot activate local %s storage", type));
1061 return;
1062 }
1063
1064 TString filename(Form("%s/%s/*/Run*_v%d_s*.root",
546242fb 1065 localSto->GetBaseFolder().Data(), GetOfflineDetName(fCurrentDetector.Data()), GetCurrentRun()));
3301427a 1066
c88ad5db 1067 AliDebug(2, Form("filename = %s", filename.Data()));
3301427a 1068
c88ad5db 1069 Log("SHUTTLE", Form("Removing remaining local files for run %d and detector %s ...",
3301427a 1070 GetCurrentRun(), fCurrentDetector.Data()));
1071
1072 RemoveFile(filename.Data());
1073
1074}
1075
1076//______________________________________________________________________________________________
1077void AliShuttle::RemoveFile(const char* filename)
1078{
9827400b 1079 //
1080 // removes local file
1081 //
3301427a 1082
1083 TString command(Form("rm -f %s", filename));
1084
1085 Int_t result = gSystem->Exec(command.Data());
1086 if(result != 0)
1087 {
1088 Log("SHUTTLE", Form("RemoveFile - %s: Cannot remove file %s!",
1089 fCurrentDetector.Data(), filename));
1090 }
73abe331 1091}
1092
b948db8d 1093//______________________________________________________________________________________________
5164a766 1094AliShuttleStatus* AliShuttle::ReadShuttleStatus()
1095{
9827400b 1096 //
1097 // Reads the AliShuttleStatus from the CDB
1098 //
5164a766 1099
2bb7b766 1100 if (fStatusEntry){
1101 delete fStatusEntry;
1102 fStatusEntry = 0;
1103 }
5164a766 1104
10a5a932 1105 fStatusEntry = AliCDBManager::Instance()->GetStorage(GetLocalCDB())
2bb7b766 1106 ->Get(Form("/SHUTTLE/STATUS/%s", fCurrentDetector.Data()), GetCurrentRun());
5164a766 1107
2bb7b766 1108 if (!fStatusEntry) return 0;
1109 fStatusEntry->SetOwner(1);
5164a766 1110
2bb7b766 1111 AliShuttleStatus* status = dynamic_cast<AliShuttleStatus*> (fStatusEntry->GetObject());
1112 if (!status) {
1113 AliError("Invalid object stored to CDB!");
1114 return 0;
1115 }
5164a766 1116
2bb7b766 1117 return status;
5164a766 1118}
1119
1120//______________________________________________________________________________________________
7bfb2090 1121Bool_t AliShuttle::WriteShuttleStatus(AliShuttleStatus* status)
5164a766 1122{
9827400b 1123 //
1124 // writes the status for one subdetector
1125 //
2bb7b766 1126
1127 if (fStatusEntry){
1128 delete fStatusEntry;
1129 fStatusEntry = 0;
1130 }
5164a766 1131
2bb7b766 1132 Int_t run = GetCurrentRun();
5164a766 1133
2bb7b766 1134 AliCDBId id(AliCDBPath("SHUTTLE", "STATUS", fCurrentDetector), run, run);
5164a766 1135
2bb7b766 1136 fStatusEntry = new AliCDBEntry(status, id, new AliCDBMetaData);
1137 fStatusEntry->SetOwner(1);
5164a766 1138
2bb7b766 1139 UInt_t result = AliCDBManager::Instance()->GetStorage(fgkLocalCDB)->Put(fStatusEntry);
7bfb2090 1140
2bb7b766 1141 if (!result) {
3301427a 1142 Log("SHUTTLE", Form("WriteShuttleStatus - Failed for %s, run %d",
1143 fCurrentDetector.Data(), run));
2bb7b766 1144 return kFALSE;
1145 }
e7f62f16 1146
1147 SendMLInfo();
7bfb2090 1148
2bb7b766 1149 return kTRUE;
5164a766 1150}
1151
1152//______________________________________________________________________________________________
1153void AliShuttle::UpdateShuttleStatus(AliShuttleStatus::Status newStatus, Bool_t increaseCount)
1154{
9827400b 1155 //
1156 // changes the AliShuttleStatus for the given detector and run to the given status
1157 //
5164a766 1158
2bb7b766 1159 if (!fStatusEntry){
1160 AliError("UNEXPECTED: fStatusEntry empty");
1161 return;
1162 }
5164a766 1163
2bb7b766 1164 AliShuttleStatus* status = dynamic_cast<AliShuttleStatus*> (fStatusEntry->GetObject());
5164a766 1165
2bb7b766 1166 if (!status){
c88ad5db 1167 Log("SHUTTLE", "UpdateShuttleStatus - UNEXPECTED: status could not be read from current CDB entry");
2bb7b766 1168 return;
1169 }
5164a766 1170
2c15234c 1171 TString actionStr = Form("UpdateShuttleStatus - %s: Changing state from %s to %s",
eba76848 1172 fCurrentDetector.Data(),
36c99a6a 1173 status->GetStatusName(),
eba76848 1174 status->GetStatusName(newStatus));
cb343cfd 1175 Log("SHUTTLE", actionStr);
1176 SetLastAction(actionStr);
5164a766 1177
2bb7b766 1178 status->SetStatus(newStatus);
1179 if (increaseCount) status->IncreaseCount();
5164a766 1180
2bb7b766 1181 AliCDBManager::Instance()->GetStorage(fgkLocalCDB)->Put(fStatusEntry);
e7f62f16 1182
1183 SendMLInfo();
5164a766 1184}
e7f62f16 1185
1186//______________________________________________________________________________________________
1187void AliShuttle::SendMLInfo()
1188{
1189 //
1190 // sends ML information about the current status of the current detector being processed
1191 //
1192
1193 AliShuttleStatus* status = dynamic_cast<AliShuttleStatus*> (fStatusEntry->GetObject());
1194
1195 if (!status){
3301427a 1196 Log("SHUTTLE", "SendMLInfo - UNEXPECTED: status could not be read from current CDB entry");
e7f62f16 1197 return;
1198 }
1199
1200 TMonaLisaText mlStatus(Form("%s_status", fCurrentDetector.Data()), status->GetStatusName());
1201 TMonaLisaValue mlRetryCount(Form("%s_count", fCurrentDetector.Data()), status->GetCount());
1202
1203 TList mlList;
1204 mlList.Add(&mlStatus);
1205 mlList.Add(&mlRetryCount);
1206
1207 fMonaLisa->SendParameters(&mlList);
1208}
1209
5164a766 1210//______________________________________________________________________________________________
1211Bool_t AliShuttle::ContinueProcessing()
1212{
9827400b 1213 // this function reads the AliShuttleStatus information from CDB and
1214 // checks if the processing should be continued
1215 // if yes it returns kTRUE and updates the AliShuttleStatus with nextStatus
2bb7b766 1216
57c1a579 1217 if (!fConfig->HostProcessDetector(fCurrentDetector)) return kFALSE;
1218
1219 AliPreprocessor* aPreprocessor =
1220 dynamic_cast<AliPreprocessor*> (fPreprocessorMap.GetValue(fCurrentDetector));
1221 if (!aPreprocessor)
1222 {
c88ad5db 1223 Log("SHUTTLE", Form("ContinueProcessing - %s: no preprocessor registered", fCurrentDetector.Data()));
57c1a579 1224 return kFALSE;
1225 }
1226
2bb7b766 1227 AliShuttleLogbookEntry::Status entryStatus =
eba76848 1228 fLogbookEntry->GetDetectorStatus(fCurrentDetector);
2bb7b766 1229
1230 if(entryStatus != AliShuttleLogbookEntry::kUnprocessed) {
c88ad5db 1231 Log("SHUTTLE", Form("ContinueProcessing - %s is %s",
2bb7b766 1232 fCurrentDetector.Data(),
1233 fLogbookEntry->GetDetectorStatusName(entryStatus)));
1234 return kFALSE;
1235 }
1236
1237 // if we get here, according to Shuttle logbook subdetector is in UNPROCESSED state
be48e3ea 1238
1239 // check if current run is first unprocessed run for current detector
1240 if (fConfig->StrictRunOrder(fCurrentDetector) &&
1241 !fFirstUnprocessed[GetDetPos(fCurrentDetector)])
1242 {
86aa42c3 1243 if (fTestMode == kNone)
1244 {
c88ad5db 1245 Log("SHUTTLE", Form("ContinueProcessing - %s requires strict run ordering"
1246 " but this is not the first unprocessed run!"));
86aa42c3 1247 return kFALSE;
1248 }
1249 else
1250 {
c88ad5db 1251 Log("SHUTTLE", Form("ContinueProcessing - In TESTMODE - "
1252 "Although %s requires strict run ordering "
1253 "and this is not the first unprocessed run, "
1254 "the SHUTTLE continues"));
86aa42c3 1255 }
be48e3ea 1256 }
1257
2bb7b766 1258 AliShuttleStatus* status = ReadShuttleStatus();
1259 if (!status) {
1260 // first time
1261 Log("SHUTTLE", Form("ContinueProcessing - %s: Processing first time",
1262 fCurrentDetector.Data()));
1263 status = new AliShuttleStatus(AliShuttleStatus::kStarted);
1264 return WriteShuttleStatus(status);
1265 }
1266
1267 // The following two cases shouldn't happen if Shuttle Logbook was correctly updated.
1268 // If it happens it may mean Logbook updating failed... let's do it now!
1269 if (status->GetStatus() == AliShuttleStatus::kDone ||
1270 status->GetStatus() == AliShuttleStatus::kFailed){
1271 Log("SHUTTLE", Form("ContinueProcessing - %s is already %s. Updating Shuttle Logbook",
1272 fCurrentDetector.Data(),
1273 status->GetStatusName(status->GetStatus())));
1274 UpdateShuttleLogbook(fCurrentDetector.Data(),
1275 status->GetStatusName(status->GetStatus()));
1276 return kFALSE;
1277 }
1278
3301427a 1279 if (status->GetStatus() == AliShuttleStatus::kStoreError) {
2bb7b766 1280 Log("SHUTTLE",
c88ad5db 1281 Form("ContinueProcessing - %s: Grid storage of one or more "
1282 "objects failed. Trying again now",
2bb7b766 1283 fCurrentDetector.Data()));
9827400b 1284 UpdateShuttleStatus(AliShuttleStatus::kStoreStarted);
1285 if (StoreOCDB()){
c88ad5db 1286 Log("SHUTTLE", Form("ContinueProcessing - %s: all objects "
1287 "successfully stored into main storage",
3301427a 1288 fCurrentDetector.Data()));
2bb7b766 1289 UpdateShuttleStatus(AliShuttleStatus::kDone);
1290 UpdateShuttleLogbook(fCurrentDetector.Data(), "DONE");
1291 } else {
1292 Log("SHUTTLE",
1293 Form("ContinueProcessing - %s: Grid storage failed again",
1294 fCurrentDetector.Data()));
9827400b 1295 UpdateShuttleStatus(AliShuttleStatus::kStoreError);
2bb7b766 1296 }
1297 return kFALSE;
1298 }
1299
1300 // if we get here, there is a restart
57c1a579 1301 Bool_t cont = kFALSE;
2bb7b766 1302
1303 // abort conditions
cb343cfd 1304 if (status->GetCount() >= fConfig->GetMaxRetries()) {
57c1a579 1305 Log("SHUTTLE", Form("ContinueProcessing - %s failed %d times in status %s - "
1306 "Updating Shuttle Logbook", fCurrentDetector.Data(),
2bb7b766 1307 status->GetCount(), status->GetStatusName()));
1308 UpdateShuttleLogbook(fCurrentDetector.Data(), "FAILED");
e7f62f16 1309 UpdateShuttleStatus(AliShuttleStatus::kFailed);
3301427a 1310
1311 // there may still be objects in local OCDB and reference storage
1312 // and FXS databases may be not updated: do it now!
9827400b 1313
1314 // TODO Currently disabled, we want to keep files in case of failure!
1315 // CleanLocalStorage(fgkLocalCDB);
1316 // CleanLocalStorage(fgkLocalRefStorage);
1317 // UpdateTableFailCase();
1318
1319 // Send mail to detector expert!
c88ad5db 1320 Log("SHUTTLE", Form("ContinueProcessing - Sending mail to %s expert...",
1321 fCurrentDetector.Data()));
9827400b 1322 if (!SendMail())
1323 Log("SHUTTLE", Form("ContinueProcessing - Could not send mail to %s expert",
1324 fCurrentDetector.Data()));
3301427a 1325
57c1a579 1326 } else {
1327 Log("SHUTTLE", Form("ContinueProcessing - %s: restarting. "
1328 "Aborted before with %s. Retry number %d.", fCurrentDetector.Data(),
1329 status->GetStatusName(), status->GetCount()));
9827400b 1330 Bool_t increaseCount = kTRUE;
c88ad5db 1331 if (status->GetStatus() == AliShuttleStatus::kDCSError ||
1332 status->GetStatus() == AliShuttleStatus::kDCSStarted)
1333 increaseCount = kFALSE;
9827400b 1334 UpdateShuttleStatus(AliShuttleStatus::kStarted, increaseCount);
57c1a579 1335 cont = kTRUE;
2bb7b766 1336 }
1337
57c1a579 1338 return cont;
5164a766 1339}
1340
1341//______________________________________________________________________________________________
2bb7b766 1342Bool_t AliShuttle::Process(AliShuttleLogbookEntry* entry)
58bc3020 1343{
73abe331 1344 //
b948db8d 1345 // Makes data retrieval for all detectors in the configuration.
2bb7b766 1346 // entry: Shuttle logbook entry, contains run paramenters and status of detectors
1347 // (Unprocessed, Inactive, Failed or Done).
d477ad88 1348 // Returns kFALSE in case of error occured and kTRUE otherwise
73abe331 1349 //
1350
9827400b 1351 if (!entry) return kFALSE;
2bb7b766 1352
1353 fLogbookEntry = entry;
1354
c88ad5db 1355 Log("SHUTTLE", Form("\t\t\t^*^*^*^*^*^*^*^*^*^*^*^* run %d: START ^*^*^*^*^*^*^*^*^*^*^*^*",
9827400b 1356 GetCurrentRun()));
2bb7b766 1357
e7f62f16 1358 // create ML instance that monitors this run
1359 fMonaLisa = new TMonaLisaWriter(Form("%d", GetCurrentRun()), "SHUTTLE", "aliendb1.cern.ch");
1360 // disable monitoring of other parameters that come e.g. from TFile
1361 gMonitoringWriter = 0;
2bb7b766 1362
e7f62f16 1363 // Send the information to ML
1364 TMonaLisaText mlStatus("SHUTTLE_status", "Processing");
9827400b 1365 TMonaLisaText mlRunType("SHUTTLE_runtype", Form("%s (%s)", entry->GetRunType(), entry->GetRunParameter("log")));
e7f62f16 1366
1367 TList mlList;
1368 mlList.Add(&mlStatus);
9827400b 1369 mlList.Add(&mlRunType);
e7f62f16 1370
1371 fMonaLisa->SendParameters(&mlList);
3301427a 1372
9827400b 1373 if (fLogbookEntry->IsDone())
1374 {
1375 Log("SHUTTLE","Process - Shuttle is already DONE. Updating logbook");
1376 UpdateShuttleLogbook("shuttle_done");
1377 fLogbookEntry = 0;
1378 return kTRUE;
1379 }
1380
1381 // read test mode if flag is set
1382 if (fReadTestMode)
1383 {
3d8bc902 1384 fTestMode = kNone;
9827400b 1385 TString logEntry(entry->GetRunParameter("log"));
1386 //printf("log entry = %s\n", logEntry.Data());
1387 TString searchStr("Testmode: ");
1388 Int_t pos = logEntry.Index(searchStr.Data());
1389 //printf("%d\n", pos);
1390 if (pos >= 0)
1391 {
1392 TSubString subStr = logEntry(pos + searchStr.Length(), logEntry.Length());
1393 //printf("%s\n", subStr.String().Data());
1394 TString newStr(subStr.Data());
1395 TObjArray* token = newStr.Tokenize(' ');
1396 if (token)
1397 {
1398 //token->Print();
1399 TObjString* tmpStr = dynamic_cast<TObjString*> (token->First());
1400 if (tmpStr)
1401 {
1402 Int_t testMode = tmpStr->String().Atoi();
1403 if (testMode > 0)
1404 {
c88ad5db 1405 Log("SHUTTLE", Form("Process - Enabling test mode %d", testMode));
9827400b 1406 SetTestMode((TestMode) testMode);
1407 }
1408 }
1409 delete token;
1410 }
1411 }
1412 }
c88ad5db 1413
eba76848 1414 fLogbookEntry->Print("all");
57f50b3c 1415
1416 // Initialization
d477ad88 1417 Bool_t hasError = kFALSE;
5164a766 1418
2bb7b766 1419 AliCDBStorage *mainCDBSto = AliCDBManager::Instance()->GetStorage(fgkMainCDB);
1420 if(mainCDBSto) mainCDBSto->QueryCDB(GetCurrentRun());
1421 AliCDBStorage *mainRefSto = AliCDBManager::Instance()->GetStorage(fgkMainRefStorage);
1422 if(mainRefSto) mainRefSto->QueryCDB(GetCurrentRun());
d477ad88 1423
57f50b3c 1424 // Loop on detectors in the configuration
b948db8d 1425 TIter iter(fConfig->GetDetectors());
2bb7b766 1426 TObjString* aDetector = 0;
b948db8d 1427
be48e3ea 1428 while ((aDetector = (TObjString*) iter.Next()))
1429 {
7bfb2090 1430 fCurrentDetector = aDetector->String();
5164a766 1431
9e080f92 1432 if (ContinueProcessing() == kFALSE) continue;
1433
c88ad5db 1434 Log("SHUTTLE", Form("\t\t\t****** run %d - %s: START ******",
2bb7b766 1435 GetCurrentRun(), aDetector->GetName()));
1436
9d733021 1437 for(Int_t iSys=0;iSys<3;iSys++) fFXSCalled[iSys]=kFALSE;
1438
c88ad5db 1439 Log(fCurrentDetector.Data(), "Process - Starting processing");
85a80aa9 1440
be48e3ea 1441 Int_t pid = fork();
1442
1443 if (pid < 0)
1444 {
c88ad5db 1445 Log("SHUTTLE", "Process - ERROR: Forking failed");
be48e3ea 1446 }
1447 else if (pid > 0)
1448 {
1449 // parent
c88ad5db 1450 Log("SHUTTLE", Form("Process - In parent process of %d - %s: Starting monitoring",
be48e3ea 1451 GetCurrentRun(), aDetector->GetName()));
1452
1453 Long_t begin = time(0);
1454
1455 int status; // to be used with waitpid, on purpose an int (not Int_t)!
1456 while (waitpid(pid, &status, WNOHANG) == 0)
1457 {
1458 Long_t expiredTime = time(0) - begin;
1459
1460 if (expiredTime > fConfig->GetPPTimeOut())
1461 {
9827400b 1462 TString tmp;
c88ad5db 1463 tmp.Form("Process - Process of %s time out. "
1464 "Run time: %d seconds. Killing...",
1465 fCurrentDetector.Data(), expiredTime);
9827400b 1466 Log("SHUTTLE", tmp);
1467 Log(fCurrentDetector, tmp);
be48e3ea 1468
1469 kill(pid, 9);
1470
3301427a 1471 UpdateShuttleStatus(AliShuttleStatus::kPPTimeOut);
be48e3ea 1472 hasError = kTRUE;
1473
1474 gSystem->Sleep(1000);
1475 }
1476 else
1477 {
be48e3ea 1478 gSystem->Sleep(1000);
9827400b 1479
1480 TString checkStr;
1481 checkStr.Form("ps -o vsize --pid %d | tail -n 1", pid);
1482 FILE* pipe = gSystem->OpenPipe(checkStr, "r");
1483 if (!pipe)
1484 {
c88ad5db 1485 Log("SHUTTLE", Form("Process - Error: "
1486 "Could not open pipe to %s", checkStr.Data()));
9827400b 1487 continue;
1488 }
1489
1490 char buffer[100];
1491 if (!fgets(buffer, 100, pipe))
1492 {
c88ad5db 1493 Log("SHUTTLE", "Process - Error: ps did not return anything");
9827400b 1494 gSystem->ClosePipe(pipe);
1495 continue;
1496 }
1497 gSystem->ClosePipe(pipe);
1498
1499 //Log("SHUTTLE", Form("ps returned %s", buffer));
1500
1501 Int_t mem = 0;
1502 if ((sscanf(buffer, "%d\n", &mem) != 1) || !mem)
1503 {
c88ad5db 1504 Log("SHUTTLE", "Process - Error: Could not parse output of ps");
9827400b 1505 continue;
1506 }
1507
1508 if (expiredTime % 60 == 0)
c88ad5db 1509 Log("SHUTTLE", Form("Process - %s: Checking process. "
1510 "Run time: %d seconds - Memory consumption: %d KB",
1511 fCurrentDetector.Data(), expiredTime, mem));
9827400b 1512
1513 if (mem > fConfig->GetPPMaxMem())
1514 {
1515 TString tmp;
c88ad5db 1516 tmp.Form("Process - Process exceeds maximum allowed memory "
1517 "(%d KB > %d KB). Killing...",
9827400b 1518 mem, fConfig->GetPPMaxMem());
1519 Log("SHUTTLE", tmp);
1520 Log(fCurrentDetector, tmp);
1521
1522 kill(pid, 9);
1523
1524 UpdateShuttleStatus(AliShuttleStatus::kPPOutOfMemory);
1525 hasError = kTRUE;
1526
1527 gSystem->Sleep(1000);
1528 }
be48e3ea 1529 }
1530 }
1531
c88ad5db 1532 Log("SHUTTLE", Form("Process - In parent process of %d - %s: Client has terminated.",
be48e3ea 1533 GetCurrentRun(), aDetector->GetName()));
1534
1535 if (WIFEXITED(status))
1536 {
1537 Int_t returnCode = WEXITSTATUS(status);
1538
c88ad5db 1539 Log("SHUTTLE", Form("Process - %s: the return code is %d", fCurrentDetector.Data(),
3301427a 1540 returnCode));
be48e3ea 1541
9827400b 1542 if (returnCode == 0) hasError = kTRUE;
be48e3ea 1543 }
1544 }
1545 else if (pid == 0)
1546 {
1547 // client
c88ad5db 1548 Log("SHUTTLE", Form("Process - In client process of %d - %s", GetCurrentRun(),
1549 aDetector->GetName()));
be48e3ea 1550
c88ad5db 1551 Log("SHUTTLE", Form("Process - Redirecting output to %s log",fCurrentDetector.Data()));
ffa29e93 1552
546242fb 1553 if ((freopen(GetLogFileName(fCurrentDetector), "a", stdout)) == 0)
ffa29e93 1554 {
c88ad5db 1555 Log("SHUTTLE", "Process - Could not freopen stdout");
ffa29e93 1556 }
1557 else
1558 {
1559 fOutputRedirected = kTRUE;
1560 if ((dup2(fileno(stdout), fileno(stderr))) < 0)
c88ad5db 1561 Log("SHUTTLE", "Process - Could not redirect stderr");
ffa29e93 1562
1563 }
1564
5bac2bde 1565 TString wd = gSystem->WorkingDirectory();
1566 TString tmpDir = Form("%s/%s_process",GetShuttleTempDir(),fCurrentDetector.Data());
1567
1568 gSystem->mkdir(tmpDir.Data());
1569 gSystem->ChangeDirectory(tmpDir.Data());
1570
9827400b 1571 Bool_t success = ProcessCurrentDetector();
5bac2bde 1572
1573 gSystem->ChangeDirectory(wd.Data());
1574
1575 gSystem->Exec(Form("rm -rf %s",tmpDir.Data()));
1576
9827400b 1577 if (success) // Preprocessor finished successfully!
1578 {
3301427a 1579 // Update time_processed field in FXS DB
1580 if (UpdateTable() == kFALSE)
5bac2bde 1581 Log("SHUTTLE", Form("Process - %s: Could not update FXS databases!",
1582 fCurrentDetector.Data()));
3301427a 1583
1584 // Transfer the data from local storage to main storage (Grid)
1585 UpdateShuttleStatus(AliShuttleStatus::kStoreStarted);
1586 if (StoreOCDB() == kFALSE)
1587 {
c88ad5db 1588 Log("SHUTTLE",
1589 Form("\t\t\t****** run %d - %s: STORAGE ERROR ******",
3301427a 1590 GetCurrentRun(), aDetector->GetName()));
1591 UpdateShuttleStatus(AliShuttleStatus::kStoreError);
9827400b 1592 success = kFALSE;
3301427a 1593 } else {
c88ad5db 1594 Log("SHUTTLE",
1595 Form("\t\t\t****** run %d - %s: DONE ******",
3301427a 1596 GetCurrentRun(), aDetector->GetName()));
1597 UpdateShuttleStatus(AliShuttleStatus::kDone);
9827400b 1598 UpdateShuttleLogbook(fCurrentDetector, "DONE");
3301427a 1599 }
c88ad5db 1600 } else
1601 {
1602 Log("SHUTTLE",
1603 Form("\t\t\t****** run %d - %s: PP ERROR ******",
1604 GetCurrentRun(), aDetector->GetName()));
be48e3ea 1605 }
1606
4b95672b 1607 for (UInt_t iSys=0; iSys<3; iSys++)
1608 {
1609 if (fFXSCalled[iSys]) fFXSlist[iSys].Clear();
1610 }
1611
c88ad5db 1612 Log("SHUTTLE", Form("Process - Client process of %d - %s is exiting now with %d.",
9827400b 1613 GetCurrentRun(), aDetector->GetName(), success));
be48e3ea 1614
1615 // the client exits here
9827400b 1616 gSystem->Exit(success);
be48e3ea 1617
1618 AliError("We should never get here!!!");
1619 }
7bfb2090 1620 }
5164a766 1621
c88ad5db 1622 Log("SHUTTLE", Form("\t\t\t^*^*^*^*^*^*^*^*^*^*^*^* run %d: FINISH ^*^*^*^*^*^*^*^*^*^*^*^*",
2bb7b766 1623 GetCurrentRun()));
1624
1625 //check if shuttle is done for this run, if so update logbook
1626 TObjArray checkEntryArray;
1627 checkEntryArray.SetOwner(1);
9e080f92 1628 TString whereClause = Form("where run=%d", GetCurrentRun());
1629 if (!QueryShuttleLogbook(whereClause.Data(), checkEntryArray) || checkEntryArray.GetEntries() == 0) {
1630 Log("SHUTTLE", Form("Process - Warning: Cannot check status of run %d on Shuttle logbook!",
1631 GetCurrentRun()));
1632 return hasError == kFALSE;
1633 }
b948db8d 1634
9e080f92 1635 AliShuttleLogbookEntry* checkEntry = dynamic_cast<AliShuttleLogbookEntry*>
1636 (checkEntryArray.At(0));
2bb7b766 1637
9e080f92 1638 if (checkEntry)
1639 {
1640 if (checkEntry->IsDone())
be48e3ea 1641 {
9e080f92 1642 Log("SHUTTLE","Process - Shuttle is DONE. Updating logbook");
1643 UpdateShuttleLogbook("shuttle_done");
1644 }
1645 else
1646 {
1647 for (UInt_t iDet=0; iDet<NDetectors(); iDet++)
be48e3ea 1648 {
9e080f92 1649 if (checkEntry->GetDetectorStatus(iDet) == AliShuttleLogbookEntry::kUnprocessed)
be48e3ea 1650 {
9e080f92 1651 AliDebug(2, Form("Run %d: setting %s as \"not first time unprocessed\"",
1652 checkEntry->GetRun(), GetDetName(iDet)));
1653 fFirstUnprocessed[iDet] = kFALSE;
be48e3ea 1654 }
1655 }
2bb7b766 1656 }
1657 }
1658
e7f62f16 1659 // remove ML instance
1660 delete fMonaLisa;
1661 fMonaLisa = 0;
1662
2bb7b766 1663 fLogbookEntry = 0;
85a80aa9 1664
a7160fe9 1665 return hasError == kFALSE;
73abe331 1666}
1667
b948db8d 1668//______________________________________________________________________________________________
9827400b 1669Bool_t AliShuttle::ProcessCurrentDetector()
73abe331 1670{
1671 //
2bb7b766 1672 // Makes data retrieval just for a specific detector (fCurrentDetector).
73abe331 1673 // Threre should be a configuration for this detector.
73abe331 1674
1d172743 1675 Log("SHUTTLE", Form("ProcessCurrentDetector - Retrieving values for %s, run %d",
1676 fCurrentDetector.Data(), GetCurrentRun()));
73abe331 1677
2d9019b4 1678 if (!CleanReferenceStorage(fCurrentDetector.Data()))
546242fb 1679 return kFALSE;
1680
1d172743 1681 TMap* dcsMap = new TMap();
3301427a 1682
1683 // call preprocessor
1684 AliPreprocessor* aPreprocessor =
1685 dynamic_cast<AliPreprocessor*> (fPreprocessorMap.GetValue(fCurrentDetector));
1686
1687 aPreprocessor->Initialize(GetCurrentRun(), GetCurrentStartTime(), GetCurrentEndTime());
1688
1689 Bool_t processDCS = aPreprocessor->ProcessDCS();
d477ad88 1690
651fdaab 1691 if (!processDCS)
1692 {
1d172743 1693 Log(fCurrentDetector, "ProcessCurrentDetector -"
1694 " The preprocessor requested to skip the retrieval of DCS values");
651fdaab 1695 }
8b739301 1696 else if (fTestMode & kSkipDCS)
2c15234c 1697 {
1d172743 1698 Log(fCurrentDetector, "ProcessCurrentDetector - In TESTMODE: Skipping DCS processing");
9827400b 1699 }
1700 else if (fTestMode & kErrorDCS)
1701 {
1d172743 1702 Log(fCurrentDetector, "ProcessCurrentDetector - In TESTMODE: Simulating DCS error");
3d8bc902 1703 UpdateShuttleStatus(AliShuttleStatus::kDCSStarted);
9827400b 1704 UpdateShuttleStatus(AliShuttleStatus::kDCSError);
1d172743 1705 delete dcsMap;
9827400b 1706 return kFALSE;
2c15234c 1707 } else {
3301427a 1708
1709 UpdateShuttleStatus(AliShuttleStatus::kDCSStarted);
1710
1d172743 1711 // Query DCS archive
1712 Int_t nServers = fConfig->GetNServers(fCurrentDetector);
a038aa70 1713
1d172743 1714 for (int iServ=0; iServ<nServers; iServ++)
2c15234c 1715 {
1d172743 1716
1717 TString host(fConfig->GetDCSHost(fCurrentDetector, iServ));
1718 Int_t port = fConfig->GetDCSPort(fCurrentDetector, iServ);
542b6cc8 1719 Int_t multiSplit = fConfig->GetMultiSplit(fCurrentDetector, iServ);
1720
1790d4b7 1721 Log(fCurrentDetector, Form("ProcessCurrentDetector -"
1722 " Querying DCS Amanda server %s:%d (%d of %d)",
1723 host.Data(), port, iServ+1, nServers));
1d172743 1724
1725 TMap* aliasMap = 0;
1726 TMap* dpMap = 0;
1727
1728 if (fConfig->GetDCSAliases(fCurrentDetector, iServ)->GetEntries() > 0)
2c15234c 1729 {
1d172743 1730 aliasMap = GetValueSet(host, port,
542b6cc8 1731 fConfig->GetDCSAliases(fCurrentDetector, iServ),
1732 kAlias, multiSplit);
1d172743 1733 if (!aliasMap)
1734 {
1735 Log(fCurrentDetector,
1736 Form("ProcessCurrentDetector -"
1737 " Error retrieving DCS aliases from server %s",
1738 host.Data()));
1739 UpdateShuttleStatus(AliShuttleStatus::kDCSError);
a038aa70 1740 delete dcsMap;
1d172743 1741 return kFALSE;
1742 }
2c15234c 1743 }
a038aa70 1744
1d172743 1745 if (fConfig->GetDCSDataPoints(fCurrentDetector, iServ)->GetEntries() > 0)
a038aa70 1746 {
1d172743 1747 dpMap = GetValueSet(host, port,
542b6cc8 1748 fConfig->GetDCSDataPoints(fCurrentDetector, iServ),
1749 kDP, multiSplit);
1d172743 1750 if (!dpMap)
1751 {
1752 Log(fCurrentDetector,
1753 Form("ProcessCurrentDetector -"
1754 " Error retrieving DCS data points from server %s",
1755 host.Data()));
1756 UpdateShuttleStatus(AliShuttleStatus::kDCSError);
1757 if (aliasMap) delete aliasMap;
1758 delete dcsMap;
1759 return kFALSE;
1760 }
a038aa70 1761 }
1d172743 1762
1763 // merge aliasMap and dpMap into dcsMap
1764 if(aliasMap) {
1765 TIter iter(aliasMap);
a038aa70 1766 TObjString* key = 0;
1767 while ((key = (TObjString*) iter.Next()))
1d172743 1768 dcsMap->Add(key, aliasMap->GetValue(key->String()));
1769
1770 aliasMap->SetOwner(kFALSE);
1771 delete aliasMap;
1772 }
1773
1774 if(dpMap) {
1775 TIter iter(dpMap);
1776 TObjString* key = 0;
1777 while ((key = (TObjString*) iter.Next()))
1778 dcsMap->Add(key, dpMap->GetValue(key->String()));
1779
1780 dpMap->SetOwner(kFALSE);
1781 delete dpMap;
a038aa70 1782 }
73abe331 1783 }
1784 }
dc25836b 1785
2bb7b766 1786 // DCS Archive DB processing successful. Call Preprocessor!
85a80aa9 1787 UpdateShuttleStatus(AliShuttleStatus::kPPStarted);
a7160fe9 1788
a038aa70 1789 UInt_t returnValue = aPreprocessor->Process(dcsMap);
b948db8d 1790
3301427a 1791 if (returnValue > 0) // Preprocessor error!
1792 {
c88ad5db 1793 Log(fCurrentDetector, Form("ProcessCurrentDetector - "
1794 "Preprocessor failed. Process returned %d.", returnValue));
cb343cfd 1795 UpdateShuttleStatus(AliShuttleStatus::kPPError);
a038aa70 1796 dcsMap->DeleteAll();
1797 delete dcsMap;
9827400b 1798 return kFALSE;
1799 }
1800
1801 // preprocessor ok!
1802 UpdateShuttleStatus(AliShuttleStatus::kPPDone);
1803 Log(fCurrentDetector, Form("ProcessCurrentDetector - %s preprocessor returned success",
1804 fCurrentDetector.Data()));
b948db8d 1805
a038aa70 1806 dcsMap->DeleteAll();
1807 delete dcsMap;
b948db8d 1808
9827400b 1809 return kTRUE;
2bb7b766 1810}
1811
1812//______________________________________________________________________________________________
1813Bool_t AliShuttle::QueryShuttleLogbook(const char* whereClause,
1814 TObjArray& entries)
1815{
9827400b 1816 // Query DAQ's Shuttle logbook and fills detector status object.
1817 // Call QueryRunParameters to query DAQ logbook for run parameters.
1818 //
2bb7b766 1819
fc5a4708 1820 entries.SetOwner(1);
1821
2bb7b766 1822 // check connection, in case connect
be48e3ea 1823 if(!Connect(3)) return kFALSE;
2bb7b766 1824
1825 TString sqlQuery;
441b0e9c 1826 sqlQuery = Form("select * from %s %s order by run", fConfig->GetShuttlelbTable(), whereClause);
2bb7b766 1827
be48e3ea 1828 TSQLResult* aResult = fServer[3]->Query(sqlQuery);
2bb7b766 1829 if (!aResult) {
1830 AliError(Form("Can't execute query <%s>!", sqlQuery.Data()));
1831 return kFALSE;
1832 }
1833
fc5a4708 1834 AliDebug(2,Form("Query = %s", sqlQuery.Data()));
1835
2bb7b766 1836 if(aResult->GetRowCount() == 0) {
c88ad5db 1837 Log("SHUTTLE", "No entries in Shuttle Logbook match request");
9827400b 1838 delete aResult;
1839 return kTRUE;
2bb7b766 1840 }
1841
1842 // TODO Check field count!
db99d43e 1843 const UInt_t nCols = 23;
2bb7b766 1844 if (aResult->GetFieldCount() != (Int_t) nCols) {
c88ad5db 1845 Log("SHUTTLE", "Invalid SQL result field number!");
2bb7b766 1846 delete aResult;
1847 return kFALSE;
1848 }
1849
2bb7b766 1850 TSQLRow* aRow;
1851 while ((aRow = aResult->Next())) {
1852 TString runString(aRow->GetField(0), aRow->GetFieldLength(0));
1853 Int_t run = runString.Atoi();
1854
eba76848 1855 AliShuttleLogbookEntry *entry = QueryRunParameters(run);
1856 if (!entry)
1857 continue;
2bb7b766 1858
1859 // loop on detectors
eba76848 1860 for(UInt_t ii = 0; ii < nCols; ii++)
1861 entry->SetDetectorStatus(aResult->GetFieldName(ii), aRow->GetField(ii));
2bb7b766 1862
eba76848 1863 entries.AddLast(entry);
2bb7b766 1864 delete aRow;
1865 }
1866
2bb7b766 1867 delete aResult;
1868 return kTRUE;
1869}
1870
1871//______________________________________________________________________________________________
eba76848 1872AliShuttleLogbookEntry* AliShuttle::QueryRunParameters(Int_t run)
2bb7b766 1873{
eba76848 1874 //
1875 // Retrieve run parameters written in the DAQ logbook and sets them into AliShuttleLogbookEntry object
1876 //
2bb7b766 1877
1878 // check connection, in case connect
be48e3ea 1879 if (!Connect(3))
eba76848 1880 return 0;
2bb7b766 1881
1882 TString sqlQuery;
2c15234c 1883 sqlQuery.Form("select * from %s where run=%d", fConfig->GetDAQlbTable(), run);
2bb7b766 1884
be48e3ea 1885 TSQLResult* aResult = fServer[3]->Query(sqlQuery);
2bb7b766 1886 if (!aResult) {
c88ad5db 1887 Log("SHUTTLE", Form("Can't execute query <%s>!", sqlQuery.Data()));
eba76848 1888 return 0;
2bb7b766 1889 }
1890
eba76848 1891 if (aResult->GetRowCount() == 0) {
2bb7b766 1892 Log("SHUTTLE", Form("QueryRunParameters - No entry in DAQ Logbook for run %d. Skipping", run));
1893 delete aResult;
eba76848 1894 return 0;
2bb7b766 1895 }
1896
eba76848 1897 if (aResult->GetRowCount() > 1) {
c88ad5db 1898 Log("SHUTTLE", Form("QueryRunParameters - UNEXPECTED: "
1899 "more than one entry in DAQ Logbook for run %d!", run));
2bb7b766 1900 delete aResult;
eba76848 1901 return 0;
2bb7b766 1902 }
1903
eba76848 1904 TSQLRow* aRow = aResult->Next();
1905 if (!aRow)
1906 {
c88ad5db 1907 Log("SHUTTLE", Form("QueryRunParameters - Could not retrieve row for run %d. Skipping", run));
eba76848 1908 delete aResult;
1909 return 0;
1910 }
2bb7b766 1911
eba76848 1912 AliShuttleLogbookEntry* entry = new AliShuttleLogbookEntry(run);
2bb7b766 1913
eba76848 1914 for (Int_t ii = 0; ii < aResult->GetFieldCount(); ii++)
1915 entry->SetRunParameter(aResult->GetFieldName(ii), aRow->GetField(ii));
2bb7b766 1916
eba76848 1917 UInt_t startTime = entry->GetStartTime();
1918 UInt_t endTime = entry->GetEndTime();
1919
1920 if (!startTime || !endTime || startTime > endTime) {
1921 Log("SHUTTLE",
1922 Form("QueryRunParameters - Invalid parameters for Run %d: startTime = %d, endTime = %d",
1923 run, startTime, endTime));
1924 delete entry;
2bb7b766 1925 delete aRow;
eba76848 1926 delete aResult;
1927 return 0;
2bb7b766 1928 }
1929
eba76848 1930 delete aRow;
2bb7b766 1931 delete aResult;
eba76848 1932
1933 return entry;
2bb7b766 1934}
1935
a038aa70 1936//______________________________________________________________________________________________
1937TMap* AliShuttle::GetValueSet(const char* host, Int_t port, const TSeqCollection* entries,
542b6cc8 1938 DCSType type, Int_t multiSplit)
a038aa70 1939{
1940 // Retrieve all "entry" data points from the DCS server
1941 // host, port: TSocket connection parameters
1942 // entries: list of name of the alias or data point
1943 // type: kAlias or kDP
1944 // returns TMap of values, 0 when failure
542b6cc8 1945
1946 AliDCSClient client(host, port, fTimeout, fRetries, multiSplit);
b41b252a 1947
a038aa70 1948 TMap* result = 0;
b41b252a 1949 if (type == kAlias)
a038aa70 1950 {
b41b252a 1951 result = client.GetAliasValues(entries, GetCurrentStartTime(),
1952 GetCurrentEndTime());
1953 }
1954 else if (type == kDP)
1955 {
1956 result = client.GetDPValues(entries, GetCurrentStartTime(),
1957 GetCurrentEndTime());
1958 }
a038aa70 1959
b41b252a 1960 if (result == 0)
1961 {
1962 Log(fCurrentDetector.Data(), Form("GetValueSet - Can't get entries! Reason: %s",
1790d4b7 1963 client.GetErrorString(client.GetResultErrorCode())));
1964 if (client.GetResultErrorCode() == AliDCSClient::fgkServerError)
1965 Log(fCurrentDetector.Data(), Form("GetValueSet - Server error code: %s",
1966 client.GetServerError().Data()));
a038aa70 1967
b41b252a 1968 return 0;
a038aa70 1969 }
b41b252a 1970
a038aa70 1971 return result;
1972}
b41b252a 1973
b948db8d 1974//______________________________________________________________________________________________
57f50b3c 1975const char* AliShuttle::GetFile(Int_t system, const char* detector,
1976 const char* id, const char* source)
b948db8d 1977{
9827400b 1978 // Get calibration file from file exchange servers
1979 // First queris the FXS database for the file name, using the run, detector, id and source info
1980 // then calls RetrieveFile(filename) for actual copy to local disk
1981 // run: current run being processed (given by Logbook entry fLogbookEntry)
1982 // detector: the Preprocessor name
1983 // id: provided as a parameter by the Preprocessor
1984 // source: provided by the Preprocessor through GetFileSources function
1985
1986 // check if test mode should simulate a FXS error
1987 if (fTestMode & kErrorFXSFiles)
1988 {
1989 Log(detector, Form("GetFile - In TESTMODE - Simulating error while connecting to %s FXS", GetSystemName(system)));
1990 return 0;
1991 }
1992
57f50b3c 1993 // check connection, in case connect
9d733021 1994 if (!Connect(system))
eba76848 1995 {
9d733021 1996 Log(detector, Form("GetFile - Couldn't connect to %s FXS database", GetSystemName(system)));
57f50b3c 1997 return 0;
1998 }
1999
2000 // Query preparation
9d733021 2001 TString sourceName(source);
d386d623 2002 Int_t nFields = 3;
2003 TString sqlQueryStart = Form("select filePath,size,fileChecksum from %s where",
2004 fConfig->GetFXSdbTable(system));
2005 TString whereClause = Form("run=%d and detector=\"%s\" and fileId=\"%s\"",
2006 GetCurrentRun(), detector, id);
2007
9d733021 2008 if (system == kDAQ)
2009 {
d386d623 2010 whereClause += Form(" and DAQsource=\"%s\"", source);
57f50b3c 2011 }
9d733021 2012 else if (system == kDCS)
eba76848 2013 {
9d733021 2014 sourceName="none";
57f50b3c 2015 }
9d733021 2016 else if (system == kHLT)
9e080f92 2017 {
d386d623 2018 whereClause += Form(" and DDLnumbers=\"%s\"", source);
9d733021 2019 nFields = 3;
9e080f92 2020 }
2021
9e080f92 2022 TString sqlQuery = Form("%s %s", sqlQueryStart.Data(), whereClause.Data());
2023
2024 AliDebug(2, Form("SQL query: \n%s",sqlQuery.Data()));
2025
2026 // Query execution
2027 TSQLResult* aResult = 0;
9d733021 2028 aResult = dynamic_cast<TSQLResult*> (fServer[system]->Query(sqlQuery));
9e080f92 2029 if (!aResult) {
9d733021 2030 Log(detector, Form("GetFileName - Can't execute SQL query to %s database for: id = %s, source = %s",
2031 GetSystemName(system), id, sourceName.Data()));
9e080f92 2032 return 0;
2033 }
2034
2035 if(aResult->GetRowCount() == 0)
2036 {
2037 Log(detector,
9d733021 2038 Form("GetFileName - No entry in %s FXS db for: id = %s, source = %s",
2039 GetSystemName(system), id, sourceName.Data()));
9e080f92 2040 delete aResult;
2041 return 0;
2042 }
2bb7b766 2043
9e080f92 2044 if (aResult->GetRowCount() > 1) {
2045 Log(detector,
9d733021 2046 Form("GetFileName - More than one entry in %s FXS db for: id = %s, source = %s",
2047 GetSystemName(system), id, sourceName.Data()));
9e080f92 2048 delete aResult;
2049 return 0;
2050 }
2051
9d733021 2052 if (aResult->GetFieldCount() != nFields) {
9e080f92 2053 Log(detector,
9d733021 2054 Form("GetFileName - Wrong field count in %s FXS db for: id = %s, source = %s",
2055 GetSystemName(system), id, sourceName.Data()));
9e080f92 2056 delete aResult;
2057 return 0;
2058 }
2059
2060 TSQLRow* aRow = dynamic_cast<TSQLRow*> (aResult->Next());
2061
2062 if (!aRow){
9d733021 2063 Log(detector, Form("GetFileName - Empty set result in %s FXS db from query: id = %s, source = %s",
2064 GetSystemName(system), id, sourceName.Data()));
9e080f92 2065 delete aResult;
2066 return 0;
2067 }
2068
2069 TString filePath(aRow->GetField(0), aRow->GetFieldLength(0));
2070 TString fileSize(aRow->GetField(1), aRow->GetFieldLength(1));
d386d623 2071 TString fileChecksum(aRow->GetField(2), aRow->GetFieldLength(2));
9e080f92 2072
2073 delete aResult;
2074 delete aRow;
2075
d386d623 2076 AliDebug(2, Form("filePath = %s; size = %s, fileChecksum = %s",
2077 filePath.Data(), fileSize.Data(), fileChecksum.Data()));
9e080f92 2078
9e080f92 2079 // retrieved file is renamed to make it unique
9d733021 2080 TString localFileName = Form("%s_%s_%d_%s_%s.shuttle",
2081 GetSystemName(system), detector, GetCurrentRun(), id, sourceName.Data());
2082
9e080f92 2083
9d733021 2084 // file retrieval from FXS
4b95672b 2085 UInt_t nRetries = 0;
2086 UInt_t maxRetries = 3;
2087 Bool_t result = kFALSE;
2088
2089 // copy!! if successful TSystem::Exec returns 0
2090 while(nRetries++ < maxRetries) {
2091 AliDebug(2, Form("Trying to copy file. Retry # %d", nRetries));
2092 result = RetrieveFile(system, filePath.Data(), localFileName.Data());
2093 if(!result)
2094 {
2095 Log(detector, Form("GetFileName - Copy of file %s from %s FXS failed",
9d733021 2096 filePath.Data(), GetSystemName(system)));
4b95672b 2097 continue;
4f0749a8 2098 }
9e080f92 2099
d386d623 2100 if (fileChecksum.Length()>0)
4b95672b 2101 {
2102 // compare md5sum of local file with the one stored in the FXS DB
2103 Int_t md5Comp = gSystem->Exec(Form("md5sum %s/%s |grep %s 2>&1 > /dev/null",
d386d623 2104 GetShuttleTempDir(), localFileName.Data(), fileChecksum.Data()));
9e080f92 2105
4b95672b 2106 if (md5Comp != 0)
2107 {
2108 Log(detector, Form("GetFileName - md5sum of file %s does not match with local copy!",
2109 filePath.Data()));
2110 result = kFALSE;
2111 continue;
2112 }
d386d623 2113 } else {
2114 Log(fCurrentDetector, Form("GetFile - md5sum of file %s not set in %s database, skipping comparison",
2115 filePath.Data(), GetSystemName(system)));
9d733021 2116 }
4b95672b 2117 if (result) break;
9e080f92 2118 }
2119
4b95672b 2120 if(!result) return 0;
2121
9d733021 2122 fFXSCalled[system]=kTRUE;
2123 TObjString *fileParams = new TObjString(Form("%s#!?!#%s", id, sourceName.Data()));
2124 fFXSlist[system].Add(fileParams);
9e080f92 2125
2126 static TString fullLocalFileName;
1bcd28db 2127 fullLocalFileName.Form("%s/%s", GetShuttleTempDir(), localFileName.Data());
36c99a6a 2128
c88ad5db 2129 Log(fCurrentDetector, Form("GetFile - Retrieved file with id %s and "
2130 "source %s from %s to %s", id, source,
2131 GetSystemName(system), fullLocalFileName.Data()));
9e080f92 2132
2133 return fullLocalFileName.Data();
2bb7b766 2134}
2135
2136//______________________________________________________________________________________________
9d733021 2137Bool_t AliShuttle::RetrieveFile(UInt_t system, const char* fxsFileName, const char* localFileName)
9e080f92 2138{
9827400b 2139 //
2140 // Copies file from FXS to local Shuttle machine
2141 //
2bb7b766 2142
9e080f92 2143 // check temp directory: trying to cd to temp; if it does not exist, create it
9d733021 2144 AliDebug(2, Form("Copy file %s from %s FXS into %s/%s",
2145 GetSystemName(system), fxsFileName, GetShuttleTempDir(), localFileName));
9e080f92 2146
36c99a6a 2147 void* dir = gSystem->OpenDirectory(GetShuttleTempDir());
9e080f92 2148 if (dir == NULL) {
36c99a6a 2149 if (gSystem->mkdir(GetShuttleTempDir(), kTRUE)) {
2150 AliError(Form("Can't open directory <%s>", GetShuttleTempDir()));
9e080f92 2151 return kFALSE;
2152 }
2153
2154 } else {
2155 gSystem->FreeDirectory(dir);
2156 }
2157
9d733021 2158 TString baseFXSFolder;
2159 if (system == kDAQ)
2160 {
2161 baseFXSFolder = "FES/";
2162 }
2163 else if (system == kDCS)
2164 {
2165 baseFXSFolder = "";
2166 }
2167 else if (system == kHLT)
2168 {
42fde080 2169 baseFXSFolder = "/opt/FXS/";
9d733021 2170 }
2171
2172
2173 TString command = Form("scp -oPort=%d -2 %s@%s:%s%s %s/%s",
2174 fConfig->GetFXSPort(system),
2175 fConfig->GetFXSUser(system),
2176 fConfig->GetFXSHost(system),
2177 baseFXSFolder.Data(),
2178 fxsFileName,
36c99a6a 2179 GetShuttleTempDir(),
9e080f92 2180 localFileName);
2181
2182 AliDebug(2, Form("%s",command.Data()));
2183
4b95672b 2184 Bool_t result = (gSystem->Exec(command.Data()) == 0);
9e080f92 2185
4b95672b 2186 return result;
9e080f92 2187}
2188
2189//______________________________________________________________________________________________
9d733021 2190TList* AliShuttle::GetFileSources(Int_t system, const char* detector, const char* id)
2191{
9827400b 2192 //
2193 // Get sources producing the condition file Id from file exchange servers
4a33bdd9 2194 // if id is NULL all sources are returned (distinct)
9827400b 2195 //
1bcd28db 2196
2197 Log(detector, Form("GetFileSources - Retrieving sources with id %s from %s", id, GetSystemName(system)));
9827400b 2198
2199 // check if test mode should simulate a FXS error
2200 if (fTestMode & kErrorFXSSources)
2201 {
2202 Log(detector, Form("GetFileSources - In TESTMODE - Simulating error while connecting to %s FXS", GetSystemName(system)));
2203 return 0;
2204 }
2205
9d733021 2206 if (system == kDCS)
2207 {
c88ad5db 2208 Log(detector, "GetFileSources - WARNING: DCS system has only one source of data!");
6297b37d 2209 TList *list = new TList();
2210 list->SetOwner(1);
2211 list->Add(new TObjString(" "));
2212 return list;
9d733021 2213 }
9e080f92 2214
2215 // check connection, in case connect
9d733021 2216 if (!Connect(system))
2217 {
4a33bdd9 2218 Log(detector, Form("GetFileSources - Couldn't connect to %s FXS database", GetSystemName(system)));
9d733021 2219 return NULL;
9e080f92 2220 }
2221
9d733021 2222 TString sourceName = 0;
2223 if (system == kDAQ)
2224 {
2225 sourceName = "DAQsource";
2226 } else if (system == kHLT)
2227 {
2228 sourceName = "DDLnumbers";
2229 }
2230
4a33bdd9 2231 TString sqlQueryStart = Form("select distinct %s from %s where", sourceName.Data(), fConfig->GetFXSdbTable(system));
2232 TString whereClause = Form("run=%d and detector=\"%s\"",
2233 GetCurrentRun(), detector);
2234 if (id)
2235 whereClause += Form(" and fileId=\"%s\"", id);
9e080f92 2236 TString sqlQuery = Form("%s %s", sqlQueryStart.Data(), whereClause.Data());
2237
2238 AliDebug(2, Form("SQL query: \n%s",sqlQuery.Data()));
2239
2240 // Query execution
2241 TSQLResult* aResult;
9d733021 2242 aResult = fServer[system]->Query(sqlQuery);
9e080f92 2243 if (!aResult) {
9d733021 2244 Log(detector, Form("GetFileSources - Can't execute SQL query to %s database for id: %s",
2245 GetSystemName(system), id));
9e080f92 2246 return 0;
2247 }
2248
86aa42c3 2249 TList *list = new TList();
2250 list->SetOwner(1);
2251
9d733021 2252 if (aResult->GetRowCount() == 0)
2253 {
9e080f92 2254 Log(detector,
9d733021 2255 Form("GetFileSources - No entry in %s FXS table for id: %s", GetSystemName(system), id));
9e080f92 2256 delete aResult;
86aa42c3 2257 return list;
9e080f92 2258 }
2259
1bcd28db 2260 Log(detector, Form("GetFileSources - Found %d sources", aResult->GetRowCount()));
9e080f92 2261
1bcd28db 2262 TSQLRow* aRow;
9d733021 2263 while ((aRow = aResult->Next()))
2264 {
9e080f92 2265
9d733021 2266 TString source(aRow->GetField(0), aRow->GetFieldLength(0));
2267 AliDebug(2, Form("%s = %s", sourceName.Data(), source.Data()));
2268 list->Add(new TObjString(source));
9e080f92 2269 delete aRow;
2270 }
9d733021 2271
9e080f92 2272 delete aResult;
2273
2274 return list;
2bb7b766 2275}
2276
4a33bdd9 2277//______________________________________________________________________________________________
2278TList* AliShuttle::GetFileIDs(Int_t system, const char* detector, const char* source)
2279{
2280 //
2281 // Get all ids of condition files produced by a given source from file exchange servers
2282 //
2283
1bcd28db 2284 Log(detector, Form("GetFileIDs - Retrieving ids with source %s with %s", source, GetSystemName(system)));
2285
4a33bdd9 2286 // check if test mode should simulate a FXS error
2287 if (fTestMode & kErrorFXSSources)
2288 {
2289 Log(detector, Form("GetFileIDs - In TESTMODE - Simulating error while connecting to %s FXS", GetSystemName(system)));
2290 return 0;
2291 }
2292
2293 // check connection, in case connect
2294 if (!Connect(system))
2295 {
2296 Log(detector, Form("GetFileIDs - Couldn't connect to %s FXS database", GetSystemName(system)));
2297 return NULL;
2298 }
2299
2300 TString sourceName = 0;
2301 if (system == kDAQ)
2302 {
2303 sourceName = "DAQsource";
2304 } else if (system == kHLT)
2305 {
2306 sourceName = "DDLnumbers";
2307 }
2308
2309 TString sqlQueryStart = Form("select fileId from %s where", fConfig->GetFXSdbTable(system));
2310 TString whereClause = Form("run=%d and detector=\"%s\"",
2311 GetCurrentRun(), detector);
2312 if (sourceName.Length() > 0 && source)
2313 whereClause += Form(" and %s=\"%s\"", sourceName.Data(), source);
2314 TString sqlQuery = Form("%s %s", sqlQueryStart.Data(), whereClause.Data());
2315
2316 AliDebug(2, Form("SQL query: \n%s",sqlQuery.Data()));
2317
2318 // Query execution
2319 TSQLResult* aResult;
2320 aResult = fServer[system]->Query(sqlQuery);
2321 if (!aResult) {
2322 Log(detector, Form("GetFileIDs - Can't execute SQL query to %s database for source: %s",
2323 GetSystemName(system), source));
2324 return 0;
2325 }
2326
2327 TList *list = new TList();
2328 list->SetOwner(1);
2329
2330 if (aResult->GetRowCount() == 0)
2331 {
2332 Log(detector,
2333 Form("GetFileIDs - No entry in %s FXS table for source: %s", GetSystemName(system), source));
2334 delete aResult;
2335 return list;
2336 }
2337
1bcd28db 2338 Log(detector, Form("GetFileIDs - Found %d ids", aResult->GetRowCount()));
2339
4a33bdd9 2340 TSQLRow* aRow;
2341
2342 while ((aRow = aResult->Next()))
2343 {
2344
2345 TString id(aRow->GetField(0), aRow->GetFieldLength(0));
2346 AliDebug(2, Form("fileId = %s", id.Data()));
2347 list->Add(new TObjString(id));
2348 delete aRow;
2349 }
2350
2351 delete aResult;
2352
2353 return list;
2354}
2355
2bb7b766 2356//______________________________________________________________________________________________
9d733021 2357Bool_t AliShuttle::Connect(Int_t system)
2bb7b766 2358{
9827400b 2359 // Connect to MySQL Server of the system's FXS MySQL databases
2360 // DAQ Logbook, Shuttle Logbook and DAQ FXS db are on the same host
2361 //
57f50b3c 2362
9d733021 2363 // check connection: if already connected return
2364 if(fServer[system] && fServer[system]->IsConnected()) return kTRUE;
57f50b3c 2365
9d733021 2366 TString dbHost, dbUser, dbPass, dbName;
57f50b3c 2367
9d733021 2368 if (system < 3) // FXS db servers
2369 {
2370 dbHost = Form("mysql://%s:%d", fConfig->GetFXSdbHost(system), fConfig->GetFXSdbPort(system));
2371 dbUser = fConfig->GetFXSdbUser(system);
2372 dbPass = fConfig->GetFXSdbPass(system);
2373 dbName = fConfig->GetFXSdbName(system);
2374 } else { // Run & Shuttle logbook servers
2375 // TODO Will the Shuttle logbook server be the same as the Run logbook server ???
2376 dbHost = Form("mysql://%s:%d", fConfig->GetDAQlbHost(), fConfig->GetDAQlbPort());
2377 dbUser = fConfig->GetDAQlbUser();
2378 dbPass = fConfig->GetDAQlbPass();
2379 dbName = fConfig->GetDAQlbDB();
2380 }
57f50b3c 2381
9d733021 2382 fServer[system] = TSQLServer::Connect(dbHost.Data(), dbUser.Data(), dbPass.Data());
2383 if (!fServer[system] || !fServer[system]->IsConnected()) {
2384 if(system < 3)
2385 {
2386 AliError(Form("Can't establish connection to FXS database for %s",
2387 AliShuttleInterface::GetSystemName(system)));
2388 } else {
2389 AliError("Can't establish connection to Run logbook.");
57f50b3c 2390 }
9d733021 2391 if(fServer[system]) delete fServer[system];
2392 return kFALSE;
2bb7b766 2393 }
57f50b3c 2394
9d733021 2395 // Get tables
2396 TSQLResult* aResult=0;
2397 switch(system){
2398 case kDAQ:
2399 aResult = fServer[kDAQ]->GetTables(dbName.Data());
2400 break;
2401 case kDCS:
2402 aResult = fServer[kDCS]->GetTables(dbName.Data());
2403 break;
2404 case kHLT:
2405 aResult = fServer[kHLT]->GetTables(dbName.Data());
2406 break;
2407 default:
2408 aResult = fServer[3]->GetTables(dbName.Data());
2409 break;
2410 }
2411
2412 delete aResult;
2bb7b766 2413 return kTRUE;
2414}
57f50b3c 2415
9e080f92 2416//______________________________________________________________________________________________
9d733021 2417Bool_t AliShuttle::UpdateTable()
9e080f92 2418{
9827400b 2419 //
2420 // Update FXS table filling time_processed field in all rows corresponding to current run and detector
2421 //
9e080f92 2422
9d733021 2423 Bool_t result = kTRUE;
9e080f92 2424
9d733021 2425 for (UInt_t system=0; system<3; system++)
2426 {
2427 if(!fFXSCalled[system]) continue;
9e080f92 2428
9d733021 2429 // check connection, in case connect
2430 if (!Connect(system))
2431 {
2432 Log(fCurrentDetector, Form("UpdateTable - Couldn't connect to %s FXS database", GetSystemName(system)));
2433 result = kFALSE;
2434 continue;
9e080f92 2435 }
9e080f92 2436
9d733021 2437 TTimeStamp now; // now
2438
2439 // Loop on FXS list entries
2440 TIter iter(&fFXSlist[system]);
2441 TObjString *aFXSentry=0;
2442 while ((aFXSentry = dynamic_cast<TObjString*> (iter.Next())))
2443 {
2444 TString aFXSentrystr = aFXSentry->String();
2445 TObjArray *aFXSarray = aFXSentrystr.Tokenize("#!?!#");
2446 if (!aFXSarray || aFXSarray->GetEntries() != 2 )
2447 {
2448 Log(fCurrentDetector, Form("UpdateTable - error updating %s FXS entry. Check string: <%s>",
2449 GetSystemName(system), aFXSentrystr.Data()));
2450 if(aFXSarray) delete aFXSarray;
2451 result = kFALSE;
2452 continue;
2453 }
2454 const char* fileId = ((TObjString*) aFXSarray->At(0))->GetName();
2455 const char* source = ((TObjString*) aFXSarray->At(1))->GetName();
2456
2457 TString whereClause;
2458 if (system == kDAQ)
2459 {
2460 whereClause = Form("where run=%d and detector=\"%s\" and fileId=\"%s\" and DAQsource=\"%s\";",
2461 GetCurrentRun(), fCurrentDetector.Data(), fileId, source);
2462 }
2463 else if (system == kDCS)
2464 {
2465 whereClause = Form("where run=%d and detector=\"%s\" and fileId=\"%s\";",
2466 GetCurrentRun(), fCurrentDetector.Data(), fileId);
2467 }
2468 else if (system == kHLT)
2469 {
2470 whereClause = Form("where run=%d and detector=\"%s\" and fileId=\"%s\" and DDLnumbers=\"%s\";",
2471 GetCurrentRun(), fCurrentDetector.Data(), fileId, source);
2472 }
2473
2474 delete aFXSarray;
9e080f92 2475
9d733021 2476 TString sqlQuery = Form("update %s set time_processed=%d %s", fConfig->GetFXSdbTable(system),
2477 now.GetSec(), whereClause.Data());
9e080f92 2478
9d733021 2479 AliDebug(2, Form("SQL query: \n%s",sqlQuery.Data()));
9e080f92 2480
9d733021 2481 // Query execution
2482 TSQLResult* aResult;
2483 aResult = dynamic_cast<TSQLResult*> (fServer[system]->Query(sqlQuery));
2484 if (!aResult)
2485 {
2486 Log(fCurrentDetector, Form("UpdateTable - %s db: can't execute SQL query <%s>",
2487 GetSystemName(system), sqlQuery.Data()));
2488 result = kFALSE;
2489 continue;
2490 }
2491 delete aResult;
9e080f92 2492 }
9e080f92 2493 }
2494
9d733021 2495 return result;
9e080f92 2496}
57f50b3c 2497
3301427a 2498//______________________________________________________________________________________________
2499Bool_t AliShuttle::UpdateTableFailCase()
2500{
9827400b 2501 // Update FXS table filling time_processed field in all rows corresponding to current run and detector
2502 // this is called in case the preprocessor is declared failed for the current run, because
2503 // the fields are updated only in case of success
3301427a 2504
2505 Bool_t result = kTRUE;
2506
2507 for (UInt_t system=0; system<3; system++)
2508 {
2509 // check connection, in case connect
2510 if (!Connect(system))
2511 {
2512 Log(fCurrentDetector, Form("UpdateTableFailCase - Couldn't connect to %s FXS database",
2513 GetSystemName(system)));
2514 result = kFALSE;
2515 continue;
2516 }
2517
2518 TTimeStamp now; // now
2519
2520 // Loop on FXS list entries
2521
2522 TString whereClause = Form("where run=%d and detector=\"%s\";",
2523 GetCurrentRun(), fCurrentDetector.Data());
2524
2525
2526 TString sqlQuery = Form("update %s set time_processed=%d %s", fConfig->GetFXSdbTable(system),
2527 now.GetSec(), whereClause.Data());
2528
2529 AliDebug(2, Form("SQL query: \n%s",sqlQuery.Data()));
2530
2531 // Query execution
2532 TSQLResult* aResult;
2533 aResult = dynamic_cast<TSQLResult*> (fServer[system]->Query(sqlQuery));
2534 if (!aResult)
2535 {
2536 Log(fCurrentDetector, Form("UpdateTableFailCase - %s db: can't execute SQL query <%s>",
2537 GetSystemName(system), sqlQuery.Data()));
2538 result = kFALSE;
2539 continue;
2540 }
2541 delete aResult;
2542 }
2543
2544 return result;
2545}
2546
2bb7b766 2547//______________________________________________________________________________________________
2548Bool_t AliShuttle::UpdateShuttleLogbook(const char* detector, const char* status)
2549{
e7f62f16 2550 //
2551 // Update Shuttle logbook filling detector or shuttle_done column
2552 // ex. of usage: UpdateShuttleLogbook("PHOS", "DONE") or UpdateShuttleLogbook("shuttle_done")
2553 //
57f50b3c 2554
2bb7b766 2555 // check connection, in case connect
be48e3ea 2556 if(!Connect(3)){
2bb7b766 2557 Log("SHUTTLE", "UpdateShuttleLogbook - Couldn't connect to DAQ Logbook.");
2558 return kFALSE;
57f50b3c 2559 }
2560
2bb7b766 2561 TString detName(detector);
2562 TString setClause;
e7f62f16 2563 if(detName == "shuttle_done")
2564 {
2bb7b766 2565 setClause = "set shuttle_done=1";
e7f62f16 2566
2567 // Send the information to ML
2568 TMonaLisaText mlStatus("SHUTTLE_status", "Done");
2569
2570 TList mlList;
2571 mlList.Add(&mlStatus);
2572
2573 fMonaLisa->SendParameters(&mlList);
2bb7b766 2574 } else {
2bb7b766 2575 TString statusStr(status);
2576 if(statusStr.Contains("done", TString::kIgnoreCase) ||
2577 statusStr.Contains("failed", TString::kIgnoreCase)){
eba76848 2578 setClause = Form("set %s=\"%s\"", detector, status);
2bb7b766 2579 } else {
2580 Log("SHUTTLE",
2581 Form("UpdateShuttleLogbook - Invalid status <%s> for detector %s",
2582 status, detector));
2583 return kFALSE;
2584 }
2585 }
57f50b3c 2586
2bb7b766 2587 TString whereClause = Form("where run=%d", GetCurrentRun());
2588
441b0e9c 2589 TString sqlQuery = Form("update %s %s %s",
2590 fConfig->GetShuttlelbTable(), setClause.Data(), whereClause.Data());
57f50b3c 2591
2bb7b766 2592 AliDebug(2, Form("SQL query: \n%s",sqlQuery.Data()));
2593
2594 // Query execution
2595 TSQLResult* aResult;
be48e3ea 2596 aResult = dynamic_cast<TSQLResult*> (fServer[3]->Query(sqlQuery));
2bb7b766 2597 if (!aResult) {
2598 Log("SHUTTLE", Form("UpdateShuttleLogbook - Can't execute query <%s>", sqlQuery.Data()));
2599 return kFALSE;
57f50b3c 2600 }
2bb7b766 2601 delete aResult;
57f50b3c 2602
2603 return kTRUE;
2604}
2605
2606//______________________________________________________________________________________________
2bb7b766 2607Int_t AliShuttle::GetCurrentRun() const
2608{
9827400b 2609 //
2610 // Get current run from logbook entry
2611 //
57f50b3c 2612
2bb7b766 2613 return fLogbookEntry ? fLogbookEntry->GetRun() : -1;
57f50b3c 2614}
2615
2616//______________________________________________________________________________________________
2bb7b766 2617UInt_t AliShuttle::GetCurrentStartTime() const
2618{
9827400b 2619 //
2620 // get current start time
2621 //
57f50b3c 2622
2bb7b766 2623 return fLogbookEntry ? fLogbookEntry->GetStartTime() : 0;
57f50b3c 2624}
2625
2626//______________________________________________________________________________________________
2bb7b766 2627UInt_t AliShuttle::GetCurrentEndTime() const
2628{
9827400b 2629 //
2630 // get current end time from logbook entry
2631 //
57f50b3c 2632
2bb7b766 2633 return fLogbookEntry ? fLogbookEntry->GetEndTime() : 0;
57f50b3c 2634}
2635
b948db8d 2636//______________________________________________________________________________________________
2637void AliShuttle::Log(const char* detector, const char* message)
2638{
9827400b 2639 //
2640 // Fill log string with a message
2641 //
b948db8d 2642
36c99a6a 2643 void* dir = gSystem->OpenDirectory(GetShuttleLogDir());
84090f85 2644 if (dir == NULL) {
36c99a6a 2645 if (gSystem->mkdir(GetShuttleLogDir(), kTRUE)) {
2646 AliError(Form("Can't open directory <%s>", GetShuttleLogDir()));
84090f85 2647 return;
2648 }
b948db8d 2649
84090f85 2650 } else {
2651 gSystem->FreeDirectory(dir);
2652 }
b948db8d 2653
cb343cfd 2654 TString toLog = Form("%s (%d): %s - ", TTimeStamp(time(0)).AsString("s"), getpid(), detector);
e7f62f16 2655 if (GetCurrentRun() >= 0)
2656 toLog += Form("run %d - ", GetCurrentRun());
2bb7b766 2657 toLog += Form("%s", message);
2658
84090f85 2659 AliInfo(toLog.Data());
ffa29e93 2660
2661 // if we redirect the log output already to the file, leave here
2662 if (fOutputRedirected && strcmp(detector, "SHUTTLE") != 0)
2663 return;
b948db8d 2664
ffa29e93 2665 TString fileName = GetLogFileName(detector);
e7f62f16 2666
84090f85 2667 gSystem->ExpandPathName(fileName);
2668
2669 ofstream logFile;
2670 logFile.open(fileName, ofstream::out | ofstream::app);
2671
2672 if (!logFile.is_open()) {
2673 AliError(Form("Could not open file %s", fileName.Data()));
2674 return;
2675 }
7bfb2090 2676
84090f85 2677 logFile << toLog.Data() << "\n";
b948db8d 2678
84090f85 2679 logFile.close();
b948db8d 2680}
2bb7b766 2681
ffa29e93 2682//______________________________________________________________________________________________
2683TString AliShuttle::GetLogFileName(const char* detector) const
2684{
2685 //
2686 // returns the name of the log file for a given sub detector
2687 //
2688
2689 TString fileName;
2690
2691 if (GetCurrentRun() >= 0)
2692 fileName.Form("%s/%s_%d.log", GetShuttleLogDir(), detector, GetCurrentRun());
2693 else
2694 fileName.Form("%s/%s.log", GetShuttleLogDir(), detector);
2695
2696 return fileName;
2697}
2698
2bb7b766 2699//______________________________________________________________________________________________
2700Bool_t AliShuttle::Collect(Int_t run)
2701{
9827400b 2702 //
2703 // Collects conditions data for all UNPROCESSED run written to DAQ LogBook in case of run = -1 (default)
2704 // If a dedicated run is given this run is processed
2705 //
2706 // In operational mode, this is the Shuttle function triggered by the EOR signal.
2707 //
2bb7b766 2708
eba76848 2709 if (run == -1)
2710 Log("SHUTTLE","Collect - Shuttle called. Collecting conditions data for unprocessed runs");
2711 else
2712 Log("SHUTTLE", Form("Collect - Shuttle called. Collecting conditions data for run %d", run));
cb343cfd 2713
2714 SetLastAction("Starting");
2bb7b766 2715
2716 TString whereClause("where shuttle_done=0");
eba76848 2717 if (run != -1)
2718 whereClause += Form(" and run=%d", run);
2bb7b766 2719
2720 TObjArray shuttleLogbookEntries;
be48e3ea 2721 if (!QueryShuttleLogbook(whereClause, shuttleLogbookEntries))
2722 {
cb343cfd 2723 Log("SHUTTLE", "Collect - Can't retrieve entries from Shuttle logbook");
2bb7b766 2724 return kFALSE;
2725 }
2726
9e080f92 2727 if (shuttleLogbookEntries.GetEntries() == 0)
2728 {
2729 if (run == -1)
2730 Log("SHUTTLE","Collect - Found no UNPROCESSED runs in Shuttle logbook");
2731 else
2732 Log("SHUTTLE", Form("Collect - Run %d is already DONE "
2733 "or it does not exist in Shuttle logbook", run));
2734 return kTRUE;
2735 }
2736
be48e3ea 2737 for (UInt_t iDet=0; iDet<NDetectors(); iDet++)
2738 fFirstUnprocessed[iDet] = kTRUE;
2739
fc5a4708 2740 if (run != -1)
be48e3ea 2741 {
2742 // query Shuttle logbook for earlier runs, check if some detectors are unprocessed,
2743 // flag them into fFirstUnprocessed array
2744 TString whereClause(Form("where shuttle_done=0 and run < %d", run));
2745 TObjArray tmpLogbookEntries;
2746 if (!QueryShuttleLogbook(whereClause, tmpLogbookEntries))
2747 {
2748 Log("SHUTTLE", "Collect - Can't retrieve entries from Shuttle logbook");
2749 return kFALSE;
2750 }
2751
2752 TIter iter(&tmpLogbookEntries);
2753 AliShuttleLogbookEntry* anEntry = 0;
2754 while ((anEntry = dynamic_cast<AliShuttleLogbookEntry*> (iter.Next())))
2755 {
2756 for (UInt_t iDet=0; iDet<NDetectors(); iDet++)
2757 {
2758 if (anEntry->GetDetectorStatus(iDet) == AliShuttleLogbookEntry::kUnprocessed)
2759 {
2760 AliDebug(2, Form("Run %d: setting %s as \"not first time unprocessed\"",
2761 anEntry->GetRun(), GetDetName(iDet)));
2762 fFirstUnprocessed[iDet] = kFALSE;
2763 }
2764 }
2765
2766 }
2767
2768 }
2769
2770 if (!RetrieveConditionsData(shuttleLogbookEntries))
2771 {
cb343cfd 2772 Log("SHUTTLE", "Collect - Process of at least one run failed");
2bb7b766 2773 return kFALSE;
2774 }
2775
36c99a6a 2776 Log("SHUTTLE", "Collect - Requested run(s) successfully processed");
eba76848 2777 return kTRUE;
2bb7b766 2778}
2779
2bb7b766 2780//______________________________________________________________________________________________
2781Bool_t AliShuttle::RetrieveConditionsData(const TObjArray& dateEntries)
2782{
9827400b 2783 //
2784 // Retrieve conditions data for all runs that aren't processed yet
2785 //
2bb7b766 2786
2787 Bool_t hasError = kFALSE;
2788
2789 TIter iter(&dateEntries);
2790 AliShuttleLogbookEntry* anEntry;
2791
2792 while ((anEntry = (AliShuttleLogbookEntry*) iter.Next())){
2793 if (!Process(anEntry)){
2794 hasError = kTRUE;
2795 }
4b95672b 2796
2797 // clean SHUTTLE temp directory
3301427a 2798 TString filename = Form("%s/*.shuttle", GetShuttleTempDir());
2799 RemoveFile(filename.Data());
2bb7b766 2800 }
2801
2802 return hasError == kFALSE;
2803}
cb343cfd 2804
2805//______________________________________________________________________________________________
2806ULong_t AliShuttle::GetTimeOfLastAction() const
2807{
9827400b 2808 //
2809 // Gets time of last action
2810 //
2811
cb343cfd 2812 ULong_t tmp;
36c99a6a 2813
cb343cfd 2814 fMonitoringMutex->Lock();
be48e3ea 2815
cb343cfd 2816 tmp = fLastActionTime;
36c99a6a 2817
cb343cfd 2818 fMonitoringMutex->UnLock();
36c99a6a 2819
cb343cfd 2820 return tmp;
2821}
2822
2823//______________________________________________________________________________________________
2824const TString AliShuttle::GetLastAction() const
2825{
9827400b 2826 //
cb343cfd 2827 // returns a string description of the last action
9827400b 2828 //
cb343cfd 2829
2830 TString tmp;
36c99a6a 2831
cb343cfd 2832 fMonitoringMutex->Lock();
2833
2834 tmp = fLastAction;
2835
2836 fMonitoringMutex->UnLock();
2837
36c99a6a 2838 return tmp;
cb343cfd 2839}
2840
2841//______________________________________________________________________________________________
2842void AliShuttle::SetLastAction(const char* action)
2843{
9827400b 2844 //
cb343cfd 2845 // updates the monitoring variables
9827400b 2846 //
36c99a6a 2847
cb343cfd 2848 fMonitoringMutex->Lock();
36c99a6a 2849
cb343cfd 2850 fLastAction = action;
2851 fLastActionTime = time(0);
2852
2853 fMonitoringMutex->UnLock();
2854}
eba76848 2855
2856//______________________________________________________________________________________________
2857const char* AliShuttle::GetRunParameter(const char* param)
2858{
9827400b 2859 //
2860 // returns run parameter read from DAQ logbook
2861 //
eba76848 2862
2863 if(!fLogbookEntry) {
2864 AliError("No logbook entry!");
2865 return 0;
2866 }
2867
2868 return fLogbookEntry->GetRunParameter(param);
2869}
57c1a579 2870
d386d623 2871//______________________________________________________________________________________________
9827400b 2872AliCDBEntry* AliShuttle::GetFromOCDB(const char* detector, const AliCDBPath& path)
d386d623 2873{
9827400b 2874 //
2875 // returns object from OCDB valid for current run
2876 //
d386d623 2877
9827400b 2878 if (fTestMode & kErrorOCDB)
2879 {
2880 Log(detector, "GetFromOCDB - In TESTMODE - Simulating error with OCDB");
2881 return 0;
2882 }
2883
d386d623 2884 AliCDBStorage *sto = AliCDBManager::Instance()->GetStorage(fgkMainCDB);
2885 if (!sto)
2886 {
9827400b 2887 Log(detector, "GetFromOCDB - Cannot activate main OCDB for query!");
d386d623 2888 return 0;
2889 }
2890
2891 return dynamic_cast<AliCDBEntry*> (sto->Get(path, GetCurrentRun()));
2892}
2893
57c1a579 2894//______________________________________________________________________________________________
2895Bool_t AliShuttle::SendMail()
2896{
9827400b 2897 //
2898 // sends a mail to the subdetector expert in case of preprocessor error
2899 //
2900
2901 if (fTestMode != kNone)
2902 return kTRUE;
57c1a579 2903
36c99a6a 2904 void* dir = gSystem->OpenDirectory(GetShuttleLogDir());
57c1a579 2905 if (dir == NULL)
2906 {
36c99a6a 2907 if (gSystem->mkdir(GetShuttleLogDir(), kTRUE))
57c1a579 2908 {
36c99a6a 2909 AliError(Form("Can't open directory <%s>", GetShuttleLogDir()));
57c1a579 2910 return kFALSE;
2911 }
2912
2913 } else {
2914 gSystem->FreeDirectory(dir);
2915 }
2916
2917 TString bodyFileName;
36c99a6a 2918 bodyFileName.Form("%s/mail.body", GetShuttleLogDir());
57c1a579 2919 gSystem->ExpandPathName(bodyFileName);
2920
2921 ofstream mailBody;
2922 mailBody.open(bodyFileName, ofstream::out);
2923
2924 if (!mailBody.is_open())
2925 {
2926 AliError(Form("Could not open mail body file %s", bodyFileName.Data()));
2927 return kFALSE;
2928 }
2929
2930 TString to="";
2931 TIter iterExperts(fConfig->GetResponsibles(fCurrentDetector));
2932 TObjString *anExpert=0;
2933 while ((anExpert = (TObjString*) iterExperts.Next()))
2934 {
2935 to += Form("%s,", anExpert->GetName());
2936 }
2937 to.Remove(to.Length()-1);
909732f7 2938 AliDebug(2, Form("to: %s",to.Data()));
57c1a579 2939
86aa42c3 2940 if (to.IsNull()) {
c88ad5db 2941 Log("SHUTTLE", "List of detector responsibles not yet set!");
36c99a6a 2942 return kFALSE;
2943 }
2944
57c1a579 2945 TString cc="alberto.colla@cern.ch";
2946
546242fb 2947 TString subject = Form("%s Shuttle preprocessor FAILED in run %d !",
57c1a579 2948 fCurrentDetector.Data(), GetCurrentRun());
909732f7 2949 AliDebug(2, Form("subject: %s", subject.Data()));
57c1a579 2950
2951 TString body = Form("Dear %s expert(s), \n\n", fCurrentDetector.Data());
2952 body += Form("SHUTTLE just detected that your preprocessor "
546242fb 2953 "failed processing run %d!!\n\n", GetCurrentRun());
2954 body += Form("Please check %s status on the SHUTTLE monitoring page: \n\n", fCurrentDetector.Data());
2955 body += Form("\thttp://pcalimonitor.cern.ch:8889/shuttle.jsp?time=168 \n\n");
2956 body += Form("Find the %s log for the current run on \n\n"
2957 "\thttp://pcalishuttle01.cern.ch:8880/logs/%s_%d.log \n\n",
2958 fCurrentDetector.Data(), fCurrentDetector.Data(), GetCurrentRun());
57c1a579 2959 body += Form("The last 10 lines of %s log file are following:\n\n");
2960
909732f7 2961 AliDebug(2, Form("Body begin: %s", body.Data()));
57c1a579 2962
2963 mailBody << body.Data();
2964 mailBody.close();
2965 mailBody.open(bodyFileName, ofstream::out | ofstream::app);
2966
9d733021 2967 TString logFileName = Form("%s/%s_%d.log", GetShuttleLogDir(), fCurrentDetector.Data(), GetCurrentRun());
57c1a579 2968 TString tailCommand = Form("tail -n 10 %s >> %s", logFileName.Data(), bodyFileName.Data());
2969 if (gSystem->Exec(tailCommand.Data()))
2970 {
2971 mailBody << Form("%s log file not found ...\n\n", fCurrentDetector.Data());
2972 }
2973
2974 TString endBody = Form("------------------------------------------------------\n\n");
36c99a6a 2975 endBody += Form("In case of problems please contact the SHUTTLE core team.\n\n");
2976 endBody += "Please do not answer this message directly, it is automatically generated.\n\n";
546242fb 2977 endBody += "Greetings,\n\n \t\t\tthe SHUTTLE\n";
57c1a579 2978
909732f7 2979 AliDebug(2, Form("Body end: %s", endBody.Data()));
57c1a579 2980
2981 mailBody << endBody.Data();
2982
2983 mailBody.close();
2984
2985 // send mail!
2986 TString mailCommand = Form("mail -s \"%s\" -c %s %s < %s",
2987 subject.Data(),
2988 cc.Data(),
2989 to.Data(),
2990 bodyFileName.Data());
909732f7 2991 AliDebug(2, Form("mail command: %s", mailCommand.Data()));
57c1a579 2992
2993 Bool_t result = gSystem->Exec(mailCommand.Data());
2994
2995 return result == 0;
2996}
d386d623 2997
441b0e9c 2998//______________________________________________________________________________________________
9827400b 2999const char* AliShuttle::GetRunType()
441b0e9c 3000{
9827400b 3001 //
3002 // returns run type read from "run type" logbook
3003 //
441b0e9c 3004
3005 if(!fLogbookEntry) {
3006 AliError("No logbook entry!");
3007 return 0;
3008 }
3009
9827400b 3010 return fLogbookEntry->GetRunType();
441b0e9c 3011}
3012
4859271b 3013//______________________________________________________________________________________________
3014Bool_t AliShuttle::GetHLTStatus()
3015{
3016 // Return HLT status (ON=1 OFF=0)
3017 // Converts the HLT status from the status string read in the run logbook (not just a bool)
3018
3019 if(!fLogbookEntry) {
3020 AliError("No logbook entry!");
3021 return 0;
3022 }
3023
3024 // TODO implement when HLTStatus is inserted in run logbook
3025 //TString hltStatus = fLogbookEntry->GetRunParameter("HLTStatus");
3026 //if(hltStatus == "OFF") {return kFALSE};
3027
3028 return kTRUE;
3029}
3030
d386d623 3031//______________________________________________________________________________________________
3032void AliShuttle::SetShuttleTempDir(const char* tmpDir)
3033{
9827400b 3034 //
3035 // sets Shuttle temp directory
3036 //
d386d623 3037
3038 fgkShuttleTempDir = gSystem->ExpandPathName(tmpDir);
3039}
3040
3041//______________________________________________________________________________________________
3042void AliShuttle::SetShuttleLogDir(const char* logDir)
3043{
9827400b 3044 //
3045 // sets Shuttle log directory
3046 //
d386d623 3047
3048 fgkShuttleLogDir = gSystem->ExpandPathName(logDir);
3049}