]> git.uio.no Git - u/mrichter/AliRoot.git/blame - SHUTTLE/AliShuttle.cxx
git://git.uio.no / u / mrichter / AliRoot.git / blame
? search:
summary | shortlog | log | commit | commitdiff | tree
blob | blame (incremental) | history | HEAD
Bug fix (Jan Fiete): recovering from StoreError, if the store to OCDB is successful,
[u/mrichter/AliRoot.git] / SHUTTLE / AliShuttle.cxx
CommitLineData
73abe331 1/**************************************************************************
2 * Copyright(c) 1998-1999, ALICE Experiment at CERN, All rights reserved. *
3 * *
4 * Author: The ALICE Off-line Project. *
5 * Contributors are mentioned in the code where appropriate. *
6 * *
7 * Permission to use, copy, modify and distribute this software and its *
8 * documentation strictly for non-commercial purposes is hereby granted *
9 * without fee, provided that the above copyright notice appears in all *
10 * copies and that both the copyright notice and this permission notice *
11 * appear in the supporting documentation. The authors make no claims *
12 * about the suitability of this software for any purpose. It is *
13 * provided "as is" without express or implied warranty. *
14 **************************************************************************/
15
16/*
17$Log$
a0aaf247 18Revision 1.79 2007/12/19 14:03:01 acolla
19
20detector name to build the lhcPeriod_DET is to be looked in "detector" column, not "partition"
21
2694f253 22Revision 1.78 2007/12/19 11:50:41 acolla
23
24Raw data tag merged files is written in /alice/data/.../lhcPeriod_DET/runNb/raw if partition is made of DET only
25
a1c5723b 26Revision 1.77 2007/12/19 11:16:16 acolla
27More meaningful log message added in GetFileSources
28
8884d71e 29Revision 1.76 2007/12/19 07:45:20 acolla
30bug fix in the name of the raw tag files (Raw instead of raw)
31
955d2abc 32Revision 1.75 2007/12/18 15:42:14 jgrosseo
33adding number of open runs to monitoring
34
fbc112e3 35Revision 1.74 2007/12/17 03:23:32 jgrosseo
36several bugfixes
37added "empty preprocessor" as placeholder for Acorde in FDR
38
4508ab8b 39Revision 1.73 2007/12/14 19:31:36 acolla
40Sending email to DCS experts is temporarily commented
41
b5f1c82e 42Revision 1.72 2007/12/13 15:44:28 acolla
43Run type added in mail sent to detector expert (eases understanding)
44
6a1146c4 45Revision 1.71 2007/12/12 14:56:14 jgrosseo
46sending shuttle_ignore to ML also in case of 0 events
47
97e3c167 48Revision 1.70 2007/12/12 13:45:35 acolla
49Monalisa started in Collect() function. Alive message to monitor is sent at each Collect and every minute during preprocessor processing.
50
ee6f7523 51Revision 1.69 2007/12/12 10:06:29 acolla
52in AliShuttle.cxx: SHUTTLE logbook is updated in case of invalid run times:
53
54time_start==0 && time_end==0
55
56logbook is NOT updated if time_start != 0 && time_end == 0, because it may mean that the run is still ongoing.
57
1abfbb60 58Revision 1.68 2007/12/11 10:15:17 acolla
59Added marking SHUTTLE=DONE for invalid runs
60(invalid start time or end time) and runs with totalEvents < 1
61
b0e53b15 62Revision 1.67 2007/12/07 19:14:36 acolla
63in AliShuttleTrigger:
64
65Added automatic collection of new runs on a regular time basis (settable from the configuration)
66
67in AliShuttleConfig: new members
68
69- triggerWait: time to wait for DIM trigger (s) before starting automatic collection of new runs
70- mode: run mode (test, prod) -> used to build log folder (logs or logs_PROD)
71
72in AliShuttle:
73
74- logs now stored in logs/#RUN/DET_#RUN.log
75
7d4cf768 76Revision 1.66 2007/12/05 10:45:19 jgrosseo
77changed order of arguments to TMonaLisaWriter
78
468df1ce 79Revision 1.65 2007/11/26 16:58:37 acolla
80Monalisa configuration added: host and table name
81
b832ec02 82Revision 1.64 2007/11/13 16:15:47 acolla
83DCS map is stored in a file in the temp folder where the detector is processed.
84If the preprocessor fails, the temp folder is not removed. This will help the debugging of the problem.
85
b1d18693 86Revision 1.63 2007/11/02 10:53:16 acolla
87Protection added to AliShuttle::CopyFileLocally
88
7d43a416 89Revision 1.62 2007/10/31 18:23:13 acolla
90Furter developement on the Shuttle:
91
92- Shuttle now connects to the Grid as alidaq. The OCDB and Reference folders
93are now built from /alice/data, e.g.:
94/alice/data/2007/LHC07a/OCDB
95
96the year and LHC period are taken from the Shuttle.
97Raw metadata files are stored by GRP to:
98/alice/data/2007/LHC07a/<runNb>/Raw/RunMetadata.root
99
100- Shuttle sends a mail to DCS experts each time DP retrieval fails.
101
675f64cd 102Revision 1.61 2007/10/30 20:33:51 acolla
103Improved managing of temporary folders, which weren't correctly handled.
104Resolved bug introduced in StoreReferenceFile, which caused SPD preprocessor fail.
105
d524ade6 106Revision 1.60 2007/10/29 18:06:16 acolla
107
108New function StoreRunMetadataFile added to preprocessor and Shuttle interface
109This function can be used by GRP only. It stores raw data tags merged file to the
110raw data folder (e.g. /alice/data/2008/LHC08a/000099999/Raw).
111
112KNOWN ISSUES:
113
1141. Shuttle cannot write to /alice/data/ because it belongs to alidaq. Tag file is stored in /alice/simulation/... for the time being.
1152. Due to a bug in TAlien::Mkdir, the creation of a folder in recursive mode (-p option) does not work. The problem
116has been corrected in the root package on the Shuttle machine.
117
c88ad5db 118Revision 1.59 2007/10/05 12:40:55 acolla
119
120Result error code added to AliDCSClient data members (it was "lost" with the new implementation of TMap* GetAliasValues and GetDPValues).
121
1790d4b7 122Revision 1.58 2007/09/28 15:27:40 acolla
123
124AliDCSClient "multiSplit" option added in the DCS configuration
125in AliDCSMessage: variable MAX_BODY_SIZE set to 500000
126
542b6cc8 127Revision 1.57 2007/09/27 16:53:13 acolla
128Detectors can have more than one AMANDA server. SHUTTLE queries the servers sequentially,
129merges the dcs aliases/DPs in one TMap and sends it to the preprocessor.
130
1d172743 131Revision 1.56 2007/09/14 16:46:14 jgrosseo
1321) Connect and Close are called before and after each query, so one can
133keep the same AliDCSClient object.
1342) The splitting of a query is moved to GetDPValues/GetAliasValues.
1353) Splitting interval can be specified in constructor
136
b41b252a 137Revision 1.55 2007/08/06 12:26:40 acolla
138Function Bool_t GetHLTStatus added to preprocessor. It returns the status of HLT
139read from the run logbook.
140
4859271b 141Revision 1.54 2007/07/12 09:51:25 jgrosseo
142removed duplicated log message in GetFile
143
4f0749a8 144Revision 1.53 2007/07/12 09:26:28 jgrosseo
145updating hlt fxs base path
146
42fde080 147Revision 1.52 2007/07/12 08:06:45 jgrosseo
148adding log messages in getfile... functions
149adding not implemented copy constructor in alishuttleconfigholder
150
1bcd28db 151Revision 1.51 2007/07/03 17:24:52 acolla
152root moved to v5-16-00. TFileMerger->Cp moved to TFile::Cp.
153
a986b218 154Revision 1.50 2007/07/02 17:19:32 acolla
155preprocessor is run in a temp directory that is removed when process is finished.
156
5bac2bde 157Revision 1.49 2007/06/29 10:45:06 acolla
158Number of columns in MySql Shuttle logbook increased by one (HLT added)
159
db99d43e 160Revision 1.48 2007/06/21 13:06:19 acolla
161GetFileSources returns dummy list with 1 source if system=DCS (better than
162returning error as it was)
163
6297b37d 164Revision 1.47 2007/06/19 17:28:56 acolla
165HLT updated; missing map bug removed.
166
dc25836b 167Revision 1.46 2007/06/09 13:01:09 jgrosseo
168Switching to retrieval of several DCS DPs at a time (multiDPrequest)
169
a038aa70 170Revision 1.45 2007/05/30 06:35:20 jgrosseo
171Adding functionality to the Shuttle/TestShuttle:
172o) Function to retrieve list of sources from a given system (GetFileSources with id=0)
173o) Function to retrieve list of IDs for a given source (GetFileIDs)
174These functions are needed for dealing with the tag files that are saved for the GRP preprocessor
175Example code has been added to the TestProcessor in TestShuttle
176
4a33bdd9 177Revision 1.44 2007/05/11 16:09:32 acolla
178Reference files for ITS, MUON and PHOS are now stored in OfflineDetName/OnlineDetName/run_...
179example: ITS/SPD/100_filename.root
180
2d9019b4 181Revision 1.43 2007/05/10 09:59:51 acolla
182Various bug fixes in StoreRefFilesToGrid; Cleaning of reference storage before processing detector (CleanReferenceStorage)
183
546242fb 184Revision 1.42 2007/05/03 08:01:39 jgrosseo
185typo in last commit :-(
186
8b739301 187Revision 1.41 2007/05/03 08:00:48 jgrosseo
188fixing log message when pp want to skip dcs value retrieval
189
651fdaab 190Revision 1.40 2007/04/27 07:06:48 jgrosseo
191GetFileSources returns empty list in case of no files, but successful query
192No mails sent in testmode
193
86aa42c3 194Revision 1.39 2007/04/17 12:43:57 acolla
195Correction in StoreOCDB; change of text in mail to detector expert
196
26758fce 197Revision 1.38 2007/04/12 08:26:18 jgrosseo
198updated comment
199
3c2a21c8 200Revision 1.37 2007/04/10 16:53:14 jgrosseo
201redirecting sub detector stdout, stderr to sub detector log file
202
3d8bc902 203Revision 1.35 2007/04/04 16:26:38 acolla
2041. Re-organization of function calls in TestPreprocessor to make it more meaningful.
2052. Added missing dependency in test preprocessors.
2063. in AliShuttle.cxx: processing time and memory consumption info on a single line.
207
886d60e6 208Revision 1.34 2007/04/04 10:33:36 jgrosseo
2091) Storing of files to the Grid is now done _after_ your preprocessors succeeded. This is transparent, which means that you can still use the same functions (Store, StoreReferenceData) to store files to the Grid. However, the Shuttle first stores them locally and transfers them after the preprocessor finished. The return code of these two functions has changed from UInt_t to Bool_t which gives you the success of the storing.
210In case of an error with the Grid, the Shuttle will retry the storing later, the preprocessor does not need to be run again.
211
2122) The meaning of the return code of the preprocessor has changed. 0 is now success and any other value means failure. This value is stored in the log and you can use it to keep details about the error condition.
213
2143) New function StoreReferenceFile to _directly_ store a file (without opening it) to the reference storage.
215
2164) The memory usage of the preprocessor is monitored. If it exceeds 2 GB it is terminated.
217
2185) New function AliPreprocessor::ProcessDCS(). If you do not need to have DCS data in all cases, you can skip the processing by implemting this function and returning kFALSE under certain conditions. E.g. if there is a certain run type.
219If you always need DCS data (like before), you do not need to implement it.
220
2216) The run type has been added to the monitoring page
222
9827400b 223Revision 1.33 2007/04/03 13:56:01 acolla
224Grid Storage at the end of preprocessing. Added virtual method to disable DCS query according to the
225run type.
226
3301427a 227Revision 1.32 2007/02/28 10:41:56 acolla
228Run type field added in SHUTTLE framework. Run type is read from "run type" logbook and retrieved by
229AliPreprocessor::GetRunType() function.
230Added some ldap definition files.
231
d386d623 232Revision 1.30 2007/02/13 11:23:21 acolla
233Moved getters and setters of Shuttle's main OCDB/Reference, local
234OCDB/Reference, temp and log folders to AliShuttleInterface
235
9d733021 236Revision 1.27 2007/01/30 17:52:42 jgrosseo
237adding monalisa monitoring
238
e7f62f16 239Revision 1.26 2007/01/23 19:20:03 acolla
240Removed old ldif files, added TOF, MCH ldif files. Added some options in
241AliShuttleConfig::Print. Added in Ali Shuttle: SetShuttleTempDir and
242SetShuttleLogDir
243
36c99a6a 244Revision 1.25 2007/01/15 19:13:52 acolla
245Moved some AliInfo to AliDebug in SendMail function
246
fc5a4708 247Revision 1.21 2006/12/07 08:51:26 jgrosseo
248update (alberto):
249table, db names in ldap configuration
250added GRP preprocessor
251DCS data can also be retrieved by data point
252
2c15234c 253Revision 1.20 2006/11/16 16:16:48 jgrosseo
254introducing strict run ordering flag
255removed giving preprocessor name to preprocessor, they have to know their name themselves ;-)
256
be48e3ea 257Revision 1.19 2006/11/06 14:23:04 jgrosseo
258major update (Alberto)
259o) reading of run parameters from the logbook
260o) online offline naming conversion
261o) standalone DCSclient package
262
eba76848 263Revision 1.18 2006/10/20 15:22:59 jgrosseo
264o) Adding time out to the execution of the preprocessors: The Shuttle forks and the parent process monitors the child
265o) Merging Collect, CollectAll, CollectNew function
266o) Removing implementation of empty copy constructors (declaration still there!)
267
cb343cfd 268Revision 1.17 2006/10/05 16:20:55 jgrosseo
269adapting to new CDB classes
270
6ec0e06c 271Revision 1.16 2006/10/05 15:46:26 jgrosseo
272applying to the new interface
273
481441a2 274Revision 1.15 2006/10/02 16:38:39 jgrosseo
275update (alberto):
276fixed memory leaks
277storing of objects that failed to be stored to the grid before
278interfacing of shuttle status table in daq system
279
2bb7b766 280Revision 1.14 2006/08/29 09:16:05 jgrosseo
281small update
282
85a80aa9 283Revision 1.13 2006/08/15 10:50:00 jgrosseo
284effc++ corrections (alberto)
285
4f0ab988 286Revision 1.12 2006/08/08 14:19:29 jgrosseo
287Update to shuttle classes (Alberto)
288
289- Possibility to set the full object's path in the Preprocessor's and
290Shuttle's Store functions
291- Possibility to extend the object's run validity in the same classes
292("startValidity" and "validityInfinite" parameters)
293- Implementation of the StoreReferenceData function to store reference
294data in a dedicated CDB storage.
295
84090f85 296Revision 1.11 2006/07/21 07:37:20 jgrosseo
297last run is stored after each run
298
7bfb2090 299Revision 1.10 2006/07/20 09:54:40 jgrosseo
300introducing status management: The processing per subdetector is divided into several steps,
301after each step the status is stored on disk. If the system crashes in any of the steps the Shuttle
302can keep track of the number of failures and skips further processing after a certain threshold is
303exceeded. These thresholds can be configured in LDAP.
304
5164a766 305Revision 1.9 2006/07/19 10:09:55 jgrosseo
306new configuration, accesst to DAQ FES (Alberto)
307
57f50b3c 308Revision 1.8 2006/07/11 12:44:36 jgrosseo
309adding parameters for extended validity range of data produced by preprocessor
310
17111222 311Revision 1.7 2006/07/10 14:37:09 jgrosseo
312small fix + todo comment
313
e090413b 314Revision 1.6 2006/07/10 13:01:41 jgrosseo
315enhanced storing of last sucessfully processed run (alberto)
316
a7160fe9 317Revision 1.5 2006/07/04 14:59:57 jgrosseo
318revision of AliDCSValue: Removed wrapper classes, reduced storage size per value by factor 2
319
45a493ce 320Revision 1.4 2006/06/12 09:11:16 jgrosseo
321coding conventions (Alberto)
322
58bc3020 323Revision 1.3 2006/06/06 14:26:40 jgrosseo
324o) removed files that were moved to STEER
325o) shuttle updated to follow the new interface (Alberto)
326
b948db8d 327Revision 1.2 2006/03/07 07:52:34 hristov
328New version (B.Yordanov)
329
d477ad88 330Revision 1.6 2005/11/19 17:19:14 byordano
331RetrieveDATEEntries and RetrieveConditionsData added
332
333Revision 1.5 2005/11/19 11:09:27 byordano
334AliShuttle declaration added
335
336Revision 1.4 2005/11/17 17:47:34 byordano
337TList changed to TObjArray
338
339Revision 1.3 2005/11/17 14:43:23 byordano
340import to local CVS
341
342Revision 1.1.1.1 2005/10/28 07:33:58 hristov
343Initial import as subdirectory in AliRoot
344
73abe331 345Revision 1.2 2005/09/13 08:41:15 byordano
346default startTime endTime added
347
348Revision 1.4 2005/08/30 09:13:02 byordano
349some docs added
350
351Revision 1.3 2005/08/29 21:15:47 byordano
352some docs added
353
354*/
355
356//
357// This class is the main manager for AliShuttle.
358// It organizes the data retrieval from DCS and call the
b948db8d 359// interface methods of AliPreprocessor.
73abe331 360// For every detector in AliShuttleConfgi (see AliShuttleConfig),
361// data for its set of aliases is retrieved. If there is registered
b948db8d 362// AliPreprocessor for this detector then it will be used
363// accroding to the schema (see AliPreprocessor).
364// If there isn't registered AliPreprocessor than the retrieved
73abe331 365// data is stored automatically to the undelying AliCDBStorage.
366// For detSpec is used the alias name.
367//
368
369#include "AliShuttle.h"
370
371#include "AliCDBManager.h"
372#include "AliCDBStorage.h"
373#include "AliCDBId.h"
84090f85 374#include "AliCDBRunRange.h"
375#include "AliCDBPath.h"
5164a766 376#include "AliCDBEntry.h"
73abe331 377#include "AliShuttleConfig.h"
eba76848 378#include "DCSClient/AliDCSClient.h"
73abe331 379#include "AliLog.h"
b948db8d 380#include "AliPreprocessor.h"
5164a766 381#include "AliShuttleStatus.h"
2bb7b766 382#include "AliShuttleLogbookEntry.h"
73abe331 383
57f50b3c 384#include <TSystem.h>
58bc3020 385#include <TObject.h>
b948db8d 386#include <TString.h>
57f50b3c 387#include <TTimeStamp.h>
73abe331 388#include <TObjString.h>
57f50b3c 389#include <TSQLServer.h>
390#include <TSQLResult.h>
391#include <TSQLRow.h>
cb343cfd 392#include <TMutex.h>
9827400b 393#include <TSystemDirectory.h>
394#include <TSystemFile.h>
a986b218 395#include <TFile.h>
9827400b 396#include <TGrid.h>
397#include <TGridResult.h>
73abe331 398
e7f62f16 399#include <TMonaLisaWriter.h>
400
5164a766 401#include <fstream>
402
cb343cfd 403#include <sys/types.h>
404#include <sys/wait.h>
405
73abe331 406ClassImp(AliShuttle)
407
b948db8d 408//______________________________________________________________________________________________
409AliShuttle::AliShuttle(const AliShuttleConfig* config,
410 UInt_t timeout, Int_t retries):
4f0ab988 411fConfig(config),
412fTimeout(timeout), fRetries(retries),
413fPreprocessorMap(),
2bb7b766 414fLogbookEntry(0),
eba76848 415fCurrentDetector(),
85a80aa9 416fStatusEntry(0),
cb343cfd 417fMonitoringMutex(0),
eba76848 418fLastActionTime(0),
e7f62f16 419fLastAction(),
9827400b 420fMonaLisa(0),
421fTestMode(kNone),
ffa29e93 422fReadTestMode(kFALSE),
423fOutputRedirected(kFALSE)
73abe331 424{
425 //
426 // config: AliShuttleConfig used
73abe331 427 // timeout: timeout used for AliDCSClient connection
428 // retries: the number of retries in case of connection error.
429 //
430
57f50b3c 431 if (!fConfig->IsValid()) AliFatal("********** !!!!! Invalid configuration !!!!! **********");
be48e3ea 432 for(int iSys=0;iSys<4;iSys++) {
57f50b3c 433 fServer[iSys]=0;
be48e3ea 434 if (iSys < 3)
2c15234c 435 fFXSlist[iSys].SetOwner(kTRUE);
57f50b3c 436 }
2bb7b766 437 fPreprocessorMap.SetOwner(kTRUE);
be48e3ea 438
439 for (UInt_t iDet=0; iDet<NDetectors(); iDet++)
440 fFirstUnprocessed[iDet] = kFALSE;
441
cb343cfd 442 fMonitoringMutex = new TMutex();
58bc3020 443}
444
b948db8d 445//______________________________________________________________________________________________
57f50b3c 446AliShuttle::~AliShuttle()
58bc3020 447{
9827400b 448 //
449 // destructor
450 //
58bc3020 451
b948db8d 452 fPreprocessorMap.DeleteAll();
be48e3ea 453 for(int iSys=0;iSys<4;iSys++)
57f50b3c 454 if(fServer[iSys]) {
455 fServer[iSys]->Close();
456 delete fServer[iSys];
eba76848 457 fServer[iSys] = 0;
57f50b3c 458 }
2bb7b766 459
460 if (fStatusEntry){
461 delete fStatusEntry;
462 fStatusEntry = 0;
463 }
cb343cfd 464
465 if (fMonitoringMutex)
466 {
467 delete fMonitoringMutex;
468 fMonitoringMutex = 0;
469 }
73abe331 470}
471
b948db8d 472//______________________________________________________________________________________________
57f50b3c 473void AliShuttle::RegisterPreprocessor(AliPreprocessor* preprocessor)
58bc3020 474{
73abe331 475 //
b948db8d 476 // Registers new AliPreprocessor.
73abe331 477 // It uses GetName() for indentificator of the pre processor.
478 // The pre processor is registered it there isn't any other
479 // with the same identificator (GetName()).
480 //
481
eba76848 482 const char* detName = preprocessor->GetName();
483 if(GetDetPos(detName) < 0)
484 AliFatal(Form("********** !!!!! Invalid detector name: %s !!!!! **********", detName));
485
486 if (fPreprocessorMap.GetValue(detName)) {
487 AliWarning(Form("AliPreprocessor %s is already registered!", detName));
73abe331 488 return;
489 }
490
eba76848 491 fPreprocessorMap.Add(new TObjString(detName), preprocessor);
73abe331 492}
b948db8d 493//______________________________________________________________________________________________
3301427a 494Bool_t AliShuttle::Store(const AliCDBPath& path, TObject* object,
84090f85 495 AliCDBMetaData* metaData, Int_t validityStart, Bool_t validityInfinite)
73abe331 496{
9827400b 497 // Stores a CDB object in the storage for offline reconstruction. Objects that are not needed for
498 // offline reconstruction, but should be stored anyway (e.g. for debugging) should NOT be stored
499 // using this function. Use StoreReferenceData instead!
500 // It calls StoreLocally function which temporarily stores the data locally; when the preprocessor
501 // finishes the data are transferred to the main storage (Grid).
b948db8d 502
3301427a 503 return StoreLocally(fgkLocalCDB, path, object, metaData, validityStart, validityInfinite);
84090f85 504}
505
506//______________________________________________________________________________________________
3301427a 507Bool_t AliShuttle::StoreReferenceData(const AliCDBPath& path, TObject* object, AliCDBMetaData* metaData)
84090f85 508{
9827400b 509 // Stores a CDB object in the storage for reference data. This objects will not be available during
510 // offline reconstrunction. Use this function for reference data only!
511 // It calls StoreLocally function which temporarily stores the data locally; when the preprocessor
512 // finishes the data are transferred to the main storage (Grid).
85a80aa9 513
3301427a 514 return StoreLocally(fgkLocalRefStorage, path, object, metaData);
85a80aa9 515}
516
517//______________________________________________________________________________________________
3301427a 518Bool_t AliShuttle::StoreLocally(const TString& localUri,
85a80aa9 519 const AliCDBPath& path, TObject* object, AliCDBMetaData* metaData,
520 Int_t validityStart, Bool_t validityInfinite)
521{
9827400b 522 // Store object temporarily in local storage. Parameters are passed by Store and StoreReferenceData functions.
523 // when the preprocessor finishes the data are transferred to the main storage (Grid).
524 // The parameters are:
525 // 1) Uri of the backup storage (Local)
526 // 2) the object's path.
527 // 3) the object to be stored
528 // 4) the metaData to be associated with the object
529 // 5) the validity start run number w.r.t. the current run,
530 // if the data is valid only for this run leave the default 0
531 // 6) specifies if the calibration data is valid for infinity (this means until updated),
532 // typical for calibration runs, the default is kFALSE
533 //
534 // returns 0 if fail, 1 otherwise
84090f85 535
9827400b 536 if (fTestMode & kErrorStorage)
537 {
538 Log(fCurrentDetector, "StoreLocally - In TESTMODE - Simulating error while storing locally");
539 return kFALSE;
540 }
541
3301427a 542 const char* cdbType = (localUri == fgkLocalCDB) ? "CDB" : "Reference";
2bb7b766 543
85a80aa9 544 Int_t firstRun = GetCurrentRun() - validityStart;
84090f85 545 if(firstRun < 0) {
9827400b 546 AliWarning("First valid run happens to be less than 0! Setting it to 0.");
84090f85 547 firstRun=0;
548 }
549
550 Int_t lastRun = -1;
551 if(validityInfinite) {
552 lastRun = AliCDBRunRange::Infinity();
553 } else {
554 lastRun = GetCurrentRun();
555 }
556
3301427a 557 // Version is set to current run, it will be used later to transfer data to Grid
558 AliCDBId id(path, firstRun, lastRun, GetCurrentRun(), -1);
2bb7b766 559
560 if(! dynamic_cast<TObjString*> (metaData->GetProperty("RunUsed(TObjString)"))){
561 TObjString runUsed = Form("%d", GetCurrentRun());
9e080f92 562 metaData->SetProperty("RunUsed(TObjString)", runUsed.Clone());
2bb7b766 563 }
84090f85 564
3301427a 565 Bool_t result = kFALSE;
84090f85 566
3301427a 567 if (!(AliCDBManager::Instance()->GetStorage(localUri))) {
568 Log("SHUTTLE", Form("StoreLocally - Cannot activate local %s storage", cdbType));
84090f85 569 } else {
3301427a 570 result = AliCDBManager::Instance()->GetStorage(localUri)
84090f85 571 ->Put(object, id, metaData);
572 }
573
574 if(!result) {
575
9827400b 576 Log(fCurrentDetector, Form("StoreLocally - Can't store object <%s>!", id.ToString().Data()));
3301427a 577 }
2bb7b766 578
3301427a 579 return result;
580}
84090f85 581
3301427a 582//______________________________________________________________________________________________
583Bool_t AliShuttle::StoreOCDB()
584{
9827400b 585 //
586 // Called when preprocessor ends successfully or when previous storage attempt failed (kStoreError status)
587 // Calls underlying StoreOCDB(const char*) function twice, for OCDB and Reference storage.
588 // Then calls StoreRefFilesToGrid to store reference files.
589 //
590
591 if (fTestMode & kErrorGrid)
592 {
593 Log("SHUTTLE", "StoreOCDB - In TESTMODE - Simulating error while storing in the Grid");
594 Log(fCurrentDetector, "StoreOCDB - In TESTMODE - Simulating error while storing in the Grid");
595 return kFALSE;
596 }
597
c88ad5db 598 Log("SHUTTLE","StoreOCDB - Storing OCDB data ...");
86aa42c3 599 Bool_t resultCDB = StoreOCDB(fgkMainCDB);
600
c88ad5db 601 Log("SHUTTLE","StoreOCDB - Storing reference data ...");
3301427a 602 Bool_t resultRef = StoreOCDB(fgkMainRefStorage);
9827400b 603
c88ad5db 604 Log("SHUTTLE","StoreOCDB - Storing reference files ...");
605 Bool_t resultRefFiles = CopyFilesToGrid("reference");
606
607 Bool_t resultMetadata = kTRUE;
608 if(fCurrentDetector == "GRP")
609 {
610 Log("StoreOCDB - SHUTTLE","Storing Run Metadata file ...");
611 resultMetadata = CopyFilesToGrid("metadata");
612 }
9827400b 613
c88ad5db 614 return resultCDB && resultRef && resultRefFiles && resultMetadata;
3301427a 615}
616
617//______________________________________________________________________________________________
618Bool_t AliShuttle::StoreOCDB(const TString& gridURI)
619{
620 //
621 // Called by StoreOCDB(), performs actual storage to the main OCDB and reference storages (Grid)
622 //
623
624 TObjArray* gridIds=0;
625
626 Bool_t result = kTRUE;
627
628 const char* type = 0;
629 TString localURI;
630 if(gridURI == fgkMainCDB) {
631 type = "OCDB";
632 localURI = fgkLocalCDB;
633 } else if(gridURI == fgkMainRefStorage) {
634 type = "reference";
635 localURI = fgkLocalRefStorage;
636 } else {
637 AliError(Form("Invalid storage URI: %s", gridURI.Data()));
638 return kFALSE;
639 }
640
641 AliCDBManager* man = AliCDBManager::Instance();
642
643 AliCDBStorage *gridSto = man->GetStorage(gridURI);
644 if(!gridSto) {
645 Log("SHUTTLE",
646 Form("StoreOCDB - cannot activate main %s storage", type));
647 return kFALSE;
648 }
649
650 gridIds = gridSto->GetQueryCDBList();
651
652 // get objects previously stored in local CDB
653 AliCDBStorage *localSto = man->GetStorage(localURI);
654 if(!localSto) {
655 Log("SHUTTLE",
656 Form("StoreOCDB - cannot activate local %s storage", type));
657 return kFALSE;
658 }
659 AliCDBPath aPath(GetOfflineDetName(fCurrentDetector.Data()),"*","*");
660 // Local objects were stored with current run as Grid version!
661 TList* localEntries = localSto->GetAll(aPath.GetPath(), GetCurrentRun(), GetCurrentRun());
662 localEntries->SetOwner(1);
663
664 // loop on local stored objects
665 TIter localIter(localEntries);
666 AliCDBEntry *aLocEntry = 0;
667 while((aLocEntry = dynamic_cast<AliCDBEntry*> (localIter.Next()))){
668 aLocEntry->SetOwner(1);
669 AliCDBId aLocId = aLocEntry->GetId();
670 aLocEntry->SetVersion(-1);
671 aLocEntry->SetSubVersion(-1);
672
673 // If local object is valid up to infinity we store it only if it is
674 // the first unprocessed run!
675 if (aLocId.GetLastRun() == AliCDBRunRange::Infinity() &&
676 !fFirstUnprocessed[GetDetPos(fCurrentDetector)])
677 {
678 Log("SHUTTLE", Form("StoreOCDB - %s: object %s has validity infinite but "
679 "there are previous unprocessed runs!",
680 fCurrentDetector.Data(), aLocId.GetPath().Data()));
4508ab8b 681 result = kFALSE;
3301427a 682 continue;
683 }
684
685 // loop on Grid valid Id's
686 Bool_t store = kTRUE;
687 TIter gridIter(gridIds);
688 AliCDBId* aGridId = 0;
689 while((aGridId = dynamic_cast<AliCDBId*> (gridIter.Next()))){
690 if(aGridId->GetPath() != aLocId.GetPath()) continue;
691 // skip all objects valid up to infinity
692 if(aGridId->GetLastRun() == AliCDBRunRange::Infinity()) continue;
693 // if we get here, it means there's already some more recent object stored on Grid!
694 store = kFALSE;
695 break;
696 }
697
698 // If we get here, the file can be stored!
699 Bool_t storeOk = gridSto->Put(aLocEntry);
700 if(!store || storeOk){
701
702 if (!store)
703 {
704 Log(fCurrentDetector.Data(),
705 Form("StoreOCDB - A more recent object already exists in %s storage: <%s>",
706 type, aGridId->ToString().Data()));
707 } else {
708 Log("SHUTTLE",
709 Form("StoreOCDB - Object <%s> successfully put into %s storage",
710 aLocId.ToString().Data(), type));
2d9019b4 711 Log(fCurrentDetector.Data(),
712 Form("StoreOCDB - Object <%s> successfully put into %s storage",
713 aLocId.ToString().Data(), type));
3301427a 714 }
84090f85 715
3301427a 716 // removing local filename...
717 TString filename;
718 localSto->IdToFilename(aLocId, filename);
c88ad5db 719 Log("SHUTTLE", Form("StoreOCDB - Removing local file %s", filename.Data()));
3301427a 720 RemoveFile(filename.Data());
721 continue;
722 } else {
723 Log("SHUTTLE",
724 Form("StoreOCDB - Grid %s storage of object <%s> failed",
725 type, aLocId.ToString().Data()));
2d9019b4 726 Log(fCurrentDetector.Data(),
727 Form("StoreOCDB - Grid %s storage of object <%s> failed",
728 type, aLocId.ToString().Data()));
3301427a 729 result = kFALSE;
b948db8d 730 }
731 }
3301427a 732 localEntries->Clear();
2bb7b766 733
b948db8d 734 return result;
3301427a 735}
736
546242fb 737//______________________________________________________________________________________________
738Bool_t AliShuttle::CleanReferenceStorage(const char* detector)
739{
2d9019b4 740 // clears the directory used to store reference files of a given subdetector
546242fb 741
742 AliCDBManager* man = AliCDBManager::Instance();
743 AliCDBStorage* sto = man->GetStorage(fgkLocalRefStorage);
2d9019b4 744 TString localBaseFolder = sto->GetBaseFolder();
745
746 TString targetDir = GetRefFilePrefix(localBaseFolder.Data(), detector);
747
d524ade6 748 Log("SHUTTLE", Form("CleanReferenceStorage - Cleaning %s", targetDir.Data()));
2d9019b4 749
750 TString begin;
751 begin.Form("%d_", GetCurrentRun());
752
753 TSystemDirectory* baseDir = new TSystemDirectory("/", targetDir);
754 if (!baseDir)
755 return kTRUE;
756
757 TList* dirList = baseDir->GetListOfFiles();
758 delete baseDir;
759
760 if (!dirList) return kTRUE;
761
762 if (dirList->GetEntries() < 3)
763 {
764 delete dirList;
765 return kTRUE;
766 }
767
768 Int_t nDirs = 0, nDel = 0;
769 TIter dirIter(dirList);
770 TSystemFile* entry = 0;
546242fb 771
2d9019b4 772 Bool_t success = kTRUE;
546242fb 773
2d9019b4 774 while ((entry = dynamic_cast<TSystemFile*> (dirIter.Next())))
775 {
776 if (entry->IsDirectory())
777 continue;
778
779 TString fileName(entry->GetName());
780 if (!fileName.BeginsWith(begin))
781 continue;
782
783 nDirs++;
784
785 // delete file
786 Int_t result = gSystem->Unlink(fileName.Data());
787
788 if (result)
789 {
d524ade6 790 Log("SHUTTLE", Form("CleanReferenceStorage - Could not delete file %s!", fileName.Data()));
2d9019b4 791 success = kFALSE;
792 } else {
793 nDel++;
794 }
795 }
796
797 if(nDirs > 0)
798 Log("SHUTTLE", Form("CleanReferenceStorage - %d (over %d) reference files in folder %s were deleted.",
799 nDel, nDirs, targetDir.Data()));
800
801
802 delete dirList;
803 return success;
804
805
806
807
808
546242fb 809
810 Int_t result = gSystem->GetPathInfo(targetDir, 0, (Long64_t*) 0, 0, 0);
811 if (result == 0)
812 {
813 // delete directory
d524ade6 814 result = gSystem->Exec(Form("rm -rf %s", targetDir.Data()));
546242fb 815 if (result != 0)
816 {
d524ade6 817 Log("SHUTTLE", Form("CleanReferenceStorage - Could not clean directory %s", targetDir.Data()));
546242fb 818 return kFALSE;
819 }
820 }
821
822 result = gSystem->mkdir(targetDir, kTRUE);
823 if (result != 0)
824 {
c88ad5db 825 Log("SHUTTLE", Form("CleanReferenceStorage - Error creating base directory %s", targetDir.Data()));
546242fb 826 return kFALSE;
827 }
828
829 return kTRUE;
830}
831
9827400b 832//______________________________________________________________________________________________
833Bool_t AliShuttle::StoreReferenceFile(const char* detector, const char* localFile, const char* gridFileName)
834{
835 //
3c2a21c8 836 // Stores reference file directly (without opening it). This function stores the file locally.
9827400b 837 //
3c2a21c8 838 // The file is stored under the following location:
839 // <base folder of local reference storage>/<DET>/<RUN#>_<gridFileName>
840 // where <gridFileName> is the second parameter given to the function
841 //
9827400b 842
843 if (fTestMode & kErrorStorage)
844 {
845 Log(fCurrentDetector, "StoreReferenceFile - In TESTMODE - Simulating error while storing locally");
846 return kFALSE;
847 }
848
849 AliCDBManager* man = AliCDBManager::Instance();
850 AliCDBStorage* sto = man->GetStorage(fgkLocalRefStorage);
851
852 TString localBaseFolder = sto->GetBaseFolder();
853
d524ade6 854 TString target = GetRefFilePrefix(localBaseFolder.Data(), detector);
855 target.Append(Form("/%d_%s", GetCurrentRun(), gridFileName));
9827400b 856
d524ade6 857 return CopyFileLocally(localFile, target);
c88ad5db 858}
859
860//______________________________________________________________________________________________
861Bool_t AliShuttle::StoreRunMetadataFile(const char* localFile, const char* gridFileName)
862{
863 //
864 // Stores Run metadata file to the Grid, in the run folder
865 //
866 // Only GRP can call this function.
867
868 if (fTestMode & kErrorStorage)
869 {
870 Log(fCurrentDetector, "StoreRunMetaDataFile - In TESTMODE - Simulating error while storing locally");
871 return kFALSE;
872 }
873
874 AliCDBManager* man = AliCDBManager::Instance();
875 AliCDBStorage* sto = man->GetStorage(fgkLocalRefStorage);
876
877 TString localBaseFolder = sto->GetBaseFolder();
878
879 // Build Run level folder
955d2abc 880 // folder = /alice/data/year/lhcPeriod/runNb/raw
c88ad5db 881
c88ad5db 882
675f64cd 883 TString lhcPeriod = GetLHCPeriod();
c88ad5db 884 if (lhcPeriod.Length() == 0)
885 {
886 Log("SHUTTLE","StoreRunMetaDataFile - LHCPeriod not found in logbook!");
887 return 0;
888 }
a1c5723b 889
890 // TODO partitions with one detector only write data into LHCperiod_DET
2694f253 891 TString partition = GetRunParameter("detector");
a1c5723b 892
893 if (partition.Length() > 0 && partition != "ALICE")
894 {
895 lhcPeriod.Append(Form("_%s", partition.Data()));
896 Log(fCurrentDetector, Form("Run data tags merged file will be written in %s",
897 lhcPeriod.Data()));
898 }
8884d71e 899
955d2abc 900 TString target = Form("%s/GRP/RunMetadata/alice/data/%d/%s/%09d/raw/%s",
675f64cd 901 localBaseFolder.Data(), GetCurrentYear(),
d524ade6 902 lhcPeriod.Data(), GetCurrentRun(), gridFileName);
c88ad5db 903
d524ade6 904 return CopyFileLocally(localFile, target);
c88ad5db 905}
906
907//______________________________________________________________________________________________
d524ade6 908Bool_t AliShuttle::CopyFileLocally(const char* localFile, const TString& target)
c88ad5db 909{
910 //
911 // Stores file locally. Called by StoreReferenceFile and StoreRunMetadataFile
d524ade6 912 // Files are temporarily stored in the local reference storage. When the preprocessor
913 // finishes, the Shuttle calls CopyFilesToGrid to transfer the files to AliEn
914 // (in reference or run level folders)
c88ad5db 915 //
916
d524ade6 917 TString targetDir(target(0, target.Last('/')));
918
919 //try to open base dir folder, if it does not exist
2d9019b4 920 void* dir = gSystem->OpenDirectory(targetDir.Data());
921 if (dir == NULL) {
922 if (gSystem->mkdir(targetDir.Data(), kTRUE)) {
a1c5723b 923 Log("SHUTTLE", Form("CopyFileLocally - Can't open directory <%s>", targetDir.Data()));
2d9019b4 924 return kFALSE;
925 }
926
927 } else {
928 gSystem->FreeDirectory(dir);
929 }
9827400b 930
7d43a416 931 Int_t result = 0;
932
933 result = gSystem->GetPathInfo(localFile, 0, (Long64_t*) 0, 0, 0);
9827400b 934 if (result)
935 {
a1c5723b 936 Log("SHUTTLE", Form("CopyFileLocally - %s does not exist", localFile));
546242fb 937 return kFALSE;
9827400b 938 }
546242fb 939
7d43a416 940 result = gSystem->GetPathInfo(target, 0, (Long64_t*) 0, 0, 0);
941 if (!result)
942 {
a1c5723b 943 Log("SHUTTLE", Form("CopyFileLocally - target file %s already exist, removing...", target.Data()));
7d43a416 944 if (gSystem->Unlink(target.Data()))
945 {
a1c5723b 946 Log("SHUTTLE", Form("CopyFileLocally - Could not remove existing target file %s!", target.Data()));
7d43a416 947 return kFALSE;
948 }
949 }
950
9827400b 951 result = gSystem->CopyFile(localFile, target);
952
953 if (result == 0)
954 {
a1c5723b 955 Log("SHUTTLE", Form("CopyFileLocally - File %s stored locally to %s", localFile, target.Data()));
9827400b 956 return kTRUE;
957 }
958 else
959 {
a1c5723b 960 Log("SHUTTLE", Form("CopyFileLocally - Could not store file %s to %s! Error code = %d",
546242fb 961 localFile, target.Data(), result));
9827400b 962 return kFALSE;
963 }
c88ad5db 964
965
966
9827400b 967}
968
969//______________________________________________________________________________________________
c88ad5db 970Bool_t AliShuttle::CopyFilesToGrid(const char* type)
9827400b 971{
972 //
c88ad5db 973 // Transfers local files to the Grid. Local files can be reference files
974 // or run metadata file (from GRP only).
9827400b 975 //
c88ad5db 976 // According to the type (ref, metadata) the files are stored under the following location:
977 // ref --> <base folder of reference storage>/<DET>/<RUN#>_<gridFileName>
978 // metadata --> <run data folder>/<MetadataFileName>
86aa42c3 979 //
c88ad5db 980
9827400b 981 AliCDBManager* man = AliCDBManager::Instance();
982 AliCDBStorage* sto = man->GetStorage(fgkLocalRefStorage);
983 if (!sto)
984 return kFALSE;
985 TString localBaseFolder = sto->GetBaseFolder();
9827400b 986
c88ad5db 987 TString dir;
988 TString alienDir;
9827400b 989 TString begin;
9827400b 990
c88ad5db 991 if (strcmp(type, "reference") == 0)
992 {
993 dir = GetRefFilePrefix(localBaseFolder.Data(), fCurrentDetector.Data());
994 AliCDBStorage* gridSto = man->GetStorage(fgkMainRefStorage);
995 if (!gridSto)
996 return kFALSE;
997 TString gridBaseFolder = gridSto->GetBaseFolder();
998 alienDir = GetRefFilePrefix(gridBaseFolder.Data(), fCurrentDetector.Data());
999 begin = Form("%d_", GetCurrentRun());
1000 }
1001 else if (strcmp(type, "metadata") == 0)
1002 {
c88ad5db 1003
675f64cd 1004 TString lhcPeriod = GetLHCPeriod();
c88ad5db 1005
1006 if (lhcPeriod.Length() == 0)
1007 {
1008 Log("SHUTTLE","CopyFilesToGrid - LHCPeriod not found in logbook!");
1009 return 0;
1010 }
1011
a1c5723b 1012 // TODO partitions with one detector only write data into LHCperiod_DET
2694f253 1013 TString partition = GetRunParameter("detector");
a1c5723b 1014
1015 if (partition.Length() > 0 && partition != "ALICE")
1016 {
1017 lhcPeriod.Append(Form("_%s", partition.Data()));
1018 }
1019
955d2abc 1020 dir = Form("%s/GRP/RunMetadata/alice/data/%d/%s/%09d/raw",
675f64cd 1021 localBaseFolder.Data(), GetCurrentYear(),
c88ad5db 1022 lhcPeriod.Data(), GetCurrentRun());
675f64cd 1023 alienDir = dir(dir.Index("/alice/data/"), dir.Length());
1024
c88ad5db 1025 begin = "";
1026 }
1027 else
1028 {
1029 Log("SHUTTLE", "CopyFilesToGrid - Unexpected: type label must be reference or metadata!");
1030 return kFALSE;
1031 }
1032
9827400b 1033 TSystemDirectory* baseDir = new TSystemDirectory("/", dir);
3d8bc902 1034 if (!baseDir)
1035 return kTRUE;
1036
2d9019b4 1037 TList* dirList = baseDir->GetListOfFiles();
1038 delete baseDir;
1039
1040 if (!dirList) return kTRUE;
1041
1042 if (dirList->GetEntries() < 3)
3d8bc902 1043 {
2d9019b4 1044 delete dirList;
9827400b 1045 return kTRUE;
3d8bc902 1046 }
2d9019b4 1047
546242fb 1048 if (!gGrid)
1049 {
c88ad5db 1050 Log("SHUTTLE", "CopyFilesToGrid - Connection to Grid failed: Cannot continue!");
2d9019b4 1051 delete dirList;
546242fb 1052 return kFALSE;
1053 }
1054
2d9019b4 1055 Int_t nDirs = 0, nTransfer = 0;
1056 TIter dirIter(dirList);
1057 TSystemFile* entry = 0;
1058
9827400b 1059 Bool_t success = kTRUE;
3d8bc902 1060 Bool_t first = kTRUE;
9827400b 1061
2d9019b4 1062 while ((entry = dynamic_cast<TSystemFile*> (dirIter.Next())))
1063 {
9827400b 1064 if (entry->IsDirectory())
1065 continue;
1066
1067 TString fileName(entry->GetName());
1068 if (!fileName.BeginsWith(begin))
1069 continue;
1070
2d9019b4 1071 nDirs++;
1072
3d8bc902 1073 if (first)
1074 {
1075 first = kFALSE;
c88ad5db 1076 // check that folder exists, otherwise create it
3d8bc902 1077 TGridResult* result = gGrid->Ls(alienDir.Data(), "a");
1078
1079 if (!result)
2d9019b4 1080 {
1081 delete dirList;
3d8bc902 1082 return kFALSE;
2d9019b4 1083 }
3d8bc902 1084
546242fb 1085 if (!result->GetFileName(1)) // TODO: It looks like element 0 is always 0!!
3d8bc902 1086 {
675f64cd 1087 // TODO It does not work currently! Bug in TAliEn::Mkdir
1088 // TODO Manually fixed in local root v5-16-00
c88ad5db 1089 if (!gGrid->Mkdir(alienDir.Data(),"-p",0))
3d8bc902 1090 {
c88ad5db 1091 Log("SHUTTLE", Form("CopyFilesToGrid - Cannot create directory %s",
3d8bc902 1092 alienDir.Data()));
2d9019b4 1093 delete dirList;
3d8bc902 1094 return kFALSE;
546242fb 1095 } else {
c88ad5db 1096 Log("SHUTTLE",Form("CopyFilesToGrid - Folder %s created", alienDir.Data()));
3d8bc902 1097 }
1098
546242fb 1099 } else {
c88ad5db 1100 Log("SHUTTLE",Form("CopyFilesToGrid - Folder %s found", alienDir.Data()));
3d8bc902 1101 }
1102 }
1103
9827400b 1104 TString fullLocalPath;
1105 fullLocalPath.Form("%s/%s", dir.Data(), fileName.Data());
1106
1107 TString fullGridPath;
1108 fullGridPath.Form("alien://%s/%s", alienDir.Data(), fileName.Data());
1109
a986b218 1110 Bool_t result = TFile::Cp(fullLocalPath, fullGridPath);
9827400b 1111
1112 if (result)
1113 {
c88ad5db 1114 Log("SHUTTLE", Form("CopyFilesToGrid - Copying local file %s to %s succeeded!",
1115 fullLocalPath.Data(), fullGridPath.Data()));
9827400b 1116 RemoveFile(fullLocalPath);
2d9019b4 1117 nTransfer++;
9827400b 1118 }
1119 else
1120 {
c88ad5db 1121 Log("SHUTTLE", Form("CopyFilesToGrid - Copying local file %s to %s FAILED!",
1122 fullLocalPath.Data(), fullGridPath.Data()));
9827400b 1123 success = kFALSE;
1124 }
1125 }
2d9019b4 1126
c88ad5db 1127 Log("SHUTTLE", Form("CopyFilesToGrid - %d (over %d) files in folder %s copied to Grid.",
1128 nTransfer, nDirs, dir.Data()));
2d9019b4 1129
1130
1131 delete dirList;
9827400b 1132 return success;
1133}
1134
2d9019b4 1135//______________________________________________________________________________________________
1136const char* AliShuttle::GetRefFilePrefix(const char* base, const char* detector)
1137{
1138 //
1139 // Get folder name of reference files
1140 //
1141
1142 TString offDetStr(GetOfflineDetName(detector));
1143 TString dir;
1144 if (offDetStr == "ITS" || offDetStr == "MUON" || offDetStr == "PHOS")
1145 {
1146 dir.Form("%s/%s/%s", base, offDetStr.Data(), detector);
1147 } else {
1148 dir.Form("%s/%s", base, offDetStr.Data());
1149 }
1150
1151 return dir.Data();
1152
1153
1154}
c88ad5db 1155
3301427a 1156//______________________________________________________________________________________________
1157void AliShuttle::CleanLocalStorage(const TString& uri)
1158{
9827400b 1159 //
1160 // Called in case the preprocessor is declared failed. Remove remaining objects from the local storages.
1161 //
3301427a 1162
1163 const char* type = 0;
1164 if(uri == fgkLocalCDB) {
1165 type = "OCDB";
1166 } else if(uri == fgkLocalRefStorage) {
546242fb 1167 type = "Reference";
3301427a 1168 } else {
1169 AliError(Form("Invalid storage URI: %s", uri.Data()));
1170 return;
1171 }
1172
1173 AliCDBManager* man = AliCDBManager::Instance();
b948db8d 1174
3301427a 1175 // open local storage
1176 AliCDBStorage *localSto = man->GetStorage(uri);
1177 if(!localSto) {
1178 Log("SHUTTLE",
1179 Form("CleanLocalStorage - cannot activate local %s storage", type));
1180 return;
1181 }
1182
1183 TString filename(Form("%s/%s/*/Run*_v%d_s*.root",
546242fb 1184 localSto->GetBaseFolder().Data(), GetOfflineDetName(fCurrentDetector.Data()), GetCurrentRun()));
3301427a 1185
c88ad5db 1186 AliDebug(2, Form("filename = %s", filename.Data()));
3301427a 1187
c88ad5db 1188 Log("SHUTTLE", Form("Removing remaining local files for run %d and detector %s ...",
3301427a 1189 GetCurrentRun(), fCurrentDetector.Data()));
1190
1191 RemoveFile(filename.Data());
1192
1193}
1194
1195//______________________________________________________________________________________________
1196void AliShuttle::RemoveFile(const char* filename)
1197{
9827400b 1198 //
1199 // removes local file
1200 //
3301427a 1201
1202 TString command(Form("rm -f %s", filename));
1203
1204 Int_t result = gSystem->Exec(command.Data());
1205 if(result != 0)
1206 {
1207 Log("SHUTTLE", Form("RemoveFile - %s: Cannot remove file %s!",
1208 fCurrentDetector.Data(), filename));
1209 }
73abe331 1210}
1211
b948db8d 1212//______________________________________________________________________________________________
5164a766 1213AliShuttleStatus* AliShuttle::ReadShuttleStatus()
1214{
9827400b 1215 //
1216 // Reads the AliShuttleStatus from the CDB
1217 //
5164a766 1218
2bb7b766 1219 if (fStatusEntry){
1220 delete fStatusEntry;
1221 fStatusEntry = 0;
1222 }
5164a766 1223
10a5a932 1224 fStatusEntry = AliCDBManager::Instance()->GetStorage(GetLocalCDB())
2bb7b766 1225 ->Get(Form("/SHUTTLE/STATUS/%s", fCurrentDetector.Data()), GetCurrentRun());
5164a766 1226
2bb7b766 1227 if (!fStatusEntry) return 0;
1228 fStatusEntry->SetOwner(1);
5164a766 1229
2bb7b766 1230 AliShuttleStatus* status = dynamic_cast<AliShuttleStatus*> (fStatusEntry->GetObject());
1231 if (!status) {
1232 AliError("Invalid object stored to CDB!");
1233 return 0;
1234 }
5164a766 1235
2bb7b766 1236 return status;
5164a766 1237}
1238
1239//______________________________________________________________________________________________
7bfb2090 1240Bool_t AliShuttle::WriteShuttleStatus(AliShuttleStatus* status)
5164a766 1241{
9827400b 1242 //
1243 // writes the status for one subdetector
1244 //
2bb7b766 1245
1246 if (fStatusEntry){
1247 delete fStatusEntry;
1248 fStatusEntry = 0;
1249 }
5164a766 1250
2bb7b766 1251 Int_t run = GetCurrentRun();
5164a766 1252
2bb7b766 1253 AliCDBId id(AliCDBPath("SHUTTLE", "STATUS", fCurrentDetector), run, run);
5164a766 1254
2bb7b766 1255 fStatusEntry = new AliCDBEntry(status, id, new AliCDBMetaData);
1256 fStatusEntry->SetOwner(1);
5164a766 1257
2bb7b766 1258 UInt_t result = AliCDBManager::Instance()->GetStorage(fgkLocalCDB)->Put(fStatusEntry);
7bfb2090 1259
2bb7b766 1260 if (!result) {
3301427a 1261 Log("SHUTTLE", Form("WriteShuttleStatus - Failed for %s, run %d",
1262 fCurrentDetector.Data(), run));
2bb7b766 1263 return kFALSE;
1264 }
e7f62f16 1265
1266 SendMLInfo();
7bfb2090 1267
2bb7b766 1268 return kTRUE;
5164a766 1269}
1270
1271//______________________________________________________________________________________________
1272void AliShuttle::UpdateShuttleStatus(AliShuttleStatus::Status newStatus, Bool_t increaseCount)
1273{
9827400b 1274 //
1275 // changes the AliShuttleStatus for the given detector and run to the given status
1276 //
5164a766 1277
2bb7b766 1278 if (!fStatusEntry){
1279 AliError("UNEXPECTED: fStatusEntry empty");
1280 return;
1281 }
5164a766 1282
2bb7b766 1283 AliShuttleStatus* status = dynamic_cast<AliShuttleStatus*> (fStatusEntry->GetObject());
5164a766 1284
2bb7b766 1285 if (!status){
c88ad5db 1286 Log("SHUTTLE", "UpdateShuttleStatus - UNEXPECTED: status could not be read from current CDB entry");
2bb7b766 1287 return;
1288 }
5164a766 1289
2c15234c 1290 TString actionStr = Form("UpdateShuttleStatus - %s: Changing state from %s to %s",
eba76848 1291 fCurrentDetector.Data(),
36c99a6a 1292 status->GetStatusName(),
eba76848 1293 status->GetStatusName(newStatus));
cb343cfd 1294 Log("SHUTTLE", actionStr);
1295 SetLastAction(actionStr);
5164a766 1296
2bb7b766 1297 status->SetStatus(newStatus);
1298 if (increaseCount) status->IncreaseCount();
5164a766 1299
2bb7b766 1300 AliCDBManager::Instance()->GetStorage(fgkLocalCDB)->Put(fStatusEntry);
e7f62f16 1301
1302 SendMLInfo();
5164a766 1303}
e7f62f16 1304
1305//______________________________________________________________________________________________
1306void AliShuttle::SendMLInfo()
1307{
1308 //
1309 // sends ML information about the current status of the current detector being processed
1310 //
1311
1312 AliShuttleStatus* status = dynamic_cast<AliShuttleStatus*> (fStatusEntry->GetObject());
1313
1314 if (!status){
3301427a 1315 Log("SHUTTLE", "SendMLInfo - UNEXPECTED: status could not be read from current CDB entry");
e7f62f16 1316 return;
1317 }
1318
1319 TMonaLisaText mlStatus(Form("%s_status", fCurrentDetector.Data()), status->GetStatusName());
1320 TMonaLisaValue mlRetryCount(Form("%s_count", fCurrentDetector.Data()), status->GetCount());
1321
1322 TList mlList;
1323 mlList.Add(&mlStatus);
1324 mlList.Add(&mlRetryCount);
1325
ee6f7523 1326 TString mlID;
1327 mlID.Form("%d", GetCurrentRun());
1328 fMonaLisa->SendParameters(&mlList, mlID);
e7f62f16 1329}
1330
5164a766 1331//______________________________________________________________________________________________
1332Bool_t AliShuttle::ContinueProcessing()
1333{
9827400b 1334 // this function reads the AliShuttleStatus information from CDB and
1335 // checks if the processing should be continued
1336 // if yes it returns kTRUE and updates the AliShuttleStatus with nextStatus
2bb7b766 1337
57c1a579 1338 if (!fConfig->HostProcessDetector(fCurrentDetector)) return kFALSE;
1339
1340 AliPreprocessor* aPreprocessor =
1341 dynamic_cast<AliPreprocessor*> (fPreprocessorMap.GetValue(fCurrentDetector));
1342 if (!aPreprocessor)
1343 {
c88ad5db 1344 Log("SHUTTLE", Form("ContinueProcessing - %s: no preprocessor registered", fCurrentDetector.Data()));
57c1a579 1345 return kFALSE;
1346 }
1347
2bb7b766 1348 AliShuttleLogbookEntry::Status entryStatus =
eba76848 1349 fLogbookEntry->GetDetectorStatus(fCurrentDetector);
2bb7b766 1350
1351 if(entryStatus != AliShuttleLogbookEntry::kUnprocessed) {
c88ad5db 1352 Log("SHUTTLE", Form("ContinueProcessing - %s is %s",
2bb7b766 1353 fCurrentDetector.Data(),
1354 fLogbookEntry->GetDetectorStatusName(entryStatus)));
1355 return kFALSE;
1356 }
1357
1358 // if we get here, according to Shuttle logbook subdetector is in UNPROCESSED state
be48e3ea 1359
1360 // check if current run is first unprocessed run for current detector
1361 if (fConfig->StrictRunOrder(fCurrentDetector) &&
1362 !fFirstUnprocessed[GetDetPos(fCurrentDetector)])
1363 {
86aa42c3 1364 if (fTestMode == kNone)
1365 {
c88ad5db 1366 Log("SHUTTLE", Form("ContinueProcessing - %s requires strict run ordering"
1367 " but this is not the first unprocessed run!"));
86aa42c3 1368 return kFALSE;
1369 }
1370 else
1371 {
c88ad5db 1372 Log("SHUTTLE", Form("ContinueProcessing - In TESTMODE - "
1373 "Although %s requires strict run ordering "
1374 "and this is not the first unprocessed run, "
1375 "the SHUTTLE continues"));
86aa42c3 1376 }
be48e3ea 1377 }
1378
2bb7b766 1379 AliShuttleStatus* status = ReadShuttleStatus();
1380 if (!status) {
1381 // first time
1382 Log("SHUTTLE", Form("ContinueProcessing - %s: Processing first time",
1383 fCurrentDetector.Data()));
1384 status = new AliShuttleStatus(AliShuttleStatus::kStarted);
1385 return WriteShuttleStatus(status);
1386 }
1387
1388 // The following two cases shouldn't happen if Shuttle Logbook was correctly updated.
1389 // If it happens it may mean Logbook updating failed... let's do it now!
1390 if (status->GetStatus() == AliShuttleStatus::kDone ||
1391 status->GetStatus() == AliShuttleStatus::kFailed){
1392 Log("SHUTTLE", Form("ContinueProcessing - %s is already %s. Updating Shuttle Logbook",
1393 fCurrentDetector.Data(),
1394 status->GetStatusName(status->GetStatus())));
1395 UpdateShuttleLogbook(fCurrentDetector.Data(),
1396 status->GetStatusName(status->GetStatus()));
1397 return kFALSE;
1398 }
1399
3301427a 1400 if (status->GetStatus() == AliShuttleStatus::kStoreError) {
2bb7b766 1401 Log("SHUTTLE",
c88ad5db 1402 Form("ContinueProcessing - %s: Grid storage of one or more "
1403 "objects failed. Trying again now",
2bb7b766 1404 fCurrentDetector.Data()));
9827400b 1405 UpdateShuttleStatus(AliShuttleStatus::kStoreStarted);
1406 if (StoreOCDB()){
c88ad5db 1407 Log("SHUTTLE", Form("ContinueProcessing - %s: all objects "
1408 "successfully stored into main storage",
3301427a 1409 fCurrentDetector.Data()));
a0aaf247 1410 UpdateShuttleStatus(AliShuttleStatus::kDone);
1411 UpdateShuttleLogbook(fCurrentDetector, "DONE");
2bb7b766 1412 } else {
1413 Log("SHUTTLE",
1414 Form("ContinueProcessing - %s: Grid storage failed again",
1415 fCurrentDetector.Data()));
9827400b 1416 UpdateShuttleStatus(AliShuttleStatus::kStoreError);
2bb7b766 1417 }
1418 return kFALSE;
1419 }
1420
1421 // if we get here, there is a restart
57c1a579 1422 Bool_t cont = kFALSE;
2bb7b766 1423
1424 // abort conditions
cb343cfd 1425 if (status->GetCount() >= fConfig->GetMaxRetries()) {
57c1a579 1426 Log("SHUTTLE", Form("ContinueProcessing - %s failed %d times in status %s - "
1427 "Updating Shuttle Logbook", fCurrentDetector.Data(),
2bb7b766 1428 status->GetCount(), status->GetStatusName()));
1429 UpdateShuttleLogbook(fCurrentDetector.Data(), "FAILED");
e7f62f16 1430 UpdateShuttleStatus(AliShuttleStatus::kFailed);
3301427a 1431
1432 // there may still be objects in local OCDB and reference storage
1433 // and FXS databases may be not updated: do it now!
9827400b 1434
1435 // TODO Currently disabled, we want to keep files in case of failure!
1436 // CleanLocalStorage(fgkLocalCDB);
1437 // CleanLocalStorage(fgkLocalRefStorage);
1438 // UpdateTableFailCase();
1439
1440 // Send mail to detector expert!
c88ad5db 1441 Log("SHUTTLE", Form("ContinueProcessing - Sending mail to %s expert...",
1442 fCurrentDetector.Data()));
9827400b 1443 if (!SendMail())
1444 Log("SHUTTLE", Form("ContinueProcessing - Could not send mail to %s expert",
1445 fCurrentDetector.Data()));
3301427a 1446
57c1a579 1447 } else {
1448 Log("SHUTTLE", Form("ContinueProcessing - %s: restarting. "
1449 "Aborted before with %s. Retry number %d.", fCurrentDetector.Data(),
1450 status->GetStatusName(), status->GetCount()));
9827400b 1451 Bool_t increaseCount = kTRUE;
c88ad5db 1452 if (status->GetStatus() == AliShuttleStatus::kDCSError ||
1453 status->GetStatus() == AliShuttleStatus::kDCSStarted)
1454 increaseCount = kFALSE;
675f64cd 1455
9827400b 1456 UpdateShuttleStatus(AliShuttleStatus::kStarted, increaseCount);
57c1a579 1457 cont = kTRUE;
2bb7b766 1458 }
1459
57c1a579 1460 return cont;
5164a766 1461}
1462
1463//______________________________________________________________________________________________
2bb7b766 1464Bool_t AliShuttle::Process(AliShuttleLogbookEntry* entry)
58bc3020 1465{
73abe331 1466 //
b948db8d 1467 // Makes data retrieval for all detectors in the configuration.
2bb7b766 1468 // entry: Shuttle logbook entry, contains run paramenters and status of detectors
1469 // (Unprocessed, Inactive, Failed or Done).
d477ad88 1470 // Returns kFALSE in case of error occured and kTRUE otherwise
73abe331 1471 //
1472
9827400b 1473 if (!entry) return kFALSE;
2bb7b766 1474
1475 fLogbookEntry = entry;
1476
c88ad5db 1477 Log("SHUTTLE", Form("\t\t\t^*^*^*^*^*^*^*^*^*^*^*^* run %d: START ^*^*^*^*^*^*^*^*^*^*^*^*",
9827400b 1478 GetCurrentRun()));
2bb7b766 1479
e7f62f16 1480 // Send the information to ML
1481 TMonaLisaText mlStatus("SHUTTLE_status", "Processing");
9827400b 1482 TMonaLisaText mlRunType("SHUTTLE_runtype", Form("%s (%s)", entry->GetRunType(), entry->GetRunParameter("log")));
e7f62f16 1483
1484 TList mlList;
1485 mlList.Add(&mlStatus);
9827400b 1486 mlList.Add(&mlRunType);
e7f62f16 1487
ee6f7523 1488 TString mlID;
1489 mlID.Form("%d", GetCurrentRun());
1490 fMonaLisa->SendParameters(&mlList, mlID);
3301427a 1491
9827400b 1492 if (fLogbookEntry->IsDone())
1493 {
1494 Log("SHUTTLE","Process - Shuttle is already DONE. Updating logbook");
1495 UpdateShuttleLogbook("shuttle_done");
1496 fLogbookEntry = 0;
1497 return kTRUE;
1498 }
1499
1500 // read test mode if flag is set
1501 if (fReadTestMode)
1502 {
3d8bc902 1503 fTestMode = kNone;
9827400b 1504 TString logEntry(entry->GetRunParameter("log"));
1505 //printf("log entry = %s\n", logEntry.Data());
1506 TString searchStr("Testmode: ");
1507 Int_t pos = logEntry.Index(searchStr.Data());
1508 //printf("%d\n", pos);
1509 if (pos >= 0)
1510 {
1511 TSubString subStr = logEntry(pos + searchStr.Length(), logEntry.Length());
1512 //printf("%s\n", subStr.String().Data());
1513 TString newStr(subStr.Data());
1514 TObjArray* token = newStr.Tokenize(' ');
1515 if (token)
1516 {
1517 //token->Print();
1518 TObjString* tmpStr = dynamic_cast<TObjString*> (token->First());
1519 if (tmpStr)
1520 {
1521 Int_t testMode = tmpStr->String().Atoi();
1522 if (testMode > 0)
1523 {
c88ad5db 1524 Log("SHUTTLE", Form("Process - Enabling test mode %d", testMode));
9827400b 1525 SetTestMode((TestMode) testMode);
1526 }
1527 }
1528 delete token;
1529 }
1530 }
1531 }
c88ad5db 1532
eba76848 1533 fLogbookEntry->Print("all");
57f50b3c 1534
1535 // Initialization
d477ad88 1536 Bool_t hasError = kFALSE;
5164a766 1537
675f64cd 1538 // Set the CDB and Reference folders according to the year and LHC period
1539 TString lhcPeriod(GetLHCPeriod());
1540 if (lhcPeriod.Length() == 0)
1541 {
7d4cf768 1542 Log("SHUTTLE","Process - LHCPeriod not found in logbook!");
1543 return 0;
675f64cd 1544 }
1545
1546 if (fgkMainCDB.Length() == 0)
1547 fgkMainCDB = Form("alien://folder=/alice/data/%d/%s/OCDB?user=alidaq?cacheFold=/tmp/OCDBCache",
1548 GetCurrentYear(), lhcPeriod.Data());
1549
1550 if (fgkMainRefStorage.Length() == 0)
1551 fgkMainRefStorage = Form("alien://folder=/alice/data/%d/%s/Reference?user=alidaq?cacheFold=/tmp/OCDBCache",
1552 GetCurrentYear(), lhcPeriod.Data());
1553
57f50b3c 1554 // Loop on detectors in the configuration
b948db8d 1555 TIter iter(fConfig->GetDetectors());
2bb7b766 1556 TObjString* aDetector = 0;
b948db8d 1557
4508ab8b 1558 Bool_t first = kTRUE;
1559
be48e3ea 1560 while ((aDetector = (TObjString*) iter.Next()))
1561 {
7bfb2090 1562 fCurrentDetector = aDetector->String();
5164a766 1563
9e080f92 1564 if (ContinueProcessing() == kFALSE) continue;
4508ab8b 1565
1566 if (first)
1567 {
1568 // only read QueryCDB when needed and only once
1569 AliCDBStorage *mainCDBSto = AliCDBManager::Instance()->GetStorage(fgkMainCDB);
1570 if(mainCDBSto) mainCDBSto->QueryCDB(GetCurrentRun());
1571 AliCDBStorage *mainRefSto = AliCDBManager::Instance()->GetStorage(fgkMainRefStorage);
1572 if(mainRefSto) mainRefSto->QueryCDB(GetCurrentRun());
1573 first = kFALSE;
1574 }
9e080f92 1575
c88ad5db 1576 Log("SHUTTLE", Form("\t\t\t****** run %d - %s: START ******",
2bb7b766 1577 GetCurrentRun(), aDetector->GetName()));
1578
9d733021 1579 for(Int_t iSys=0;iSys<3;iSys++) fFXSCalled[iSys]=kFALSE;
1580
c88ad5db 1581 Log(fCurrentDetector.Data(), "Process - Starting processing");
85a80aa9 1582
be48e3ea 1583 Int_t pid = fork();
1584
1585 if (pid < 0)
1586 {
c88ad5db 1587 Log("SHUTTLE", "Process - ERROR: Forking failed");
be48e3ea 1588 }
1589 else if (pid > 0)
1590 {
1591 // parent
c88ad5db 1592 Log("SHUTTLE", Form("Process - In parent process of %d - %s: Starting monitoring",
be48e3ea 1593 GetCurrentRun(), aDetector->GetName()));
1594
1595 Long_t begin = time(0);
1596
1597 int status; // to be used with waitpid, on purpose an int (not Int_t)!
1598 while (waitpid(pid, &status, WNOHANG) == 0)
1599 {
1600 Long_t expiredTime = time(0) - begin;
1601
1602 if (expiredTime > fConfig->GetPPTimeOut())
1603 {
9827400b 1604 TString tmp;
c88ad5db 1605 tmp.Form("Process - Process of %s time out. "
1606 "Run time: %d seconds. Killing...",
1607 fCurrentDetector.Data(), expiredTime);
9827400b 1608 Log("SHUTTLE", tmp);
1609 Log(fCurrentDetector, tmp);
be48e3ea 1610
1611 kill(pid, 9);
1612
3301427a 1613 UpdateShuttleStatus(AliShuttleStatus::kPPTimeOut);
be48e3ea 1614 hasError = kTRUE;
1615
1616 gSystem->Sleep(1000);
1617 }
1618 else
1619 {
be48e3ea 1620 gSystem->Sleep(1000);
9827400b 1621
1622 TString checkStr;
1623 checkStr.Form("ps -o vsize --pid %d | tail -n 1", pid);
1624 FILE* pipe = gSystem->OpenPipe(checkStr, "r");
1625 if (!pipe)
1626 {
c88ad5db 1627 Log("SHUTTLE", Form("Process - Error: "
1628 "Could not open pipe to %s", checkStr.Data()));
9827400b 1629 continue;
1630 }
1631
1632 char buffer[100];
1633 if (!fgets(buffer, 100, pipe))
1634 {
c88ad5db 1635 Log("SHUTTLE", "Process - Error: ps did not return anything");
9827400b 1636 gSystem->ClosePipe(pipe);
1637 continue;
1638 }
1639 gSystem->ClosePipe(pipe);
1640
1641 //Log("SHUTTLE", Form("ps returned %s", buffer));
1642
1643 Int_t mem = 0;
1644 if ((sscanf(buffer, "%d\n", &mem) != 1) || !mem)
1645 {
c88ad5db 1646 Log("SHUTTLE", "Process - Error: Could not parse output of ps");
9827400b 1647 continue;
1648 }
1649
1650 if (expiredTime % 60 == 0)
ee6f7523 1651 {
c88ad5db 1652 Log("SHUTTLE", Form("Process - %s: Checking process. "
1653 "Run time: %d seconds - Memory consumption: %d KB",
1654 fCurrentDetector.Data(), expiredTime, mem));
ee6f7523 1655 SendAlive();
1656 }
9827400b 1657
1658 if (mem > fConfig->GetPPMaxMem())
1659 {
1660 TString tmp;
c88ad5db 1661 tmp.Form("Process - Process exceeds maximum allowed memory "
1662 "(%d KB > %d KB). Killing...",
9827400b 1663 mem, fConfig->GetPPMaxMem());
1664 Log("SHUTTLE", tmp);
1665 Log(fCurrentDetector, tmp);
1666
1667 kill(pid, 9);
1668
1669 UpdateShuttleStatus(AliShuttleStatus::kPPOutOfMemory);
1670 hasError = kTRUE;
1671
1672 gSystem->Sleep(1000);
1673 }
be48e3ea 1674 }
1675 }
1676
c88ad5db 1677 Log("SHUTTLE", Form("Process - In parent process of %d - %s: Client has terminated.",
be48e3ea 1678 GetCurrentRun(), aDetector->GetName()));
1679
1680 if (WIFEXITED(status))
1681 {
1682 Int_t returnCode = WEXITSTATUS(status);
1683
c88ad5db 1684 Log("SHUTTLE", Form("Process - %s: the return code is %d", fCurrentDetector.Data(),
3301427a 1685 returnCode));
be48e3ea 1686
9827400b 1687 if (returnCode == 0) hasError = kTRUE;
be48e3ea 1688 }
1689 }
1690 else if (pid == 0)
1691 {
1692 // client
c88ad5db 1693 Log("SHUTTLE", Form("Process - In client process of %d - %s", GetCurrentRun(),
1694 aDetector->GetName()));
be48e3ea 1695
c88ad5db 1696 Log("SHUTTLE", Form("Process - Redirecting output to %s log",fCurrentDetector.Data()));
ffa29e93 1697
546242fb 1698 if ((freopen(GetLogFileName(fCurrentDetector), "a", stdout)) == 0)
ffa29e93 1699 {
c88ad5db 1700 Log("SHUTTLE", "Process - Could not freopen stdout");
ffa29e93 1701 }
1702 else
1703 {
1704 fOutputRedirected = kTRUE;
1705 if ((dup2(fileno(stdout), fileno(stderr))) < 0)
c88ad5db 1706 Log("SHUTTLE", "Process - Could not redirect stderr");
ffa29e93 1707
1708 }
1709
5bac2bde 1710 TString wd = gSystem->WorkingDirectory();
675f64cd 1711 TString tmpDir = Form("%s/%s_%d_process", GetShuttleTempDir(),
1712 fCurrentDetector.Data(), GetCurrentRun());
5bac2bde 1713
d524ade6 1714 Int_t result = gSystem->GetPathInfo(tmpDir.Data(), 0, (Long64_t*) 0, 0, 0);
1715 if (!result) // temp dir already exists!
1716 {
1717 Log(fCurrentDetector.Data(),
1718 Form("Process - %s dir already exists! Removing...", tmpDir.Data()));
1719 gSystem->Exec(Form("rm -rf %s",tmpDir.Data()));
675f64cd 1720 }
1721
1722 if (gSystem->mkdir(tmpDir.Data(), 1))
1723 {
1724 Log(fCurrentDetector.Data(), "Process - could not make temp directory!!");
1725 gSystem->Exit(1);
d524ade6 1726 }
1727
1728 if (!gSystem->ChangeDirectory(tmpDir.Data()))
1729 {
1730 Log(fCurrentDetector.Data(), "Process - could not change directory!!");
1731 gSystem->Exit(1);
1732 }
5bac2bde 1733
9827400b 1734 Bool_t success = ProcessCurrentDetector();
5bac2bde 1735
1736 gSystem->ChangeDirectory(wd.Data());
b1d18693 1737
9827400b 1738 if (success) // Preprocessor finished successfully!
1739 {
b1d18693 1740 // remove temporary folder
fbc112e3 1741 // temporary commented (JF)
1742 //gSystem->Exec(Form("rm -rf %s",tmpDir.Data()));
b1d18693 1743
3301427a 1744 // Update time_processed field in FXS DB
1745 if (UpdateTable() == kFALSE)
5bac2bde 1746 Log("SHUTTLE", Form("Process - %s: Could not update FXS databases!",
1747 fCurrentDetector.Data()));
3301427a 1748
1749 // Transfer the data from local storage to main storage (Grid)
1750 UpdateShuttleStatus(AliShuttleStatus::kStoreStarted);
1751 if (StoreOCDB() == kFALSE)
1752 {
c88ad5db 1753 Log("SHUTTLE",
1754 Form("\t\t\t****** run %d - %s: STORAGE ERROR ******",
3301427a 1755 GetCurrentRun(), aDetector->GetName()));
1756 UpdateShuttleStatus(AliShuttleStatus::kStoreError);
9827400b 1757 success = kFALSE;
3301427a 1758 } else {
c88ad5db 1759 Log("SHUTTLE",
1760 Form("\t\t\t****** run %d - %s: DONE ******",
3301427a 1761 GetCurrentRun(), aDetector->GetName()));
1762 UpdateShuttleStatus(AliShuttleStatus::kDone);
9827400b 1763 UpdateShuttleLogbook(fCurrentDetector, "DONE");
3301427a 1764 }
c88ad5db 1765 } else
1766 {
1767 Log("SHUTTLE",
1768 Form("\t\t\t****** run %d - %s: PP ERROR ******",
1769 GetCurrentRun(), aDetector->GetName()));
be48e3ea 1770 }
1771
4b95672b 1772 for (UInt_t iSys=0; iSys<3; iSys++)
1773 {
1774 if (fFXSCalled[iSys]) fFXSlist[iSys].Clear();
1775 }
1776
c88ad5db 1777 Log("SHUTTLE", Form("Process - Client process of %d - %s is exiting now with %d.",
9827400b 1778 GetCurrentRun(), aDetector->GetName(), success));
be48e3ea 1779
1780 // the client exits here
9827400b 1781 gSystem->Exit(success);
be48e3ea 1782
1783 AliError("We should never get here!!!");
1784 }
7bfb2090 1785 }
5164a766 1786
c88ad5db 1787 Log("SHUTTLE", Form("\t\t\t^*^*^*^*^*^*^*^*^*^*^*^* run %d: FINISH ^*^*^*^*^*^*^*^*^*^*^*^*",
2bb7b766 1788 GetCurrentRun()));
1789
1790 //check if shuttle is done for this run, if so update logbook
1791 TObjArray checkEntryArray;
1792 checkEntryArray.SetOwner(1);
9e080f92 1793 TString whereClause = Form("where run=%d", GetCurrentRun());
b0e53b15 1794 if (!QueryShuttleLogbook(whereClause.Data(), checkEntryArray) ||
1795 checkEntryArray.GetEntries() == 0) {
9e080f92 1796 Log("SHUTTLE", Form("Process - Warning: Cannot check status of run %d on Shuttle logbook!",
1797 GetCurrentRun()));
1798 return hasError == kFALSE;
1799 }
b948db8d 1800
9e080f92 1801 AliShuttleLogbookEntry* checkEntry = dynamic_cast<AliShuttleLogbookEntry*>
1802 (checkEntryArray.At(0));
2bb7b766 1803
9e080f92 1804 if (checkEntry)
1805 {
1806 if (checkEntry->IsDone())
be48e3ea 1807 {
9e080f92 1808 Log("SHUTTLE","Process - Shuttle is DONE. Updating logbook");
1809 UpdateShuttleLogbook("shuttle_done");
1810 }
1811 else
1812 {
1813 for (UInt_t iDet=0; iDet<NDetectors(); iDet++)
be48e3ea 1814 {
9e080f92 1815 if (checkEntry->GetDetectorStatus(iDet) == AliShuttleLogbookEntry::kUnprocessed)
be48e3ea 1816 {
9e080f92 1817 AliDebug(2, Form("Run %d: setting %s as \"not first time unprocessed\"",
1818 checkEntry->GetRun(), GetDetName(iDet)));
1819 fFirstUnprocessed[iDet] = kFALSE;
be48e3ea 1820 }
1821 }
2bb7b766 1822 }
1823 }
1824
1825 fLogbookEntry = 0;
85a80aa9 1826
a7160fe9 1827 return hasError == kFALSE;
73abe331 1828}
1829
b948db8d 1830//______________________________________________________________________________________________
9827400b 1831Bool_t AliShuttle::ProcessCurrentDetector()
73abe331 1832{
1833 //
2bb7b766 1834 // Makes data retrieval just for a specific detector (fCurrentDetector).
73abe331 1835 // Threre should be a configuration for this detector.
73abe331 1836
1d172743 1837 Log("SHUTTLE", Form("ProcessCurrentDetector - Retrieving values for %s, run %d",
1838 fCurrentDetector.Data(), GetCurrentRun()));
73abe331 1839
d524ade6 1840 TString wd = gSystem->WorkingDirectory();
1841
2d9019b4 1842 if (!CleanReferenceStorage(fCurrentDetector.Data()))
546242fb 1843 return kFALSE;
d524ade6 1844
1845 gSystem->ChangeDirectory(wd.Data());
1846
1d172743 1847 TMap* dcsMap = new TMap();
3301427a 1848
1849 // call preprocessor
1850 AliPreprocessor* aPreprocessor =
1851 dynamic_cast<AliPreprocessor*> (fPreprocessorMap.GetValue(fCurrentDetector));
1852
1853 aPreprocessor->Initialize(GetCurrentRun(), GetCurrentStartTime(), GetCurrentEndTime());
1854
1855 Bool_t processDCS = aPreprocessor->ProcessDCS();
d477ad88 1856
651fdaab 1857 if (!processDCS)
1858 {
1d172743 1859 Log(fCurrentDetector, "ProcessCurrentDetector -"
1860 " The preprocessor requested to skip the retrieval of DCS values");
651fdaab 1861 }
8b739301 1862 else if (fTestMode & kSkipDCS)
2c15234c 1863 {
1d172743 1864 Log(fCurrentDetector, "ProcessCurrentDetector - In TESTMODE: Skipping DCS processing");
9827400b 1865 }
1866 else if (fTestMode & kErrorDCS)
1867 {
1d172743 1868 Log(fCurrentDetector, "ProcessCurrentDetector - In TESTMODE: Simulating DCS error");
3d8bc902 1869 UpdateShuttleStatus(AliShuttleStatus::kDCSStarted);
9827400b 1870 UpdateShuttleStatus(AliShuttleStatus::kDCSError);
1d172743 1871 delete dcsMap;
9827400b 1872 return kFALSE;
2c15234c 1873 } else {
3301427a 1874
1875 UpdateShuttleStatus(AliShuttleStatus::kDCSStarted);
1876
1d172743 1877 // Query DCS archive
1878 Int_t nServers = fConfig->GetNServers(fCurrentDetector);
a038aa70 1879
1d172743 1880 for (int iServ=0; iServ<nServers; iServ++)
2c15234c 1881 {
1d172743 1882
1883 TString host(fConfig->GetDCSHost(fCurrentDetector, iServ));
1884 Int_t port = fConfig->GetDCSPort(fCurrentDetector, iServ);
542b6cc8 1885 Int_t multiSplit = fConfig->GetMultiSplit(fCurrentDetector, iServ);
1886
1790d4b7 1887 Log(fCurrentDetector, Form("ProcessCurrentDetector -"
1888 " Querying DCS Amanda server %s:%d (%d of %d)",
1889 host.Data(), port, iServ+1, nServers));
1d172743 1890
1891 TMap* aliasMap = 0;
1892 TMap* dpMap = 0;
1893
1894 if (fConfig->GetDCSAliases(fCurrentDetector, iServ)->GetEntries() > 0)
2c15234c 1895 {
1d172743 1896 aliasMap = GetValueSet(host, port,
542b6cc8 1897 fConfig->GetDCSAliases(fCurrentDetector, iServ),
1898 kAlias, multiSplit);
1d172743 1899 if (!aliasMap)
1900 {
1901 Log(fCurrentDetector,
1902 Form("ProcessCurrentDetector -"
675f64cd 1903 " Error retrieving DCS aliases from server %s."
1904 " Sending mail to DCS experts!", host.Data()));
1d172743 1905 UpdateShuttleStatus(AliShuttleStatus::kDCSError);
675f64cd 1906
b5f1c82e 1907 //if (!SendMailToDCS())
1908 // Log("SHUTTLE", Form("ProcessCurrentDetector - Could not send mail to DCS experts!"));
675f64cd 1909
a038aa70 1910 delete dcsMap;
1d172743 1911 return kFALSE;
1912 }
2c15234c 1913 }
a038aa70 1914
1d172743 1915 if (fConfig->GetDCSDataPoints(fCurrentDetector, iServ)->GetEntries() > 0)
a038aa70 1916 {
1d172743 1917 dpMap = GetValueSet(host, port,
542b6cc8 1918 fConfig->GetDCSDataPoints(fCurrentDetector, iServ),
1919 kDP, multiSplit);
1d172743 1920 if (!dpMap)
1921 {
1922 Log(fCurrentDetector,
1923 Form("ProcessCurrentDetector -"
675f64cd 1924 " Error retrieving DCS data points from server %s."
1925 " Sending mail to DCS experts!", host.Data()));
1d172743 1926 UpdateShuttleStatus(AliShuttleStatus::kDCSError);
675f64cd 1927
b5f1c82e 1928 //if (!SendMailToDCS())
1929 // Log("SHUTTLE", Form("ProcessCurrentDetector - Could not send mail to DCS experts!"));
675f64cd 1930
1d172743 1931 if (aliasMap) delete aliasMap;
1932 delete dcsMap;
1933 return kFALSE;
1934 }
a038aa70 1935 }
1d172743 1936
1937 // merge aliasMap and dpMap into dcsMap
1938 if(aliasMap) {
1939 TIter iter(aliasMap);
a038aa70 1940 TObjString* key = 0;
1941 while ((key = (TObjString*) iter.Next()))
1d172743 1942 dcsMap->Add(key, aliasMap->GetValue(key->String()));
1943
1944 aliasMap->SetOwner(kFALSE);
1945 delete aliasMap;
1946 }
1947
1948 if(dpMap) {
1949 TIter iter(dpMap);
1950 TObjString* key = 0;
1951 while ((key = (TObjString*) iter.Next()))
1952 dcsMap->Add(key, dpMap->GetValue(key->String()));
1953
1954 dpMap->SetOwner(kFALSE);
1955 delete dpMap;
a038aa70 1956 }
73abe331 1957 }
1958 }
dc25836b 1959
b1d18693 1960 // save map into file, to help debugging in case of preprocessor error
fbc112e3 1961 TFile* f = TFile::Open("DCSMap.root","recreate");
b1d18693 1962 f->cd();
1963 dcsMap->Write("DCSMap", TObject::kSingleKey);
1964 f->Close();
fbc112e3 1965 delete f;
b1d18693 1966
2bb7b766 1967 // DCS Archive DB processing successful. Call Preprocessor!
85a80aa9 1968 UpdateShuttleStatus(AliShuttleStatus::kPPStarted);
a7160fe9 1969
a038aa70 1970 UInt_t returnValue = aPreprocessor->Process(dcsMap);
b948db8d 1971
3301427a 1972 if (returnValue > 0) // Preprocessor error!
1973 {
c88ad5db 1974 Log(fCurrentDetector, Form("ProcessCurrentDetector - "
1975 "Preprocessor failed. Process returned %d.", returnValue));
cb343cfd 1976 UpdateShuttleStatus(AliShuttleStatus::kPPError);
a038aa70 1977 dcsMap->DeleteAll();
1978 delete dcsMap;
9827400b 1979 return kFALSE;
1980 }
1981
1982 // preprocessor ok!
1983 UpdateShuttleStatus(AliShuttleStatus::kPPDone);
1984 Log(fCurrentDetector, Form("ProcessCurrentDetector - %s preprocessor returned success",
1985 fCurrentDetector.Data()));
b948db8d 1986
a038aa70 1987 dcsMap->DeleteAll();
1988 delete dcsMap;
b948db8d 1989
9827400b 1990 return kTRUE;
2bb7b766 1991}
1992
fbc112e3 1993//______________________________________________________________________________________________
1994void AliShuttle::CountOpenRuns()
1995{
1996 // Query DAQ's Shuttle logbook and sends the number of open runs to ML
1997
1998 // check connection, in case connect
1999 if (!Connect(3))
2000 return;
2001
2002 TString sqlQuery;
2003 sqlQuery = Form("select count(*) from %s where shuttle_done=0", fConfig->GetShuttlelbTable());
2004
2005 TSQLResult* aResult = fServer[3]->Query(sqlQuery);
2006 if (!aResult) {
2007 AliError(Form("Can't execute query <%s>!", sqlQuery.Data()));
2008 return;
2009 }
2010
2011 AliDebug(2,Form("Query = %s", sqlQuery.Data()));
2012
2013 if (aResult->GetRowCount() == 0) {
2014 AliError(Form("No result for query %s received", sqlQuery.Data()));
2015 return;
2016 }
2017
2018 if (aResult->GetFieldCount() != 1) {
2019 AliError(Form("Invalid field count for query %s received", sqlQuery.Data()));
2020 return;
2021 }
2022
2023 TSQLRow* aRow = aResult->Next();
2024 if (!aRow) {
2025 AliError(Form("Could not receive result of query %s", sqlQuery.Data()));
2026 return;
2027 }
2028
2029 TString result(aRow->GetField(0), aRow->GetFieldLength(0));
2030 Int_t count = result.Atoi();
2031
2032 Log("SHUTTLE", Form("%d unprocessed runs", count));
2033
2034 delete aRow;
2035 delete aResult;
2036
2037 TMonaLisaValue mlStatus("SHUTTLE_openruns", count);
2038
2039 TList mlList;
2040 mlList.Add(&mlStatus);
2041
2042 fMonaLisa->SendParameters(&mlList, "__PROCESSINGINFO__");
2043}
2044
2bb7b766 2045//______________________________________________________________________________________________
2046Bool_t AliShuttle::QueryShuttleLogbook(const char* whereClause,
2047 TObjArray& entries)
2048{
9827400b 2049 // Query DAQ's Shuttle logbook and fills detector status object.
2050 // Call QueryRunParameters to query DAQ logbook for run parameters.
2051 //
2bb7b766 2052
fc5a4708 2053 entries.SetOwner(1);
2054
2bb7b766 2055 // check connection, in case connect
fbc112e3 2056 if (!Connect(3)) return kFALSE;
2bb7b766 2057
2058 TString sqlQuery;
441b0e9c 2059 sqlQuery = Form("select * from %s %s order by run", fConfig->GetShuttlelbTable(), whereClause);
2bb7b766 2060
be48e3ea 2061 TSQLResult* aResult = fServer[3]->Query(sqlQuery);
2bb7b766 2062 if (!aResult) {
2063 AliError(Form("Can't execute query <%s>!", sqlQuery.Data()));
2064 return kFALSE;
2065 }
2066
fc5a4708 2067 AliDebug(2,Form("Query = %s", sqlQuery.Data()));
2068
2bb7b766 2069 if(aResult->GetRowCount() == 0) {
c88ad5db 2070 Log("SHUTTLE", "No entries in Shuttle Logbook match request");
9827400b 2071 delete aResult;
2072 return kTRUE;
2bb7b766 2073 }
2074
2075 // TODO Check field count!
db99d43e 2076 const UInt_t nCols = 23;
2bb7b766 2077 if (aResult->GetFieldCount() != (Int_t) nCols) {
c88ad5db 2078 Log("SHUTTLE", "Invalid SQL result field number!");
2bb7b766 2079 delete aResult;
2080 return kFALSE;
2081 }
2082
2bb7b766 2083 TSQLRow* aRow;
2084 while ((aRow = aResult->Next())) {
2085 TString runString(aRow->GetField(0), aRow->GetFieldLength(0));
2086 Int_t run = runString.Atoi();
2087
eba76848 2088 AliShuttleLogbookEntry *entry = QueryRunParameters(run);
2089 if (!entry)
2090 continue;
2bb7b766 2091
2092 // loop on detectors
eba76848 2093 for(UInt_t ii = 0; ii < nCols; ii++)
2094 entry->SetDetectorStatus(aResult->GetFieldName(ii), aRow->GetField(ii));
2bb7b766 2095
eba76848 2096 entries.AddLast(entry);
2bb7b766 2097 delete aRow;
2098 }
2099
2bb7b766 2100 delete aResult;
2101 return kTRUE;
2102}
2103
2104//______________________________________________________________________________________________
eba76848 2105AliShuttleLogbookEntry* AliShuttle::QueryRunParameters(Int_t run)
2bb7b766 2106{
eba76848 2107 //
2108 // Retrieve run parameters written in the DAQ logbook and sets them into AliShuttleLogbookEntry object
2109 //
2bb7b766 2110
2111 // check connection, in case connect
be48e3ea 2112 if (!Connect(3))
eba76848 2113 return 0;
2bb7b766 2114
2115 TString sqlQuery;
2c15234c 2116 sqlQuery.Form("select * from %s where run=%d", fConfig->GetDAQlbTable(), run);
2bb7b766 2117
be48e3ea 2118 TSQLResult* aResult = fServer[3]->Query(sqlQuery);
2bb7b766 2119 if (!aResult) {
c88ad5db 2120 Log("SHUTTLE", Form("Can't execute query <%s>!", sqlQuery.Data()));
eba76848 2121 return 0;
2bb7b766 2122 }
2123
eba76848 2124 if (aResult->GetRowCount() == 0) {
2bb7b766 2125 Log("SHUTTLE", Form("QueryRunParameters - No entry in DAQ Logbook for run %d. Skipping", run));
2126 delete aResult;
eba76848 2127 return 0;
2bb7b766 2128 }
2129
eba76848 2130 if (aResult->GetRowCount() > 1) {
c88ad5db 2131 Log("SHUTTLE", Form("QueryRunParameters - UNEXPECTED: "
2132 "more than one entry in DAQ Logbook for run %d!", run));
2bb7b766 2133 delete aResult;
eba76848 2134 return 0;
2bb7b766 2135 }
2136
eba76848 2137 TSQLRow* aRow = aResult->Next();
2138 if (!aRow)
2139 {
c88ad5db 2140 Log("SHUTTLE", Form("QueryRunParameters - Could not retrieve row for run %d. Skipping", run));
eba76848 2141 delete aResult;
2142 return 0;
2143 }
2bb7b766 2144
eba76848 2145 AliShuttleLogbookEntry* entry = new AliShuttleLogbookEntry(run);
2bb7b766 2146
eba76848 2147 for (Int_t ii = 0; ii < aResult->GetFieldCount(); ii++)
2148 entry->SetRunParameter(aResult->GetFieldName(ii), aRow->GetField(ii));
2bb7b766 2149
eba76848 2150 UInt_t startTime = entry->GetStartTime();
2151 UInt_t endTime = entry->GetEndTime();
2152
1abfbb60 2153// if (!startTime || !endTime || startTime > endTime)
2154// {
2155// Log("SHUTTLE",
2156// Form("QueryRunParameters - Invalid parameters for Run %d: startTime = %d, endTime = %d. Skipping!",
2157// run, startTime, endTime));
2158//
2159// Log("SHUTTLE", Form("Marking SHUTTLE done for run %d", run));
2160// fLogbookEntry = entry;
2161// if (!UpdateShuttleLogbook("shuttle_done"))
2162// {
2163// AliError(Form("Could not update logbook for run %d !", run));
2164// }
2165// fLogbookEntry = 0;
2166//
2167// delete entry;
2168// delete aRow;
2169// delete aResult;
2170// return 0;
2171// }
2172
2173 if (!startTime)
b0e53b15 2174 {
2175 Log("SHUTTLE",
1abfbb60 2176 Form("QueryRunParameters - Invalid parameters for Run %d: "
2177 "startTime = %d, endTime = %d. Skipping!",
2178 run, startTime, endTime));
b0e53b15 2179
2180 Log("SHUTTLE", Form("Marking SHUTTLE done for run %d", run));
2181 fLogbookEntry = entry;
ee6f7523 2182 if (!UpdateShuttleLogbook("shuttle_ignored"))
b0e53b15 2183 {
2184 AliError(Form("Could not update logbook for run %d !", run));
2185 }
2186 fLogbookEntry = 0;
2187
2188 delete entry;
2189 delete aRow;
2190 delete aResult;
2191 return 0;
2192 }
2193
1abfbb60 2194 if (startTime && !endTime)
2195 {
2196 // TODO Here we don't mark SHUTTLE done, because this may mean
2197 //the run is still ongoing!!
2198 Log("SHUTTLE",
2199 Form("QueryRunParameters - Invalid parameters for Run %d: "
2200 "startTime = %d, endTime = %d. Skipping (Shuttle won't be marked as DONE)!",
2201 run, startTime, endTime));
2202
2203 //Log("SHUTTLE", Form("Marking SHUTTLE done for run %d", run));
2204 //fLogbookEntry = entry;
2205 //if (!UpdateShuttleLogbook("shuttle_done"))
2206 //{
2207 // AliError(Form("Could not update logbook for run %d !", run));
2208 //}
2209 //fLogbookEntry = 0;
2210
2211 delete entry;
2212 delete aRow;
2213 delete aResult;
2214 return 0;
2215 }
2216
2217 if (startTime && endTime && (startTime > endTime))
2218 {
2219 Log("SHUTTLE",
2220 Form("QueryRunParameters - Invalid parameters for Run %d: "
2221 "startTime = %d, endTime = %d. Skipping!",
2222 run, startTime, endTime));
2223
2224 Log("SHUTTLE", Form("Marking SHUTTLE done for run %d", run));
2225 fLogbookEntry = entry;
ee6f7523 2226 if (!UpdateShuttleLogbook("shuttle_ignored"))
1abfbb60 2227 {
2228 AliError(Form("Could not update logbook for run %d !", run));
2229 }
2230 fLogbookEntry = 0;
2231
2232 delete entry;
2233 delete aRow;
2234 delete aResult;
2235 return 0;
2236 }
2237
b0e53b15 2238 TString totEventsStr = entry->GetRunParameter("totalEvents");
2239 Int_t totEvents = totEventsStr.Atoi();
2240 if (totEvents < 1)
2241 {
eba76848 2242 Log("SHUTTLE",
b0e53b15 2243 Form("QueryRunParameters - Run %d has 0 events - Skipping!", run));
2244
2245 Log("SHUTTLE", Form("Marking SHUTTLE done for run %d", run));
2246 fLogbookEntry = entry;
97e3c167 2247 if (!UpdateShuttleLogbook("shuttle_ignored"))
b0e53b15 2248 {
2249 AliError(Form("Could not update logbook for run %d !", run));
2250 }
2251 fLogbookEntry = 0;
2252
eba76848 2253 delete entry;
2bb7b766 2254 delete aRow;
eba76848 2255 delete aResult;
2256 return 0;
2bb7b766 2257 }
2258
eba76848 2259 delete aRow;
2bb7b766 2260 delete aResult;
eba76848 2261
2262 return entry;
2bb7b766 2263}
2264
a038aa70 2265//______________________________________________________________________________________________
2266TMap* AliShuttle::GetValueSet(const char* host, Int_t port, const TSeqCollection* entries,
542b6cc8 2267 DCSType type, Int_t multiSplit)
a038aa70 2268{
2269 // Retrieve all "entry" data points from the DCS server
2270 // host, port: TSocket connection parameters
2271 // entries: list of name of the alias or data point
2272 // type: kAlias or kDP
2273 // returns TMap of values, 0 when failure
542b6cc8 2274
2275 AliDCSClient client(host, port, fTimeout, fRetries, multiSplit);
b41b252a 2276
a038aa70 2277 TMap* result = 0;
b41b252a 2278 if (type == kAlias)
a038aa70 2279 {
b41b252a 2280 result = client.GetAliasValues(entries, GetCurrentStartTime(),
2281 GetCurrentEndTime());
2282 }
2283 else if (type == kDP)
2284 {
2285 result = client.GetDPValues(entries, GetCurrentStartTime(),
2286 GetCurrentEndTime());
2287 }
a038aa70 2288
b41b252a 2289 if (result == 0)
2290 {
2291 Log(fCurrentDetector.Data(), Form("GetValueSet - Can't get entries! Reason: %s",
1790d4b7 2292 client.GetErrorString(client.GetResultErrorCode())));
2293 if (client.GetResultErrorCode() == AliDCSClient::fgkServerError)
2294 Log(fCurrentDetector.Data(), Form("GetValueSet - Server error code: %s",
2295 client.GetServerError().Data()));
a038aa70 2296
b41b252a 2297 return 0;
a038aa70 2298 }
b41b252a 2299
a038aa70 2300 return result;
2301}
b41b252a 2302
b948db8d 2303//______________________________________________________________________________________________
57f50b3c 2304const char* AliShuttle::GetFile(Int_t system, const char* detector,
2305 const char* id, const char* source)
b948db8d 2306{
9827400b 2307 // Get calibration file from file exchange servers
2308 // First queris the FXS database for the file name, using the run, detector, id and source info
2309 // then calls RetrieveFile(filename) for actual copy to local disk
2310 // run: current run being processed (given by Logbook entry fLogbookEntry)
2311 // detector: the Preprocessor name
2312 // id: provided as a parameter by the Preprocessor
2313 // source: provided by the Preprocessor through GetFileSources function
2314
2315 // check if test mode should simulate a FXS error
2316 if (fTestMode & kErrorFXSFiles)
2317 {
2318 Log(detector, Form("GetFile - In TESTMODE - Simulating error while connecting to %s FXS", GetSystemName(system)));
2319 return 0;
2320 }
2321
57f50b3c 2322 // check connection, in case connect
9d733021 2323 if (!Connect(system))
eba76848 2324 {
9d733021 2325 Log(detector, Form("GetFile - Couldn't connect to %s FXS database", GetSystemName(system)));
57f50b3c 2326 return 0;
2327 }
2328
2329 // Query preparation
9d733021 2330 TString sourceName(source);
d386d623 2331 Int_t nFields = 3;
2332 TString sqlQueryStart = Form("select filePath,size,fileChecksum from %s where",
2333 fConfig->GetFXSdbTable(system));
2334 TString whereClause = Form("run=%d and detector=\"%s\" and fileId=\"%s\"",
2335 GetCurrentRun(), detector, id);
2336
9d733021 2337 if (system == kDAQ)
2338 {
d386d623 2339 whereClause += Form(" and DAQsource=\"%s\"", source);
57f50b3c 2340 }
9d733021 2341 else if (system == kDCS)
eba76848 2342 {
9d733021 2343 sourceName="none";
57f50b3c 2344 }
9d733021 2345 else if (system == kHLT)
9e080f92 2346 {
d386d623 2347 whereClause += Form(" and DDLnumbers=\"%s\"", source);
9d733021 2348 nFields = 3;
9e080f92 2349 }
2350
9e080f92 2351 TString sqlQuery = Form("%s %s", sqlQueryStart.Data(), whereClause.Data());
2352
2353 AliDebug(2, Form("SQL query: \n%s",sqlQuery.Data()));
2354
2355 // Query execution
2356 TSQLResult* aResult = 0;
9d733021 2357 aResult = dynamic_cast<TSQLResult*> (fServer[system]->Query(sqlQuery));
9e080f92 2358 if (!aResult) {
9d733021 2359 Log(detector, Form("GetFileName - Can't execute SQL query to %s database for: id = %s, source = %s",
2360 GetSystemName(system), id, sourceName.Data()));
9e080f92 2361 return 0;
2362 }
2363
2364 if(aResult->GetRowCount() == 0)
2365 {
2366 Log(detector,
9d733021 2367 Form("GetFileName - No entry in %s FXS db for: id = %s, source = %s",
2368 GetSystemName(system), id, sourceName.Data()));
9e080f92 2369 delete aResult;
2370 return 0;
2371 }
2bb7b766 2372
9e080f92 2373 if (aResult->GetRowCount() > 1) {
2374 Log(detector,
9d733021 2375 Form("GetFileName - More than one entry in %s FXS db for: id = %s, source = %s",
2376 GetSystemName(system), id, sourceName.Data()));
9e080f92 2377 delete aResult;
2378 return 0;
2379 }
2380
9d733021 2381 if (aResult->GetFieldCount() != nFields) {
9e080f92 2382 Log(detector,
9d733021 2383 Form("GetFileName - Wrong field count in %s FXS db for: id = %s, source = %s",
2384 GetSystemName(system), id, sourceName.Data()));
9e080f92 2385 delete aResult;
2386 return 0;
2387 }
2388
2389 TSQLRow* aRow = dynamic_cast<TSQLRow*> (aResult->Next());
2390
2391 if (!aRow){
9d733021 2392 Log(detector, Form("GetFileName - Empty set result in %s FXS db from query: id = %s, source = %s",
2393 GetSystemName(system), id, sourceName.Data()));
9e080f92 2394 delete aResult;
2395 return 0;
2396 }
2397
2398 TString filePath(aRow->GetField(0), aRow->GetFieldLength(0));
2399 TString fileSize(aRow->GetField(1), aRow->GetFieldLength(1));
d386d623 2400 TString fileChecksum(aRow->GetField(2), aRow->GetFieldLength(2));
9e080f92 2401
2402 delete aResult;
2403 delete aRow;
2404
d386d623 2405 AliDebug(2, Form("filePath = %s; size = %s, fileChecksum = %s",
2406 filePath.Data(), fileSize.Data(), fileChecksum.Data()));
9e080f92 2407
9e080f92 2408 // retrieved file is renamed to make it unique
675f64cd 2409 TString localFileName = Form("%s/%s_%d_process/%s_%s_%d_%s_%s.shuttle",
2410 GetShuttleTempDir(), detector, GetCurrentRun(),
d524ade6 2411 GetSystemName(system), detector, GetCurrentRun(),
2412 id, sourceName.Data());
9d733021 2413
9e080f92 2414
9d733021 2415 // file retrieval from FXS
4b95672b 2416 UInt_t nRetries = 0;
2417 UInt_t maxRetries = 3;
2418 Bool_t result = kFALSE;
2419
2420 // copy!! if successful TSystem::Exec returns 0
2421 while(nRetries++ < maxRetries) {
2422 AliDebug(2, Form("Trying to copy file. Retry # %d", nRetries));
2423 result = RetrieveFile(system, filePath.Data(), localFileName.Data());
2424 if(!result)
2425 {
2426 Log(detector, Form("GetFileName - Copy of file %s from %s FXS failed",
9d733021 2427 filePath.Data(), GetSystemName(system)));
4b95672b 2428 continue;
4f0749a8 2429 }
9e080f92 2430
d386d623 2431 if (fileChecksum.Length()>0)
4b95672b 2432 {
2433 // compare md5sum of local file with the one stored in the FXS DB
d524ade6 2434 Int_t md5Comp = gSystem->Exec(Form("md5sum %s |grep %s 2>&1 > /dev/null",
2435 localFileName.Data(), fileChecksum.Data()));
9e080f92 2436
4b95672b 2437 if (md5Comp != 0)
2438 {
2439 Log(detector, Form("GetFileName - md5sum of file %s does not match with local copy!",
2440 filePath.Data()));
2441 result = kFALSE;
2442 continue;
2443 }
d386d623 2444 } else {
2445 Log(fCurrentDetector, Form("GetFile - md5sum of file %s not set in %s database, skipping comparison",
2446 filePath.Data(), GetSystemName(system)));
9d733021 2447 }
4b95672b 2448 if (result) break;
9e080f92 2449 }
2450
4b95672b 2451 if(!result) return 0;
2452
9d733021 2453 fFXSCalled[system]=kTRUE;
2454 TObjString *fileParams = new TObjString(Form("%s#!?!#%s", id, sourceName.Data()));
2455 fFXSlist[system].Add(fileParams);
9e080f92 2456
675f64cd 2457 static TString staticLocalFileName;
2458 staticLocalFileName.Form("%s", localFileName.Data());
2459
c88ad5db 2460 Log(fCurrentDetector, Form("GetFile - Retrieved file with id %s and "
2461 "source %s from %s to %s", id, source,
d524ade6 2462 GetSystemName(system), localFileName.Data()));
675f64cd 2463
d524ade6 2464 return staticLocalFileName.Data();
2bb7b766 2465}
2466
2467//______________________________________________________________________________________________
9d733021 2468Bool_t AliShuttle::RetrieveFile(UInt_t system, const char* fxsFileName, const char* localFileName)
9e080f92 2469{
9827400b 2470 //
2471 // Copies file from FXS to local Shuttle machine
2472 //
2bb7b766 2473
9e080f92 2474 // check temp directory: trying to cd to temp; if it does not exist, create it
d524ade6 2475 AliDebug(2, Form("Copy file %s from %s FXS into %s",
2476 GetSystemName(system), fxsFileName, localFileName));
2477
2478 TString tmpDir(localFileName);
2479
2480 tmpDir = tmpDir(0,tmpDir.Last('/'));
9e080f92 2481
d524ade6 2482 Int_t noDir = gSystem->GetPathInfo(tmpDir.Data(), 0, (Long64_t*) 0, 0, 0);
2483 if (noDir) // temp dir does not exists!
2484 {
2485 if (gSystem->mkdir(tmpDir.Data(), 1))
2486 {
2487 Log(fCurrentDetector.Data(), "RetrieveFile - could not make temp directory!!");
9e080f92 2488 return kFALSE;
2489 }
9e080f92 2490 }
2491
9d733021 2492 TString baseFXSFolder;
2493 if (system == kDAQ)
2494 {
2495 baseFXSFolder = "FES/";
2496 }
2497 else if (system == kDCS)
2498 {
2499 baseFXSFolder = "";
2500 }
2501 else if (system == kHLT)
2502 {
42fde080 2503 baseFXSFolder = "/opt/FXS/";
9d733021 2504 }
2505
2506
d524ade6 2507 TString command = Form("scp -oPort=%d -2 %s@%s:%s%s %s",
9d733021 2508 fConfig->GetFXSPort(system),
2509 fConfig->GetFXSUser(system),
2510 fConfig->GetFXSHost(system),
2511 baseFXSFolder.Data(),
2512 fxsFileName,
9e080f92 2513 localFileName);
2514
2515 AliDebug(2, Form("%s",command.Data()));
2516
4b95672b 2517 Bool_t result = (gSystem->Exec(command.Data()) == 0);
9e080f92 2518
4b95672b 2519 return result;
9e080f92 2520}
2521
2522//______________________________________________________________________________________________
9d733021 2523TList* AliShuttle::GetFileSources(Int_t system, const char* detector, const char* id)
2524{
9827400b 2525 //
2526 // Get sources producing the condition file Id from file exchange servers
4a33bdd9 2527 // if id is NULL all sources are returned (distinct)
9827400b 2528 //
1bcd28db 2529
8884d71e 2530 if (id)
2531 {
2532 Log(detector, Form("GetFileSources - Querying %s FXS for files with id %s produced by %s", GetSystemName(system), id, detector));
2533 } else {
2534 Log(detector, Form("GetFileSources - Querying %s FXS for files produced by %s", GetSystemName(system), detector));
2535 }
9827400b 2536
2537 // check if test mode should simulate a FXS error
2538 if (fTestMode & kErrorFXSSources)
2539 {
2540 Log(detector, Form("GetFileSources - In TESTMODE - Simulating error while connecting to %s FXS", GetSystemName(system)));
2541 return 0;
2542 }
2543
9d733021 2544 if (system == kDCS)
2545 {
c88ad5db 2546 Log(detector, "GetFileSources - WARNING: DCS system has only one source of data!");
6297b37d 2547 TList *list = new TList();
2548 list->SetOwner(1);
2549 list->Add(new TObjString(" "));
2550 return list;
9d733021 2551 }
9e080f92 2552
2553 // check connection, in case connect
9d733021 2554 if (!Connect(system))
2555 {
4a33bdd9 2556 Log(detector, Form("GetFileSources - Couldn't connect to %s FXS database", GetSystemName(system)));
9d733021 2557 return NULL;
9e080f92 2558 }
2559
9d733021 2560 TString sourceName = 0;
2561 if (system == kDAQ)
2562 {
2563 sourceName = "DAQsource";
2564 } else if (system == kHLT)
2565 {
2566 sourceName = "DDLnumbers";
2567 }
2568
4a33bdd9 2569 TString sqlQueryStart = Form("select distinct %s from %s where", sourceName.Data(), fConfig->GetFXSdbTable(system));
2570 TString whereClause = Form("run=%d and detector=\"%s\"",
2571 GetCurrentRun(), detector);
2572 if (id)
2573 whereClause += Form(" and fileId=\"%s\"", id);
9e080f92 2574 TString sqlQuery = Form("%s %s", sqlQueryStart.Data(), whereClause.Data());
2575
2576 AliDebug(2, Form("SQL query: \n%s",sqlQuery.Data()));
2577
2578 // Query execution
2579 TSQLResult* aResult;
9d733021 2580 aResult = fServer[system]->Query(sqlQuery);
9e080f92 2581 if (!aResult) {
9d733021 2582 Log(detector, Form("GetFileSources - Can't execute SQL query to %s database for id: %s",
2583 GetSystemName(system), id));
9e080f92 2584 return 0;
2585 }
2586
86aa42c3 2587 TList *list = new TList();
2588 list->SetOwner(1);
2589
9d733021 2590 if (aResult->GetRowCount() == 0)
2591 {
9e080f92 2592 Log(detector,
9d733021 2593 Form("GetFileSources - No entry in %s FXS table for id: %s", GetSystemName(system), id));
9e080f92 2594 delete aResult;
86aa42c3 2595 return list;
9e080f92 2596 }
2597
1bcd28db 2598 Log(detector, Form("GetFileSources - Found %d sources", aResult->GetRowCount()));
9e080f92 2599
1bcd28db 2600 TSQLRow* aRow;
9d733021 2601 while ((aRow = aResult->Next()))
2602 {
9e080f92 2603
9d733021 2604 TString source(aRow->GetField(0), aRow->GetFieldLength(0));
2605 AliDebug(2, Form("%s = %s", sourceName.Data(), source.Data()));
2606 list->Add(new TObjString(source));
9e080f92 2607 delete aRow;
2608 }
9d733021 2609
9e080f92 2610 delete aResult;
2611
2612 return list;
2bb7b766 2613}
2614
4a33bdd9 2615//______________________________________________________________________________________________
2616TList* AliShuttle::GetFileIDs(Int_t system, const char* detector, const char* source)
2617{
2618 //
2619 // Get all ids of condition files produced by a given source from file exchange servers
2620 //
2621
1bcd28db 2622 Log(detector, Form("GetFileIDs - Retrieving ids with source %s with %s", source, GetSystemName(system)));
2623
4a33bdd9 2624 // check if test mode should simulate a FXS error
2625 if (fTestMode & kErrorFXSSources)
2626 {
2627 Log(detector, Form("GetFileIDs - In TESTMODE - Simulating error while connecting to %s FXS", GetSystemName(system)));
2628 return 0;
2629 }
2630
2631 // check connection, in case connect
2632 if (!Connect(system))
2633 {
2634 Log(detector, Form("GetFileIDs - Couldn't connect to %s FXS database", GetSystemName(system)));
2635 return NULL;
2636 }
2637
2638 TString sourceName = 0;
2639 if (system == kDAQ)
2640 {
2641 sourceName = "DAQsource";
2642 } else if (system == kHLT)
2643 {
2644 sourceName = "DDLnumbers";
2645 }
2646
2647 TString sqlQueryStart = Form("select fileId from %s where", fConfig->GetFXSdbTable(system));
2648 TString whereClause = Form("run=%d and detector=\"%s\"",
2649 GetCurrentRun(), detector);
2650 if (sourceName.Length() > 0 && source)
2651 whereClause += Form(" and %s=\"%s\"", sourceName.Data(), source);
2652 TString sqlQuery = Form("%s %s", sqlQueryStart.Data(), whereClause.Data());
2653
2654 AliDebug(2, Form("SQL query: \n%s",sqlQuery.Data()));
2655
2656 // Query execution
2657 TSQLResult* aResult;
2658 aResult = fServer[system]->Query(sqlQuery);
2659 if (!aResult) {
2660 Log(detector, Form("GetFileIDs - Can't execute SQL query to %s database for source: %s",
2661 GetSystemName(system), source));
2662 return 0;
2663 }
2664
2665 TList *list = new TList();
2666 list->SetOwner(1);
2667
2668 if (aResult->GetRowCount() == 0)
2669 {
2670 Log(detector,
2671 Form("GetFileIDs - No entry in %s FXS table for source: %s", GetSystemName(system), source));
2672 delete aResult;
2673 return list;
2674 }
2675
1bcd28db 2676 Log(detector, Form("GetFileIDs - Found %d ids", aResult->GetRowCount()));
2677
4a33bdd9 2678 TSQLRow* aRow;
2679
2680 while ((aRow = aResult->Next()))
2681 {
2682
2683 TString id(aRow->GetField(0), aRow->GetFieldLength(0));
2684 AliDebug(2, Form("fileId = %s", id.Data()));
2685 list->Add(new TObjString(id));
2686 delete aRow;
2687 }
2688
2689 delete aResult;
2690
2691 return list;
2692}
2693
2bb7b766 2694//______________________________________________________________________________________________
9d733021 2695Bool_t AliShuttle::Connect(Int_t system)
2bb7b766 2696{
9827400b 2697 // Connect to MySQL Server of the system's FXS MySQL databases
2698 // DAQ Logbook, Shuttle Logbook and DAQ FXS db are on the same host
2699 //
57f50b3c 2700
9d733021 2701 // check connection: if already connected return
2702 if(fServer[system] && fServer[system]->IsConnected()) return kTRUE;
57f50b3c 2703
9d733021 2704 TString dbHost, dbUser, dbPass, dbName;
57f50b3c 2705
9d733021 2706 if (system < 3) // FXS db servers
2707 {
2708 dbHost = Form("mysql://%s:%d", fConfig->GetFXSdbHost(system), fConfig->GetFXSdbPort(system));
2709 dbUser = fConfig->GetFXSdbUser(system);
2710 dbPass = fConfig->GetFXSdbPass(system);
2711 dbName = fConfig->GetFXSdbName(system);
2712 } else { // Run & Shuttle logbook servers
2713 // TODO Will the Shuttle logbook server be the same as the Run logbook server ???
2714 dbHost = Form("mysql://%s:%d", fConfig->GetDAQlbHost(), fConfig->GetDAQlbPort());
2715 dbUser = fConfig->GetDAQlbUser();
2716 dbPass = fConfig->GetDAQlbPass();
2717 dbName = fConfig->GetDAQlbDB();
2718 }
57f50b3c 2719
9d733021 2720 fServer[system] = TSQLServer::Connect(dbHost.Data(), dbUser.Data(), dbPass.Data());
2721 if (!fServer[system] || !fServer[system]->IsConnected()) {
2722 if(system < 3)
2723 {
2724 AliError(Form("Can't establish connection to FXS database for %s",
2725 AliShuttleInterface::GetSystemName(system)));
2726 } else {
2727 AliError("Can't establish connection to Run logbook.");
57f50b3c 2728 }
9d733021 2729 if(fServer[system]) delete fServer[system];
2730 return kFALSE;
2bb7b766 2731 }
57f50b3c 2732
9d733021 2733 // Get tables
2734 TSQLResult* aResult=0;
2735 switch(system){
2736 case kDAQ:
2737 aResult = fServer[kDAQ]->GetTables(dbName.Data());
2738 break;
2739 case kDCS:
2740 aResult = fServer[kDCS]->GetTables(dbName.Data());
2741 break;
2742 case kHLT:
2743 aResult = fServer[kHLT]->GetTables(dbName.Data());
2744 break;
2745 default:
2746 aResult = fServer[3]->GetTables(dbName.Data());
2747 break;
2748 }
2749
2750 delete aResult;
2bb7b766 2751 return kTRUE;
2752}
57f50b3c 2753
9e080f92 2754//______________________________________________________________________________________________
9d733021 2755Bool_t AliShuttle::UpdateTable()
9e080f92 2756{
9827400b 2757 //
2758 // Update FXS table filling time_processed field in all rows corresponding to current run and detector
2759 //
9e080f92 2760
9d733021 2761 Bool_t result = kTRUE;
9e080f92 2762
9d733021 2763 for (UInt_t system=0; system<3; system++)
2764 {
2765 if(!fFXSCalled[system]) continue;
9e080f92 2766
9d733021 2767 // check connection, in case connect
2768 if (!Connect(system))
2769 {
2770 Log(fCurrentDetector, Form("UpdateTable - Couldn't connect to %s FXS database", GetSystemName(system)));
2771 result = kFALSE;
2772 continue;
9e080f92 2773 }
9e080f92 2774
9d733021 2775 TTimeStamp now; // now
2776
2777 // Loop on FXS list entries
2778 TIter iter(&fFXSlist[system]);
2779 TObjString *aFXSentry=0;
2780 while ((aFXSentry = dynamic_cast<TObjString*> (iter.Next())))
2781 {
2782 TString aFXSentrystr = aFXSentry->String();
2783 TObjArray *aFXSarray = aFXSentrystr.Tokenize("#!?!#");
2784 if (!aFXSarray || aFXSarray->GetEntries() != 2 )
2785 {
2786 Log(fCurrentDetector, Form("UpdateTable - error updating %s FXS entry. Check string: <%s>",
2787 GetSystemName(system), aFXSentrystr.Data()));
2788 if(aFXSarray) delete aFXSarray;
2789 result = kFALSE;
2790 continue;
2791 }
2792 const char* fileId = ((TObjString*) aFXSarray->At(0))->GetName();
2793 const char* source = ((TObjString*) aFXSarray->At(1))->GetName();
2794
2795 TString whereClause;
2796 if (system == kDAQ)
2797 {
2798 whereClause = Form("where run=%d and detector=\"%s\" and fileId=\"%s\" and DAQsource=\"%s\";",
2799 GetCurrentRun(), fCurrentDetector.Data(), fileId, source);
2800 }
2801 else if (system == kDCS)
2802 {
2803 whereClause = Form("where run=%d and detector=\"%s\" and fileId=\"%s\";",
2804 GetCurrentRun(), fCurrentDetector.Data(), fileId);
2805 }
2806 else if (system == kHLT)
2807 {
2808 whereClause = Form("where run=%d and detector=\"%s\" and fileId=\"%s\" and DDLnumbers=\"%s\";",
2809 GetCurrentRun(), fCurrentDetector.Data(), fileId, source);
2810 }
2811
2812 delete aFXSarray;
9e080f92 2813
9d733021 2814 TString sqlQuery = Form("update %s set time_processed=%d %s", fConfig->GetFXSdbTable(system),
2815 now.GetSec(), whereClause.Data());
9e080f92 2816
9d733021 2817 AliDebug(2, Form("SQL query: \n%s",sqlQuery.Data()));
9e080f92 2818
9d733021 2819 // Query execution
2820 TSQLResult* aResult;
2821 aResult = dynamic_cast<TSQLResult*> (fServer[system]->Query(sqlQuery));
2822 if (!aResult)
2823 {
2824 Log(fCurrentDetector, Form("UpdateTable - %s db: can't execute SQL query <%s>",
2825 GetSystemName(system), sqlQuery.Data()));
2826 result = kFALSE;
2827 continue;
2828 }
2829 delete aResult;
9e080f92 2830 }
9e080f92 2831 }
2832
9d733021 2833 return result;
9e080f92 2834}
57f50b3c 2835
3301427a 2836//______________________________________________________________________________________________
2837Bool_t AliShuttle::UpdateTableFailCase()
2838{
9827400b 2839 // Update FXS table filling time_processed field in all rows corresponding to current run and detector
2840 // this is called in case the preprocessor is declared failed for the current run, because
2841 // the fields are updated only in case of success
3301427a 2842
2843 Bool_t result = kTRUE;
2844
2845 for (UInt_t system=0; system<3; system++)
2846 {
2847 // check connection, in case connect
2848 if (!Connect(system))
2849 {
2850 Log(fCurrentDetector, Form("UpdateTableFailCase - Couldn't connect to %s FXS database",
2851 GetSystemName(system)));
2852 result = kFALSE;
2853 continue;
2854 }
2855
2856 TTimeStamp now; // now
2857
2858 // Loop on FXS list entries
2859
2860 TString whereClause = Form("where run=%d and detector=\"%s\";",
2861 GetCurrentRun(), fCurrentDetector.Data());
2862
2863
2864 TString sqlQuery = Form("update %s set time_processed=%d %s", fConfig->GetFXSdbTable(system),
2865 now.GetSec(), whereClause.Data());
2866
2867 AliDebug(2, Form("SQL query: \n%s",sqlQuery.Data()));
2868
2869 // Query execution
2870 TSQLResult* aResult;
2871 aResult = dynamic_cast<TSQLResult*> (fServer[system]->Query(sqlQuery));
2872 if (!aResult)
2873 {
2874 Log(fCurrentDetector, Form("UpdateTableFailCase - %s db: can't execute SQL query <%s>",
2875 GetSystemName(system), sqlQuery.Data()));
2876 result = kFALSE;
2877 continue;
2878 }
2879 delete aResult;
2880 }
2881
2882 return result;
2883}
2884
2bb7b766 2885//______________________________________________________________________________________________
2886Bool_t AliShuttle::UpdateShuttleLogbook(const char* detector, const char* status)
2887{
e7f62f16 2888 //
2889 // Update Shuttle logbook filling detector or shuttle_done column
2890 // ex. of usage: UpdateShuttleLogbook("PHOS", "DONE") or UpdateShuttleLogbook("shuttle_done")
2891 //
57f50b3c 2892
2bb7b766 2893 // check connection, in case connect
be48e3ea 2894 if(!Connect(3)){
2bb7b766 2895 Log("SHUTTLE", "UpdateShuttleLogbook - Couldn't connect to DAQ Logbook.");
2896 return kFALSE;
57f50b3c 2897 }
2898
2bb7b766 2899 TString detName(detector);
2900 TString setClause;
ee6f7523 2901 if (detName == "shuttle_done" || detName == "shuttle_ignored")
e7f62f16 2902 {
2bb7b766 2903 setClause = "set shuttle_done=1";
e7f62f16 2904
ee6f7523 2905 if (detName == "shuttle_done")
b0e53b15 2906 {
2907 // Send the information to ML
2908 TMonaLisaText mlStatus("SHUTTLE_status", "Done");
e7f62f16 2909
b0e53b15 2910 TList mlList;
2911 mlList.Add(&mlStatus);
2912
ee6f7523 2913 TString mlID;
2914 mlID.Form("%d", GetCurrentRun());
2915 fMonaLisa->SendParameters(&mlList, mlID);
b0e53b15 2916 }
2bb7b766 2917 } else {
2bb7b766 2918 TString statusStr(status);
2919 if(statusStr.Contains("done", TString::kIgnoreCase) ||
2920 statusStr.Contains("failed", TString::kIgnoreCase)){
eba76848 2921 setClause = Form("set %s=\"%s\"", detector, status);
2bb7b766 2922 } else {
2923 Log("SHUTTLE",
2924 Form("UpdateShuttleLogbook - Invalid status <%s> for detector %s",
2925 status, detector));
2926 return kFALSE;
2927 }
2928 }
57f50b3c 2929
2bb7b766 2930 TString whereClause = Form("where run=%d", GetCurrentRun());
2931
441b0e9c 2932 TString sqlQuery = Form("update %s %s %s",
2933 fConfig->GetShuttlelbTable(), setClause.Data(), whereClause.Data());
57f50b3c 2934
2bb7b766 2935 AliDebug(2, Form("SQL query: \n%s",sqlQuery.Data()));
2936
2937 // Query execution
2938 TSQLResult* aResult;
be48e3ea 2939 aResult = dynamic_cast<TSQLResult*> (fServer[3]->Query(sqlQuery));
2bb7b766 2940 if (!aResult) {
2941 Log("SHUTTLE", Form("UpdateShuttleLogbook - Can't execute query <%s>", sqlQuery.Data()));
2942 return kFALSE;
57f50b3c 2943 }
2bb7b766 2944 delete aResult;
57f50b3c 2945
2946 return kTRUE;
2947}
2948
2949//______________________________________________________________________________________________
2bb7b766 2950Int_t AliShuttle::GetCurrentRun() const
2951{
9827400b 2952 //
2953 // Get current run from logbook entry
2954 //
57f50b3c 2955
2bb7b766 2956 return fLogbookEntry ? fLogbookEntry->GetRun() : -1;
57f50b3c 2957}
2958
2959//______________________________________________________________________________________________
2bb7b766 2960UInt_t AliShuttle::GetCurrentStartTime() const
2961{
9827400b 2962 //
2963 // get current start time
2964 //
57f50b3c 2965
2bb7b766 2966 return fLogbookEntry ? fLogbookEntry->GetStartTime() : 0;
57f50b3c 2967}
2968
2969//______________________________________________________________________________________________
2bb7b766 2970UInt_t AliShuttle::GetCurrentEndTime() const
2971{
9827400b 2972 //
2973 // get current end time from logbook entry
2974 //
57f50b3c 2975
2bb7b766 2976 return fLogbookEntry ? fLogbookEntry->GetEndTime() : 0;
57f50b3c 2977}
2978
675f64cd 2979//______________________________________________________________________________________________
2980UInt_t AliShuttle::GetCurrentYear() const
2981{
2982 //
2983 // Get current year from logbook entry
2984 //
2985
2986 if (!fLogbookEntry) return 0;
2987
2988 TTimeStamp startTime(GetCurrentStartTime());
2989 TString year = Form("%d",startTime.GetDate());
2990 year = year(0,4);
2991
2992 return year.Atoi();
2993}
2994
2995//______________________________________________________________________________________________
2996const char* AliShuttle::GetLHCPeriod() const
2997{
2998 //
2999 // Get current LHC period from logbook entry
3000 //
3001
3002 if (!fLogbookEntry) return 0;
3003
3004 return fLogbookEntry->GetRunParameter("LHCperiod");
3005}
3006
b948db8d 3007//______________________________________________________________________________________________
3008void AliShuttle::Log(const char* detector, const char* message)
3009{
9827400b 3010 //
3011 // Fill log string with a message
3012 //
b948db8d 3013
7d4cf768 3014 TString logRunDir = GetShuttleLogDir();
3015 if (GetCurrentRun() >=0)
3016 logRunDir += Form("/%d", GetCurrentRun());
3017
3018 void* dir = gSystem->OpenDirectory(logRunDir.Data());
84090f85 3019 if (dir == NULL) {
7d4cf768 3020 if (gSystem->mkdir(logRunDir.Data(), kTRUE)) {
36c99a6a 3021 AliError(Form("Can't open directory <%s>", GetShuttleLogDir()));
84090f85 3022 return;
3023 }
b948db8d 3024
84090f85 3025 } else {
3026 gSystem->FreeDirectory(dir);
3027 }
b948db8d 3028
cb343cfd 3029 TString toLog = Form("%s (%d): %s - ", TTimeStamp(time(0)).AsString("s"), getpid(), detector);
e7f62f16 3030 if (GetCurrentRun() >= 0)
3031 toLog += Form("run %d - ", GetCurrentRun());
2bb7b766 3032 toLog += Form("%s", message);
3033
84090f85 3034 AliInfo(toLog.Data());
ffa29e93 3035
3036 // if we redirect the log output already to the file, leave here
3037 if (fOutputRedirected && strcmp(detector, "SHUTTLE") != 0)
3038 return;
b948db8d 3039
ffa29e93 3040 TString fileName = GetLogFileName(detector);
e7f62f16 3041
84090f85 3042 gSystem->ExpandPathName(fileName);
3043
3044 ofstream logFile;
3045 logFile.open(fileName, ofstream::out | ofstream::app);
3046
3047 if (!logFile.is_open()) {
3048 AliError(Form("Could not open file %s", fileName.Data()));
3049 return;
3050 }
7bfb2090 3051
84090f85 3052 logFile << toLog.Data() << "\n";
b948db8d 3053
84090f85 3054 logFile.close();
b948db8d 3055}
2bb7b766 3056
ffa29e93 3057//______________________________________________________________________________________________
3058TString AliShuttle::GetLogFileName(const char* detector) const
3059{
3060 //
3061 // returns the name of the log file for a given sub detector
3062 //
3063
3064 TString fileName;
3065
3066 if (GetCurrentRun() >= 0)
7d4cf768 3067 {
3068 fileName.Form("%s/%d/%s_%d.log", GetShuttleLogDir(), GetCurrentRun(),
3069 detector, GetCurrentRun());
3070 } else {
ffa29e93 3071 fileName.Form("%s/%s.log", GetShuttleLogDir(), detector);
7d4cf768 3072 }
ffa29e93 3073
3074 return fileName;
3075}
3076
ee6f7523 3077//______________________________________________________________________________________________
3078void AliShuttle::SendAlive()
3079{
3080 // sends alive message to ML
3081
3082 TMonaLisaText mlStatus("SHUTTLE_status", "Alive");
3083
3084 TList mlList;
3085 mlList.Add(&mlStatus);
3086
3087 fMonaLisa->SendParameters(&mlList, "__PROCESSINGINFO__");
3088}
3089
2bb7b766 3090//______________________________________________________________________________________________
3091Bool_t AliShuttle::Collect(Int_t run)
3092{
9827400b 3093 //
3094 // Collects conditions data for all UNPROCESSED run written to DAQ LogBook in case of run = -1 (default)
3095 // If a dedicated run is given this run is processed
3096 //
3097 // In operational mode, this is the Shuttle function triggered by the EOR signal.
3098 //
2bb7b766 3099
eba76848 3100 if (run == -1)
3101 Log("SHUTTLE","Collect - Shuttle called. Collecting conditions data for unprocessed runs");
3102 else
3103 Log("SHUTTLE", Form("Collect - Shuttle called. Collecting conditions data for run %d", run));
cb343cfd 3104
3105 SetLastAction("Starting");
2bb7b766 3106
ee6f7523 3107 // create ML instance
3108 if (!fMonaLisa)
3109 fMonaLisa = new TMonaLisaWriter(fConfig->GetMonitorHost(), fConfig->GetMonitorTable());
3110
ee6f7523 3111 SendAlive();
fbc112e3 3112 CountOpenRuns();
ee6f7523 3113
2bb7b766 3114 TString whereClause("where shuttle_done=0");
eba76848 3115 if (run != -1)
3116 whereClause += Form(" and run=%d", run);
2bb7b766 3117
3118 TObjArray shuttleLogbookEntries;
be48e3ea 3119 if (!QueryShuttleLogbook(whereClause, shuttleLogbookEntries))
3120 {
cb343cfd 3121 Log("SHUTTLE", "Collect - Can't retrieve entries from Shuttle logbook");
2bb7b766 3122 return kFALSE;
3123 }
3124
9e080f92 3125 if (shuttleLogbookEntries.GetEntries() == 0)
3126 {
3127 if (run == -1)
3128 Log("SHUTTLE","Collect - Found no UNPROCESSED runs in Shuttle logbook");
3129 else
3130 Log("SHUTTLE", Form("Collect - Run %d is already DONE "
3131 "or it does not exist in Shuttle logbook", run));
3132 return kTRUE;
3133 }
3134
be48e3ea 3135 for (UInt_t iDet=0; iDet<NDetectors(); iDet++)
3136 fFirstUnprocessed[iDet] = kTRUE;
3137
fc5a4708 3138 if (run != -1)
be48e3ea 3139 {
3140 // query Shuttle logbook for earlier runs, check if some detectors are unprocessed,
3141 // flag them into fFirstUnprocessed array
3142 TString whereClause(Form("where shuttle_done=0 and run < %d", run));
3143 TObjArray tmpLogbookEntries;
3144 if (!QueryShuttleLogbook(whereClause, tmpLogbookEntries))
3145 {
3146 Log("SHUTTLE", "Collect - Can't retrieve entries from Shuttle logbook");
3147 return kFALSE;
3148 }
3149
3150 TIter iter(&tmpLogbookEntries);
3151 AliShuttleLogbookEntry* anEntry = 0;
3152 while ((anEntry = dynamic_cast<AliShuttleLogbookEntry*> (iter.Next())))
3153 {
3154 for (UInt_t iDet=0; iDet<NDetectors(); iDet++)
3155 {
3156 if (anEntry->GetDetectorStatus(iDet) == AliShuttleLogbookEntry::kUnprocessed)
3157 {
3158 AliDebug(2, Form("Run %d: setting %s as \"not first time unprocessed\"",
3159 anEntry->GetRun(), GetDetName(iDet)));
3160 fFirstUnprocessed[iDet] = kFALSE;
3161 }
3162 }
3163
3164 }
3165
3166 }
3167
3168 if (!RetrieveConditionsData(shuttleLogbookEntries))
3169 {
cb343cfd 3170 Log("SHUTTLE", "Collect - Process of at least one run failed");
2bb7b766 3171 return kFALSE;
3172 }
3173
36c99a6a 3174 Log("SHUTTLE", "Collect - Requested run(s) successfully processed");
eba76848 3175 return kTRUE;
2bb7b766 3176}
3177
2bb7b766 3178//______________________________________________________________________________________________
3179Bool_t AliShuttle::RetrieveConditionsData(const TObjArray& dateEntries)
3180{
9827400b 3181 //
3182 // Retrieve conditions data for all runs that aren't processed yet
3183 //
2bb7b766 3184
3185 Bool_t hasError = kFALSE;
3186
3187 TIter iter(&dateEntries);
3188 AliShuttleLogbookEntry* anEntry;
3189
3190 while ((anEntry = (AliShuttleLogbookEntry*) iter.Next())){
3191 if (!Process(anEntry)){
3192 hasError = kTRUE;
3193 }
4b95672b 3194
3195 // clean SHUTTLE temp directory
d524ade6 3196 //TString filename = Form("%s/*.shuttle", GetShuttleTempDir());
3197 //RemoveFile(filename.Data());
2bb7b766 3198 }
3199
3200 return hasError == kFALSE;
3201}
cb343cfd 3202
3203//______________________________________________________________________________________________
3204ULong_t AliShuttle::GetTimeOfLastAction() const
3205{
9827400b 3206 //
3207 // Gets time of last action
3208 //
3209
cb343cfd 3210 ULong_t tmp;
36c99a6a 3211
cb343cfd 3212 fMonitoringMutex->Lock();
be48e3ea 3213
cb343cfd 3214 tmp = fLastActionTime;
36c99a6a 3215
cb343cfd 3216 fMonitoringMutex->UnLock();
36c99a6a 3217
cb343cfd 3218 return tmp;
3219}
3220
3221//______________________________________________________________________________________________
3222const TString AliShuttle::GetLastAction() const
3223{
9827400b 3224 //
cb343cfd 3225 // returns a string description of the last action
9827400b 3226 //
cb343cfd 3227
3228 TString tmp;
36c99a6a 3229
cb343cfd 3230 fMonitoringMutex->Lock();
3231
3232 tmp = fLastAction;
3233
3234 fMonitoringMutex->UnLock();
3235
36c99a6a 3236 return tmp;
cb343cfd 3237}
3238
3239//______________________________________________________________________________________________
3240void AliShuttle::SetLastAction(const char* action)
3241{
9827400b 3242 //
cb343cfd 3243 // updates the monitoring variables
9827400b 3244 //
36c99a6a 3245
cb343cfd 3246 fMonitoringMutex->Lock();
36c99a6a 3247
cb343cfd 3248 fLastAction = action;
3249 fLastActionTime = time(0);
3250
3251 fMonitoringMutex->UnLock();
3252}
eba76848 3253
3254//______________________________________________________________________________________________
3255const char* AliShuttle::GetRunParameter(const char* param)
3256{
9827400b 3257 //
3258 // returns run parameter read from DAQ logbook
3259 //
eba76848 3260
3261 if(!fLogbookEntry) {
3262 AliError("No logbook entry!");
3263 return 0;
3264 }
3265
3266 return fLogbookEntry->GetRunParameter(param);
3267}
57c1a579 3268
d386d623 3269//______________________________________________________________________________________________
9827400b 3270AliCDBEntry* AliShuttle::GetFromOCDB(const char* detector, const AliCDBPath& path)
d386d623 3271{
9827400b 3272 //
3273 // returns object from OCDB valid for current run
3274 //
d386d623 3275
9827400b 3276 if (fTestMode & kErrorOCDB)
3277 {
3278 Log(detector, "GetFromOCDB - In TESTMODE - Simulating error with OCDB");
3279 return 0;
3280 }
3281
d386d623 3282 AliCDBStorage *sto = AliCDBManager::Instance()->GetStorage(fgkMainCDB);
3283 if (!sto)
3284 {
9827400b 3285 Log(detector, "GetFromOCDB - Cannot activate main OCDB for query!");
d386d623 3286 return 0;
3287 }
3288
3289 return dynamic_cast<AliCDBEntry*> (sto->Get(path, GetCurrentRun()));
3290}
3291
57c1a579 3292//______________________________________________________________________________________________
3293Bool_t AliShuttle::SendMail()
3294{
9827400b 3295 //
3296 // sends a mail to the subdetector expert in case of preprocessor error
3297 //
3298
3299 if (fTestMode != kNone)
3300 return kTRUE;
57c1a579 3301
57c1a579 3302 TString to="";
3303 TIter iterExperts(fConfig->GetResponsibles(fCurrentDetector));
3304 TObjString *anExpert=0;
3305 while ((anExpert = (TObjString*) iterExperts.Next()))
3306 {
3307 to += Form("%s,", anExpert->GetName());
3308 }
4508ab8b 3309 if (to.Length() > 0)
3310 to.Remove(to.Length()-1);
909732f7 3311 AliDebug(2, Form("to: %s",to.Data()));
57c1a579 3312
86aa42c3 3313 if (to.IsNull()) {
c88ad5db 3314 Log("SHUTTLE", "List of detector responsibles not yet set!");
36c99a6a 3315 return kFALSE;
3316 }
3317
fbc112e3 3318 void* dir = gSystem->OpenDirectory(GetShuttleLogDir());
3319 if (dir == NULL)
3320 {
3321 if (gSystem->mkdir(GetShuttleLogDir(), kTRUE))
3322 {
3323 Log("SHUTTLE", Form("SendMail - Can't open directory <%s>", GetShuttleLogDir()));
3324 return kFALSE;
3325 }
3326
3327 } else {
3328 gSystem->FreeDirectory(dir);
3329 }
3330
4508ab8b 3331 TString bodyFileName;
3332 bodyFileName.Form("%s/mail.body", GetShuttleLogDir());
3333 gSystem->ExpandPathName(bodyFileName);
3334
3335 ofstream mailBody;
3336 mailBody.open(bodyFileName, ofstream::out);
3337
3338 if (!mailBody.is_open())
3339 {
3340 Log("SHUTTLE", Form("Could not open mail body file %s", bodyFileName.Data()));
3341 return kFALSE;
3342 }
3343
57c1a579 3344 TString cc="alberto.colla@cern.ch";
3345
6a1146c4 3346 TString subject = Form("%s Shuttle preprocessor FAILED in run %d (run type = %s)!",
3347 fCurrentDetector.Data(), GetCurrentRun(), GetRunType());
909732f7 3348 AliDebug(2, Form("subject: %s", subject.Data()));
57c1a579 3349
3350 TString body = Form("Dear %s expert(s), \n\n", fCurrentDetector.Data());
3351 body += Form("SHUTTLE just detected that your preprocessor "
6a1146c4 3352 "failed processing run %d (run type = %s)!!\n\n",
3353 GetCurrentRun(), GetRunType());
7d4cf768 3354 body += Form("Please check %s status on the SHUTTLE monitoring page: \n\n",
3355 fCurrentDetector.Data());
b0e53b15 3356 if (fConfig->GetRunMode() == AliShuttleConfig::kTest)
3357 {
3358 body += Form("\thttp://pcalimonitor.cern.ch:8889/shuttle.jsp?time=168 \n\n");
3359 } else {
4508ab8b 3360 body += Form("\thttp://pcalimonitor.cern.ch/shuttle.jsp?instance=PROD&time=168 \n\n");
b0e53b15 3361 }
3362
7d4cf768 3363
3364 TString logFolder = "logs";
3365 if (fConfig->GetRunMode() == AliShuttleConfig::kProd)
3366 logFolder += "_PROD";
3367
3368
546242fb 3369 body += Form("Find the %s log for the current run on \n\n"
7d4cf768 3370 "\thttp://pcalishuttle01.cern.ch:8880/%s/%d/%s_%d.log \n\n",
3371 fCurrentDetector.Data(), logFolder.Data(), GetCurrentRun(),
3372 fCurrentDetector.Data(), GetCurrentRun());
3373 body += Form("The last 10 lines of %s log file are following:\n\n", fCurrentDetector.Data());
57c1a579 3374
909732f7 3375 AliDebug(2, Form("Body begin: %s", body.Data()));
57c1a579 3376
3377 mailBody << body.Data();
3378 mailBody.close();
3379 mailBody.open(bodyFileName, ofstream::out | ofstream::app);
3380
7d4cf768 3381 TString logFileName = Form("%s/%d/%s_%d.log", GetShuttleLogDir(),
3382 GetCurrentRun(), fCurrentDetector.Data(), GetCurrentRun());
57c1a579 3383 TString tailCommand = Form("tail -n 10 %s >> %s", logFileName.Data(), bodyFileName.Data());
3384 if (gSystem->Exec(tailCommand.Data()))
3385 {
3386 mailBody << Form("%s log file not found ...\n\n", fCurrentDetector.Data());
3387 }
3388
3389 TString endBody = Form("------------------------------------------------------\n\n");
675f64cd 3390 endBody += Form("In case of problems please contact the SHUTTLE core team.\n\n");
3391 endBody += "Please do not answer this message directly, it is automatically generated.\n\n";
3392 endBody += "Greetings,\n\n \t\t\tthe SHUTTLE\n";
3393
3394 AliDebug(2, Form("Body end: %s", endBody.Data()));
3395
3396 mailBody << endBody.Data();
3397
3398 mailBody.close();
3399
3400 // send mail!
3401 TString mailCommand = Form("mail -s \"%s\" -c %s %s < %s",
3402 subject.Data(),
3403 cc.Data(),
3404 to.Data(),
3405 bodyFileName.Data());
3406 AliDebug(2, Form("mail command: %s", mailCommand.Data()));
3407
3408 Bool_t result = gSystem->Exec(mailCommand.Data());
3409
3410 return result == 0;
3411}
3412
3413//______________________________________________________________________________________________
3414Bool_t AliShuttle::SendMailToDCS()
3415{
3416 //
3417 // sends a mail to the DCS experts in case of DCS error
3418 //
3419
3420 if (fTestMode != kNone)
3421 return kTRUE;
3422
3423 void* dir = gSystem->OpenDirectory(GetShuttleLogDir());
3424 if (dir == NULL)
3425 {
3426 if (gSystem->mkdir(GetShuttleLogDir(), kTRUE))
3427 {
3428 Log("SHUTTLE", Form("SendMailToDCS - Can't open directory <%s>", GetShuttleLogDir()));
3429 return kFALSE;
3430 }
3431
3432 } else {
3433 gSystem->FreeDirectory(dir);
3434 }
3435
3436 TString bodyFileName;
3437 bodyFileName.Form("%s/mail.body", GetShuttleLogDir());
3438 gSystem->ExpandPathName(bodyFileName);
3439
3440 ofstream mailBody;
3441 mailBody.open(bodyFileName, ofstream::out);
3442
3443 if (!mailBody.is_open())
3444 {
3445 Log("SHUTTLE", Form("SendMailToDCS - Could not open mail body file %s", bodyFileName.Data()));
3446 return kFALSE;
3447 }
3448
3449 TString to="Vladimir.Fekete@cern.ch, Svetozar.Kapusta@cern.ch";
3450 //TString to="alberto.colla@cern.ch";
3451 AliDebug(2, Form("to: %s",to.Data()));
3452
3453 if (to.IsNull()) {
3454 Log("SHUTTLE", "List of detector responsibles not yet set!");
3455 return kFALSE;
3456 }
3457
3458 TString cc="alberto.colla@cern.ch";
3459
3460 TString subject = Form("Retrieval of data points for %s FAILED in run %d !",
3461 fCurrentDetector.Data(), GetCurrentRun());
3462 AliDebug(2, Form("subject: %s", subject.Data()));
3463
3464 TString body = Form("Dear DCS experts, \n\n");
3465 body += Form("SHUTTLE couldn\'t retrieve the data points for detector %s "
3466 "in run %d!!\n\n", fCurrentDetector.Data(), GetCurrentRun());
7d4cf768 3467 body += Form("Please check %s status on the SHUTTLE monitoring page: \n\n",
3468 fCurrentDetector.Data());
b0e53b15 3469 if (fConfig->GetRunMode() == AliShuttleConfig::kTest)
3470 {
3471 body += Form("\thttp://pcalimonitor.cern.ch:8889/shuttle.jsp?time=168 \n\n");
3472 } else {
3473 body += Form("\thttp://pcalimonitor.cern.ch/shuttle.jsp?instance=PROD?time=168 \n\n");
3474 }
7d4cf768 3475
3476 TString logFolder = "logs";
3477 if (fConfig->GetRunMode() == AliShuttleConfig::kProd)
3478 logFolder += "_PROD";
3479
3480
675f64cd 3481 body += Form("Find the %s log for the current run on \n\n"
7d4cf768 3482 "\thttp://pcalishuttle01.cern.ch:8880/%s/%d/%s_%d.log \n\n",
3483 fCurrentDetector.Data(), logFolder.Data(), GetCurrentRun(),
3484 fCurrentDetector.Data(), GetCurrentRun());
3485 body += Form("The last 10 lines of %s log file are following:\n\n", fCurrentDetector.Data());
675f64cd 3486
3487 AliDebug(2, Form("Body begin: %s", body.Data()));
3488
3489 mailBody << body.Data();
3490 mailBody.close();
3491 mailBody.open(bodyFileName, ofstream::out | ofstream::app);
3492
7d4cf768 3493 TString logFileName = Form("%s/%d/%s_%d.log", GetShuttleLogDir(), GetCurrentRun(),
3494 fCurrentDetector.Data(), GetCurrentRun());
675f64cd 3495 TString tailCommand = Form("tail -n 10 %s >> %s", logFileName.Data(), bodyFileName.Data());
3496 if (gSystem->Exec(tailCommand.Data()))
3497 {
3498 mailBody << Form("%s log file not found ...\n\n", fCurrentDetector.Data());
3499 }
3500
3501 TString endBody = Form("------------------------------------------------------\n\n");
36c99a6a 3502 endBody += Form("In case of problems please contact the SHUTTLE core team.\n\n");
3503 endBody += "Please do not answer this message directly, it is automatically generated.\n\n";
546242fb 3504 endBody += "Greetings,\n\n \t\t\tthe SHUTTLE\n";
57c1a579 3505
909732f7 3506 AliDebug(2, Form("Body end: %s", endBody.Data()));
57c1a579 3507
3508 mailBody << endBody.Data();
3509
3510 mailBody.close();
3511
3512 // send mail!
3513 TString mailCommand = Form("mail -s \"%s\" -c %s %s < %s",
3514 subject.Data(),
3515 cc.Data(),
3516 to.Data(),
3517 bodyFileName.Data());
909732f7 3518 AliDebug(2, Form("mail command: %s", mailCommand.Data()));
57c1a579 3519
3520 Bool_t result = gSystem->Exec(mailCommand.Data());
3521
3522 return result == 0;
3523}
d386d623 3524
441b0e9c 3525//______________________________________________________________________________________________
9827400b 3526const char* AliShuttle::GetRunType()
441b0e9c 3527{
9827400b 3528 //
3529 // returns run type read from "run type" logbook
3530 //
441b0e9c 3531
3532 if(!fLogbookEntry) {
3533 AliError("No logbook entry!");
3534 return 0;
3535 }
3536
9827400b 3537 return fLogbookEntry->GetRunType();
441b0e9c 3538}
3539
4859271b 3540//______________________________________________________________________________________________
3541Bool_t AliShuttle::GetHLTStatus()
3542{
3543 // Return HLT status (ON=1 OFF=0)
3544 // Converts the HLT status from the status string read in the run logbook (not just a bool)
3545
3546 if(!fLogbookEntry) {
3547 AliError("No logbook entry!");
3548 return 0;
3549 }
3550
3551 // TODO implement when HLTStatus is inserted in run logbook
3552 //TString hltStatus = fLogbookEntry->GetRunParameter("HLTStatus");
3553 //if(hltStatus == "OFF") {return kFALSE};
3554
3555 return kTRUE;
3556}
3557
d386d623 3558//______________________________________________________________________________________________
3559void AliShuttle::SetShuttleTempDir(const char* tmpDir)
3560{
9827400b 3561 //
3562 // sets Shuttle temp directory
3563 //
d386d623 3564
3565 fgkShuttleTempDir = gSystem->ExpandPathName(tmpDir);
3566}
3567
3568//______________________________________________________________________________________________
3569void AliShuttle::SetShuttleLogDir(const char* logDir)
3570{
9827400b 3571 //
3572 // sets Shuttle log directory
3573 //
d386d623 3574
3575 fgkShuttleLogDir = gSystem->ExpandPathName(logDir);
3576}