/*
$Log$
+Revision 1.64 2007/11/13 16:15:47 acolla
+DCS map is stored in a file in the temp folder where the detector is processed.
+If the preprocessor fails, the temp folder is not removed. This will help the debugging of the problem.
+
+Revision 1.63 2007/11/02 10:53:16 acolla
+Protection added to AliShuttle::CopyFileLocally
+
+Revision 1.62 2007/10/31 18:23:13 acolla
+Furter developement on the Shuttle:
+
+- Shuttle now connects to the Grid as alidaq. The OCDB and Reference folders
+are now built from /alice/data, e.g.:
+/alice/data/2007/LHC07a/OCDB
+
+the year and LHC period are taken from the Shuttle.
+Raw metadata files are stored by GRP to:
+/alice/data/2007/LHC07a/<runNb>/Raw/RunMetadata.root
+
+- Shuttle sends a mail to DCS experts each time DP retrieval fails.
+
+Revision 1.61 2007/10/30 20:33:51 acolla
+Improved managing of temporary folders, which weren't correctly handled.
+Resolved bug introduced in StoreReferenceFile, which caused SPD preprocessor fail.
+
+Revision 1.60 2007/10/29 18:06:16 acolla
+
+New function StoreRunMetadataFile added to preprocessor and Shuttle interface
+This function can be used by GRP only. It stores raw data tags merged file to the
+raw data folder (e.g. /alice/data/2008/LHC08a/000099999/Raw).
+
+KNOWN ISSUES:
+
+1. Shuttle cannot write to /alice/data/ because it belongs to alidaq. Tag file is stored in /alice/simulation/... for the time being.
+2. Due to a bug in TAlien::Mkdir, the creation of a folder in recursive mode (-p option) does not work. The problem
+has been corrected in the root package on the Shuttle machine.
+
+Revision 1.59 2007/10/05 12:40:55 acolla
+
+Result error code added to AliDCSClient data members (it was "lost" with the new implementation of TMap* GetAliasValues and GetDPValues).
+
+Revision 1.58 2007/09/28 15:27:40 acolla
+
+AliDCSClient "multiSplit" option added in the DCS configuration
+in AliDCSMessage: variable MAX_BODY_SIZE set to 500000
+
+Revision 1.57 2007/09/27 16:53:13 acolla
+Detectors can have more than one AMANDA server. SHUTTLE queries the servers sequentially,
+merges the dcs aliases/DPs in one TMap and sends it to the preprocessor.
+
+Revision 1.56 2007/09/14 16:46:14 jgrosseo
+1) Connect and Close are called before and after each query, so one can
+keep the same AliDCSClient object.
+2) The splitting of a query is moved to GetDPValues/GetAliasValues.
+3) Splitting interval can be specified in constructor
+
+Revision 1.55 2007/08/06 12:26:40 acolla
+Function Bool_t GetHLTStatus added to preprocessor. It returns the status of HLT
+read from the run logbook.
+
+Revision 1.54 2007/07/12 09:51:25 jgrosseo
+removed duplicated log message in GetFile
+
+Revision 1.53 2007/07/12 09:26:28 jgrosseo
+updating hlt fxs base path
+
+Revision 1.52 2007/07/12 08:06:45 jgrosseo
+adding log messages in getfile... functions
+adding not implemented copy constructor in alishuttleconfigholder
+
+Revision 1.51 2007/07/03 17:24:52 acolla
+root moved to v5-16-00. TFileMerger->Cp moved to TFile::Cp.
+
+Revision 1.50 2007/07/02 17:19:32 acolla
+preprocessor is run in a temp directory that is removed when process is finished.
+
+Revision 1.49 2007/06/29 10:45:06 acolla
+Number of columns in MySql Shuttle logbook increased by one (HLT added)
+
+Revision 1.48 2007/06/21 13:06:19 acolla
+GetFileSources returns dummy list with 1 source if system=DCS (better than
+returning error as it was)
+
+Revision 1.47 2007/06/19 17:28:56 acolla
+HLT updated; missing map bug removed.
+
+Revision 1.46 2007/06/09 13:01:09 jgrosseo
+Switching to retrieval of several DCS DPs at a time (multiDPrequest)
+
+Revision 1.45 2007/05/30 06:35:20 jgrosseo
+Adding functionality to the Shuttle/TestShuttle:
+o) Function to retrieve list of sources from a given system (GetFileSources with id=0)
+o) Function to retrieve list of IDs for a given source (GetFileIDs)
+These functions are needed for dealing with the tag files that are saved for the GRP preprocessor
+Example code has been added to the TestProcessor in TestShuttle
+
+Revision 1.44 2007/05/11 16:09:32 acolla
+Reference files for ITS, MUON and PHOS are now stored in OfflineDetName/OnlineDetName/run_...
+example: ITS/SPD/100_filename.root
+
+Revision 1.43 2007/05/10 09:59:51 acolla
+Various bug fixes in StoreRefFilesToGrid; Cleaning of reference storage before processing detector (CleanReferenceStorage)
+
+Revision 1.42 2007/05/03 08:01:39 jgrosseo
+typo in last commit :-(
+
+Revision 1.41 2007/05/03 08:00:48 jgrosseo
+fixing log message when pp want to skip dcs value retrieval
+
+Revision 1.40 2007/04/27 07:06:48 jgrosseo
+GetFileSources returns empty list in case of no files, but successful query
+No mails sent in testmode
+
+Revision 1.39 2007/04/17 12:43:57 acolla
+Correction in StoreOCDB; change of text in mail to detector expert
+
+Revision 1.38 2007/04/12 08:26:18 jgrosseo
+updated comment
+
+Revision 1.37 2007/04/10 16:53:14 jgrosseo
+redirecting sub detector stdout, stderr to sub detector log file
+
+Revision 1.35 2007/04/04 16:26:38 acolla
+1. Re-organization of function calls in TestPreprocessor to make it more meaningful.
+2. Added missing dependency in test preprocessors.
+3. in AliShuttle.cxx: processing time and memory consumption info on a single line.
+
+Revision 1.34 2007/04/04 10:33:36 jgrosseo
+1) Storing of files to the Grid is now done _after_ your preprocessors succeeded. This is transparent, which means that you can still use the same functions (Store, StoreReferenceData) to store files to the Grid. However, the Shuttle first stores them locally and transfers them after the preprocessor finished. The return code of these two functions has changed from UInt_t to Bool_t which gives you the success of the storing.
+In case of an error with the Grid, the Shuttle will retry the storing later, the preprocessor does not need to be run again.
+
+2) The meaning of the return code of the preprocessor has changed. 0 is now success and any other value means failure. This value is stored in the log and you can use it to keep details about the error condition.
+
+3) New function StoreReferenceFile to _directly_ store a file (without opening it) to the reference storage.
+
+4) The memory usage of the preprocessor is monitored. If it exceeds 2 GB it is terminated.
+
+5) New function AliPreprocessor::ProcessDCS(). If you do not need to have DCS data in all cases, you can skip the processing by implemting this function and returning kFALSE under certain conditions. E.g. if there is a certain run type.
+If you always need DCS data (like before), you do not need to implement it.
+
+6) The run type has been added to the monitoring page
+
+Revision 1.33 2007/04/03 13:56:01 acolla
+Grid Storage at the end of preprocessing. Added virtual method to disable DCS query according to the
+run type.
+
Revision 1.32 2007/02/28 10:41:56 acolla
Run type field added in SHUTTLE framework. Run type is read from "run type" logbook and retrieved by
AliPreprocessor::GetRunType() function.
#include <TSQLResult.h>
#include <TSQLRow.h>
#include <TMutex.h>
+#include <TSystemDirectory.h>
+#include <TSystemFile.h>
+#include <TFile.h>
+#include <TGrid.h>
+#include <TGridResult.h>
#include <TMonaLisaWriter.h>
ClassImp(AliShuttle)
-Bool_t AliShuttle::fgkProcessDCS(kTRUE);
-
//______________________________________________________________________________________________
AliShuttle::AliShuttle(const AliShuttleConfig* config,
UInt_t timeout, Int_t retries):
fMonitoringMutex(0),
fLastActionTime(0),
fLastAction(),
-fMonaLisa(0)
+fMonaLisa(0),
+fTestMode(kNone),
+fReadTestMode(kFALSE),
+fOutputRedirected(kFALSE)
{
//
// config: AliShuttleConfig used
//______________________________________________________________________________________________
AliShuttle::~AliShuttle()
{
-// destructor
+ //
+ // destructor
+ //
fPreprocessorMap.DeleteAll();
for(int iSys=0;iSys<4;iSys++)
Bool_t AliShuttle::Store(const AliCDBPath& path, TObject* object,
AliCDBMetaData* metaData, Int_t validityStart, Bool_t validityInfinite)
{
- // Stores a CDB object in the storage for offline reconstruction. Objects that are not needed for
- // offline reconstruction, but should be stored anyway (e.g. for debugging) should NOT be stored
- // using this function. Use StoreReferenceData instead!
- // It calls StoreLocally function which temporarily stores the data locally; when the preprocessor
- // finishes the data are transferred to the main storage (Grid).
+ // Stores a CDB object in the storage for offline reconstruction. Objects that are not needed for
+ // offline reconstruction, but should be stored anyway (e.g. for debugging) should NOT be stored
+ // using this function. Use StoreReferenceData instead!
+ // It calls StoreLocally function which temporarily stores the data locally; when the preprocessor
+ // finishes the data are transferred to the main storage (Grid).
return StoreLocally(fgkLocalCDB, path, object, metaData, validityStart, validityInfinite);
-
}
//______________________________________________________________________________________________
Bool_t AliShuttle::StoreReferenceData(const AliCDBPath& path, TObject* object, AliCDBMetaData* metaData)
{
- // Stores a CDB object in the storage for reference data. This objects will not be available during
- // offline reconstrunction. Use this function for reference data only!
- // It calls StoreLocally function which temporarily stores the data locally; when the preprocessor
- // finishes the data are transferred to the main storage (Grid).
+ // Stores a CDB object in the storage for reference data. This objects will not be available during
+ // offline reconstrunction. Use this function for reference data only!
+ // It calls StoreLocally function which temporarily stores the data locally; when the preprocessor
+ // finishes the data are transferred to the main storage (Grid).
return StoreLocally(fgkLocalRefStorage, path, object, metaData);
-
}
//______________________________________________________________________________________________
const AliCDBPath& path, TObject* object, AliCDBMetaData* metaData,
Int_t validityStart, Bool_t validityInfinite)
{
- // Store object temporarily in local storage. Parameters are passed by Store and StoreReferenceData functions.
- // when the preprocessor finishes the data are transferred to the main storage (Grid).
- // The parameters are:
- // 1) Uri of the backup storage (Local)
- // 2) the object's path.
- // 3) the object to be stored
- // 4) the metaData to be associated with the object
- // 5) the validity start run number w.r.t. the current run,
- // if the data is valid only for this run leave the default 0
- // 6) specifies if the calibration data is valid for infinity (this means until updated),
- // typical for calibration runs, the default is kFALSE
- //
- // returns 0 if fail, 1 otherwise
+ // Store object temporarily in local storage. Parameters are passed by Store and StoreReferenceData functions.
+ // when the preprocessor finishes the data are transferred to the main storage (Grid).
+ // The parameters are:
+ // 1) Uri of the backup storage (Local)
+ // 2) the object's path.
+ // 3) the object to be stored
+ // 4) the metaData to be associated with the object
+ // 5) the validity start run number w.r.t. the current run,
+ // if the data is valid only for this run leave the default 0
+ // 6) specifies if the calibration data is valid for infinity (this means until updated),
+ // typical for calibration runs, the default is kFALSE
+ //
+ // returns 0 if fail, 1 otherwise
+ if (fTestMode & kErrorStorage)
+ {
+ Log(fCurrentDetector, "StoreLocally - In TESTMODE - Simulating error while storing locally");
+ return kFALSE;
+ }
+
const char* cdbType = (localUri == fgkLocalCDB) ? "CDB" : "Reference";
Int_t firstRun = GetCurrentRun() - validityStart;
if(firstRun < 0) {
- AliError("First valid run happens to be less than 0! Setting it to 0.");
+ AliWarning("First valid run happens to be less than 0! Setting it to 0.");
firstRun=0;
}
if(!result) {
- Log("SHUTTLE", Form("StoreLocally - Can't store %s data!", fCurrentDetector.Data()));
+ Log(fCurrentDetector, Form("StoreLocally - Can't store object <%s>!", id.ToString().Data()));
}
return result;
//______________________________________________________________________________________________
Bool_t AliShuttle::StoreOCDB()
{
- // Called when preprocessor ends successfully or when previous storage attempt failed (kStoreError status)
- // Calls underlying StoreOCDB(const char*) function twice, for OCDB and Reference storage.
-
- AliInfo("Storing OCDB data ...");
+ //
+ // Called when preprocessor ends successfully or when previous storage attempt failed (kStoreError status)
+ // Calls underlying StoreOCDB(const char*) function twice, for OCDB and Reference storage.
+ // Then calls StoreRefFilesToGrid to store reference files.
+ //
+
+ if (fTestMode & kErrorGrid)
+ {
+ Log("SHUTTLE", "StoreOCDB - In TESTMODE - Simulating error while storing in the Grid");
+ Log(fCurrentDetector, "StoreOCDB - In TESTMODE - Simulating error while storing in the Grid");
+ return kFALSE;
+ }
+
+ Log("SHUTTLE","StoreOCDB - Storing OCDB data ...");
Bool_t resultCDB = StoreOCDB(fgkMainCDB);
- AliInfo("Storing reference data ...");
+ Log("SHUTTLE","StoreOCDB - Storing reference data ...");
Bool_t resultRef = StoreOCDB(fgkMainRefStorage);
-
- return resultCDB && resultRef;
+
+ Log("SHUTTLE","StoreOCDB - Storing reference files ...");
+ Bool_t resultRefFiles = CopyFilesToGrid("reference");
+
+ Bool_t resultMetadata = kTRUE;
+ if(fCurrentDetector == "GRP")
+ {
+ Log("StoreOCDB - SHUTTLE","Storing Run Metadata file ...");
+ resultMetadata = CopyFilesToGrid("metadata");
+ }
+
+ return resultCDB && resultRef && resultRefFiles && resultMetadata;
}
//______________________________________________________________________________________________
Log("SHUTTLE",
Form("StoreOCDB - Object <%s> successfully put into %s storage",
aLocId.ToString().Data(), type));
+ Log(fCurrentDetector.Data(),
+ Form("StoreOCDB - Object <%s> successfully put into %s storage",
+ aLocId.ToString().Data(), type));
}
// removing local filename...
TString filename;
localSto->IdToFilename(aLocId, filename);
- AliInfo(Form("Removing local file %s", filename.Data()));
+ Log("SHUTTLE", Form("StoreOCDB - Removing local file %s", filename.Data()));
RemoveFile(filename.Data());
continue;
} else {
Log("SHUTTLE",
Form("StoreOCDB - Grid %s storage of object <%s> failed",
type, aLocId.ToString().Data()));
+ Log(fCurrentDetector.Data(),
+ Form("StoreOCDB - Grid %s storage of object <%s> failed",
+ type, aLocId.ToString().Data()));
result = kFALSE;
}
}
return result;
}
+//______________________________________________________________________________________________
+Bool_t AliShuttle::CleanReferenceStorage(const char* detector)
+{
+ // clears the directory used to store reference files of a given subdetector
+
+ AliCDBManager* man = AliCDBManager::Instance();
+ AliCDBStorage* sto = man->GetStorage(fgkLocalRefStorage);
+ TString localBaseFolder = sto->GetBaseFolder();
+
+ TString targetDir = GetRefFilePrefix(localBaseFolder.Data(), detector);
+
+ Log("SHUTTLE", Form("CleanReferenceStorage - Cleaning %s", targetDir.Data()));
+
+ TString begin;
+ begin.Form("%d_", GetCurrentRun());
+
+ TSystemDirectory* baseDir = new TSystemDirectory("/", targetDir);
+ if (!baseDir)
+ return kTRUE;
+
+ TList* dirList = baseDir->GetListOfFiles();
+ delete baseDir;
+
+ if (!dirList) return kTRUE;
+
+ if (dirList->GetEntries() < 3)
+ {
+ delete dirList;
+ return kTRUE;
+ }
+
+ Int_t nDirs = 0, nDel = 0;
+ TIter dirIter(dirList);
+ TSystemFile* entry = 0;
+
+ Bool_t success = kTRUE;
+
+ while ((entry = dynamic_cast<TSystemFile*> (dirIter.Next())))
+ {
+ if (entry->IsDirectory())
+ continue;
+
+ TString fileName(entry->GetName());
+ if (!fileName.BeginsWith(begin))
+ continue;
+
+ nDirs++;
+
+ // delete file
+ Int_t result = gSystem->Unlink(fileName.Data());
+
+ if (result)
+ {
+ Log("SHUTTLE", Form("CleanReferenceStorage - Could not delete file %s!", fileName.Data()));
+ success = kFALSE;
+ } else {
+ nDel++;
+ }
+ }
+
+ if(nDirs > 0)
+ Log("SHUTTLE", Form("CleanReferenceStorage - %d (over %d) reference files in folder %s were deleted.",
+ nDel, nDirs, targetDir.Data()));
+
+
+ delete dirList;
+ return success;
+
+
+
+
+
+
+ Int_t result = gSystem->GetPathInfo(targetDir, 0, (Long64_t*) 0, 0, 0);
+ if (result == 0)
+ {
+ // delete directory
+ result = gSystem->Exec(Form("rm -rf %s", targetDir.Data()));
+ if (result != 0)
+ {
+ Log("SHUTTLE", Form("CleanReferenceStorage - Could not clean directory %s", targetDir.Data()));
+ return kFALSE;
+ }
+ }
+
+ result = gSystem->mkdir(targetDir, kTRUE);
+ if (result != 0)
+ {
+ Log("SHUTTLE", Form("CleanReferenceStorage - Error creating base directory %s", targetDir.Data()));
+ return kFALSE;
+ }
+
+ return kTRUE;
+}
+
+//______________________________________________________________________________________________
+Bool_t AliShuttle::StoreReferenceFile(const char* detector, const char* localFile, const char* gridFileName)
+{
+ //
+ // Stores reference file directly (without opening it). This function stores the file locally.
+ //
+ // The file is stored under the following location:
+ // <base folder of local reference storage>/<DET>/<RUN#>_<gridFileName>
+ // where <gridFileName> is the second parameter given to the function
+ //
+
+ if (fTestMode & kErrorStorage)
+ {
+ Log(fCurrentDetector, "StoreReferenceFile - In TESTMODE - Simulating error while storing locally");
+ return kFALSE;
+ }
+
+ AliCDBManager* man = AliCDBManager::Instance();
+ AliCDBStorage* sto = man->GetStorage(fgkLocalRefStorage);
+
+ TString localBaseFolder = sto->GetBaseFolder();
+
+ TString target = GetRefFilePrefix(localBaseFolder.Data(), detector);
+ target.Append(Form("/%d_%s", GetCurrentRun(), gridFileName));
+
+ return CopyFileLocally(localFile, target);
+}
+
+//______________________________________________________________________________________________
+Bool_t AliShuttle::StoreRunMetadataFile(const char* localFile, const char* gridFileName)
+{
+ //
+ // Stores Run metadata file to the Grid, in the run folder
+ //
+ // Only GRP can call this function.
+
+ if (fTestMode & kErrorStorage)
+ {
+ Log(fCurrentDetector, "StoreRunMetaDataFile - In TESTMODE - Simulating error while storing locally");
+ return kFALSE;
+ }
+
+ AliCDBManager* man = AliCDBManager::Instance();
+ AliCDBStorage* sto = man->GetStorage(fgkLocalRefStorage);
+
+ TString localBaseFolder = sto->GetBaseFolder();
+
+ // Build Run level folder
+ // folder = /alice/data/year/lhcPeriod/runNb/Raw
+
+
+ TString lhcPeriod = GetLHCPeriod();
+ if (lhcPeriod.Length() == 0)
+ {
+ Log("SHUTTLE","StoreRunMetaDataFile - LHCPeriod not found in logbook!");
+ return 0;
+ }
+
+ TString target = Form("%s/GRP/RunMetadata/alice/data/%d/%s/%09d/Raw/%s",
+ localBaseFolder.Data(), GetCurrentYear(),
+ lhcPeriod.Data(), GetCurrentRun(), gridFileName);
+
+ return CopyFileLocally(localFile, target);
+}
+
+//______________________________________________________________________________________________
+Bool_t AliShuttle::CopyFileLocally(const char* localFile, const TString& target)
+{
+ //
+ // Stores file locally. Called by StoreReferenceFile and StoreRunMetadataFile
+ // Files are temporarily stored in the local reference storage. When the preprocessor
+ // finishes, the Shuttle calls CopyFilesToGrid to transfer the files to AliEn
+ // (in reference or run level folders)
+ //
+
+ TString targetDir(target(0, target.Last('/')));
+
+ //try to open base dir folder, if it does not exist
+ void* dir = gSystem->OpenDirectory(targetDir.Data());
+ if (dir == NULL) {
+ if (gSystem->mkdir(targetDir.Data(), kTRUE)) {
+ Log("SHUTTLE", Form("StoreFileLocally - Can't open directory <%s>", targetDir.Data()));
+ return kFALSE;
+ }
+
+ } else {
+ gSystem->FreeDirectory(dir);
+ }
+
+ Int_t result = 0;
+
+ result = gSystem->GetPathInfo(localFile, 0, (Long64_t*) 0, 0, 0);
+ if (result)
+ {
+ Log("SHUTTLE", Form("StoreFileLocally - %s does not exist", localFile));
+ return kFALSE;
+ }
+
+ result = gSystem->GetPathInfo(target, 0, (Long64_t*) 0, 0, 0);
+ if (!result)
+ {
+ Log("SHUTTLE", Form("StoreFileLocally - target file %s already exist, removing...", target.Data()));
+ if (gSystem->Unlink(target.Data()))
+ {
+ Log("SHUTTLE", Form("StoreFileLocally - Could not remove existing target file %s!", target.Data()));
+ return kFALSE;
+ }
+ }
+
+ result = gSystem->CopyFile(localFile, target);
+
+ if (result == 0)
+ {
+ Log("SHUTTLE", Form("StoreFileLocally - File %s stored locally to %s", localFile, target.Data()));
+ return kTRUE;
+ }
+ else
+ {
+ Log("SHUTTLE", Form("StoreFileLocally - Could not store file %s to %s! Error code = %d",
+ localFile, target.Data(), result));
+ return kFALSE;
+ }
+
+
+
+}
+
+//______________________________________________________________________________________________
+Bool_t AliShuttle::CopyFilesToGrid(const char* type)
+{
+ //
+ // Transfers local files to the Grid. Local files can be reference files
+ // or run metadata file (from GRP only).
+ //
+ // According to the type (ref, metadata) the files are stored under the following location:
+ // ref --> <base folder of reference storage>/<DET>/<RUN#>_<gridFileName>
+ // metadata --> <run data folder>/<MetadataFileName>
+ //
+
+ AliCDBManager* man = AliCDBManager::Instance();
+ AliCDBStorage* sto = man->GetStorage(fgkLocalRefStorage);
+ if (!sto)
+ return kFALSE;
+ TString localBaseFolder = sto->GetBaseFolder();
+
+ TString dir;
+ TString alienDir;
+ TString begin;
+
+ if (strcmp(type, "reference") == 0)
+ {
+ dir = GetRefFilePrefix(localBaseFolder.Data(), fCurrentDetector.Data());
+ AliCDBStorage* gridSto = man->GetStorage(fgkMainRefStorage);
+ if (!gridSto)
+ return kFALSE;
+ TString gridBaseFolder = gridSto->GetBaseFolder();
+ alienDir = GetRefFilePrefix(gridBaseFolder.Data(), fCurrentDetector.Data());
+ begin = Form("%d_", GetCurrentRun());
+ }
+ else if (strcmp(type, "metadata") == 0)
+ {
+
+ TString lhcPeriod = GetLHCPeriod();
+
+ if (lhcPeriod.Length() == 0)
+ {
+ Log("SHUTTLE","CopyFilesToGrid - LHCPeriod not found in logbook!");
+ return 0;
+ }
+
+ dir = Form("%s/GRP/RunMetadata/alice/data/%d/%s/%09d/Raw",
+ localBaseFolder.Data(), GetCurrentYear(),
+ lhcPeriod.Data(), GetCurrentRun());
+ alienDir = dir(dir.Index("/alice/data/"), dir.Length());
+
+ begin = "";
+ }
+ else
+ {
+ Log("SHUTTLE", "CopyFilesToGrid - Unexpected: type label must be reference or metadata!");
+ return kFALSE;
+ }
+
+ TSystemDirectory* baseDir = new TSystemDirectory("/", dir);
+ if (!baseDir)
+ return kTRUE;
+
+ TList* dirList = baseDir->GetListOfFiles();
+ delete baseDir;
+
+ if (!dirList) return kTRUE;
+
+ if (dirList->GetEntries() < 3)
+ {
+ delete dirList;
+ return kTRUE;
+ }
+
+ if (!gGrid)
+ {
+ Log("SHUTTLE", "CopyFilesToGrid - Connection to Grid failed: Cannot continue!");
+ delete dirList;
+ return kFALSE;
+ }
+
+ Int_t nDirs = 0, nTransfer = 0;
+ TIter dirIter(dirList);
+ TSystemFile* entry = 0;
+
+ Bool_t success = kTRUE;
+ Bool_t first = kTRUE;
+
+ while ((entry = dynamic_cast<TSystemFile*> (dirIter.Next())))
+ {
+ if (entry->IsDirectory())
+ continue;
+
+ TString fileName(entry->GetName());
+ if (!fileName.BeginsWith(begin))
+ continue;
+
+ nDirs++;
+
+ if (first)
+ {
+ first = kFALSE;
+ // check that folder exists, otherwise create it
+ TGridResult* result = gGrid->Ls(alienDir.Data(), "a");
+
+ if (!result)
+ {
+ delete dirList;
+ return kFALSE;
+ }
+
+ if (!result->GetFileName(1)) // TODO: It looks like element 0 is always 0!!
+ {
+ // TODO It does not work currently! Bug in TAliEn::Mkdir
+ // TODO Manually fixed in local root v5-16-00
+ if (!gGrid->Mkdir(alienDir.Data(),"-p",0))
+ {
+ Log("SHUTTLE", Form("CopyFilesToGrid - Cannot create directory %s",
+ alienDir.Data()));
+ delete dirList;
+ return kFALSE;
+ } else {
+ Log("SHUTTLE",Form("CopyFilesToGrid - Folder %s created", alienDir.Data()));
+ }
+
+ } else {
+ Log("SHUTTLE",Form("CopyFilesToGrid - Folder %s found", alienDir.Data()));
+ }
+ }
+
+ TString fullLocalPath;
+ fullLocalPath.Form("%s/%s", dir.Data(), fileName.Data());
+
+ TString fullGridPath;
+ fullGridPath.Form("alien://%s/%s", alienDir.Data(), fileName.Data());
+
+ Bool_t result = TFile::Cp(fullLocalPath, fullGridPath);
+
+ if (result)
+ {
+ Log("SHUTTLE", Form("CopyFilesToGrid - Copying local file %s to %s succeeded!",
+ fullLocalPath.Data(), fullGridPath.Data()));
+ RemoveFile(fullLocalPath);
+ nTransfer++;
+ }
+ else
+ {
+ Log("SHUTTLE", Form("CopyFilesToGrid - Copying local file %s to %s FAILED!",
+ fullLocalPath.Data(), fullGridPath.Data()));
+ success = kFALSE;
+ }
+ }
+
+ Log("SHUTTLE", Form("CopyFilesToGrid - %d (over %d) files in folder %s copied to Grid.",
+ nTransfer, nDirs, dir.Data()));
+
+
+ delete dirList;
+ return success;
+}
+
+//______________________________________________________________________________________________
+const char* AliShuttle::GetRefFilePrefix(const char* base, const char* detector)
+{
+ //
+ // Get folder name of reference files
+ //
+
+ TString offDetStr(GetOfflineDetName(detector));
+ TString dir;
+ if (offDetStr == "ITS" || offDetStr == "MUON" || offDetStr == "PHOS")
+ {
+ dir.Form("%s/%s/%s", base, offDetStr.Data(), detector);
+ } else {
+ dir.Form("%s/%s", base, offDetStr.Data());
+ }
+
+ return dir.Data();
+
+
+}
+
//______________________________________________________________________________________________
void AliShuttle::CleanLocalStorage(const TString& uri)
{
-// Called in case the preprocessor is declared failed. Remove remaining objects from the local storages.
+ //
+ // Called in case the preprocessor is declared failed. Remove remaining objects from the local storages.
+ //
const char* type = 0;
if(uri == fgkLocalCDB) {
type = "OCDB";
} else if(uri == fgkLocalRefStorage) {
- type = "reference";
+ type = "Reference";
} else {
AliError(Form("Invalid storage URI: %s", uri.Data()));
return;
}
TString filename(Form("%s/%s/*/Run*_v%d_s*.root",
- localSto->GetBaseFolder().Data(), fCurrentDetector.Data(), GetCurrentRun()));
+ localSto->GetBaseFolder().Data(), GetOfflineDetName(fCurrentDetector.Data()), GetCurrentRun()));
- AliInfo(Form("filename = %s", filename.Data()));
+ AliDebug(2, Form("filename = %s", filename.Data()));
- AliInfo(Form("Removing remaining local files from run %d and detector %s ...",
+ Log("SHUTTLE", Form("Removing remaining local files for run %d and detector %s ...",
GetCurrentRun(), fCurrentDetector.Data()));
RemoveFile(filename.Data());
//______________________________________________________________________________________________
void AliShuttle::RemoveFile(const char* filename)
{
-// removes local file
+ //
+ // removes local file
+ //
TString command(Form("rm -f %s", filename));
//______________________________________________________________________________________________
AliShuttleStatus* AliShuttle::ReadShuttleStatus()
{
-// Reads the AliShuttleStatus from the CDB
+ //
+ // Reads the AliShuttleStatus from the CDB
+ //
if (fStatusEntry){
delete fStatusEntry;
//______________________________________________________________________________________________
Bool_t AliShuttle::WriteShuttleStatus(AliShuttleStatus* status)
{
-// writes the status for one subdetector
+ //
+ // writes the status for one subdetector
+ //
if (fStatusEntry){
delete fStatusEntry;
//______________________________________________________________________________________________
void AliShuttle::UpdateShuttleStatus(AliShuttleStatus::Status newStatus, Bool_t increaseCount)
{
- // changes the AliShuttleStatus for the given detector and run to the given status
+ //
+ // changes the AliShuttleStatus for the given detector and run to the given status
+ //
if (!fStatusEntry){
AliError("UNEXPECTED: fStatusEntry empty");
AliShuttleStatus* status = dynamic_cast<AliShuttleStatus*> (fStatusEntry->GetObject());
if (!status){
- Log("SHUTTLE", "UNEXPECTED: status could not be read from current CDB entry");
+ Log("SHUTTLE", "UpdateShuttleStatus - UNEXPECTED: status could not be read from current CDB entry");
return;
}
//______________________________________________________________________________________________
Bool_t AliShuttle::ContinueProcessing()
{
-// this function reads the AliShuttleStatus information from CDB and
-// checks if the processing should be continued
-// if yes it returns kTRUE and updates the AliShuttleStatus with nextStatus
+ // this function reads the AliShuttleStatus information from CDB and
+ // checks if the processing should be continued
+ // if yes it returns kTRUE and updates the AliShuttleStatus with nextStatus
if (!fConfig->HostProcessDetector(fCurrentDetector)) return kFALSE;
dynamic_cast<AliPreprocessor*> (fPreprocessorMap.GetValue(fCurrentDetector));
if (!aPreprocessor)
{
- AliInfo(Form("%s: no preprocessor registered", fCurrentDetector.Data()));
+ Log("SHUTTLE", Form("ContinueProcessing - %s: no preprocessor registered", fCurrentDetector.Data()));
return kFALSE;
}
fLogbookEntry->GetDetectorStatus(fCurrentDetector);
if(entryStatus != AliShuttleLogbookEntry::kUnprocessed) {
- AliInfo(Form("ContinueProcessing - %s is %s",
+ Log("SHUTTLE", Form("ContinueProcessing - %s is %s",
fCurrentDetector.Data(),
fLogbookEntry->GetDetectorStatusName(entryStatus)));
return kFALSE;
if (fConfig->StrictRunOrder(fCurrentDetector) &&
!fFirstUnprocessed[GetDetPos(fCurrentDetector)])
{
- Log("SHUTTLE", Form("ContinueProcessing - %s requires strict run ordering but this is not the first unprocessed run!"));
- return kFALSE;
+ if (fTestMode == kNone)
+ {
+ Log("SHUTTLE", Form("ContinueProcessing - %s requires strict run ordering"
+ " but this is not the first unprocessed run!"));
+ return kFALSE;
+ }
+ else
+ {
+ Log("SHUTTLE", Form("ContinueProcessing - In TESTMODE - "
+ "Although %s requires strict run ordering "
+ "and this is not the first unprocessed run, "
+ "the SHUTTLE continues"));
+ }
}
AliShuttleStatus* status = ReadShuttleStatus();
if (status->GetStatus() == AliShuttleStatus::kStoreError) {
Log("SHUTTLE",
- Form("ContinueProcessing - %s: Grid storage of one or more objects failed. Trying again now",
+ Form("ContinueProcessing - %s: Grid storage of one or more "
+ "objects failed. Trying again now",
fCurrentDetector.Data()));
- if(StoreOCDB()){
- Log("SHUTTLE", Form("ContinueProcessing - %s: all objects successfully stored into main storage",
+ UpdateShuttleStatus(AliShuttleStatus::kStoreStarted);
+ if (StoreOCDB()){
+ Log("SHUTTLE", Form("ContinueProcessing - %s: all objects "
+ "successfully stored into main storage",
fCurrentDetector.Data()));
UpdateShuttleStatus(AliShuttleStatus::kDone);
UpdateShuttleLogbook(fCurrentDetector.Data(), "DONE");
Log("SHUTTLE",
Form("ContinueProcessing - %s: Grid storage failed again",
fCurrentDetector.Data()));
- // trigger ML information manually because we do not had a status change
- SendMLInfo();
+ UpdateShuttleStatus(AliShuttleStatus::kStoreError);
}
return kFALSE;
}
// there may still be objects in local OCDB and reference storage
// and FXS databases may be not updated: do it now!
- CleanLocalStorage(fgkLocalCDB);
- CleanLocalStorage(fgkLocalRefStorage);
- UpdateTableFailCase();
+
+ // TODO Currently disabled, we want to keep files in case of failure!
+ // CleanLocalStorage(fgkLocalCDB);
+ // CleanLocalStorage(fgkLocalRefStorage);
+ // UpdateTableFailCase();
+
+ // Send mail to detector expert!
+ Log("SHUTTLE", Form("ContinueProcessing - Sending mail to %s expert...",
+ fCurrentDetector.Data()));
+ if (!SendMail())
+ Log("SHUTTLE", Form("ContinueProcessing - Could not send mail to %s expert",
+ fCurrentDetector.Data()));
} else {
Log("SHUTTLE", Form("ContinueProcessing - %s: restarting. "
"Aborted before with %s. Retry number %d.", fCurrentDetector.Data(),
status->GetStatusName(), status->GetCount()));
- UpdateShuttleStatus(AliShuttleStatus::kStarted, kTRUE);
+ Bool_t increaseCount = kTRUE;
+ if (status->GetStatus() == AliShuttleStatus::kDCSError ||
+ status->GetStatus() == AliShuttleStatus::kDCSStarted)
+ increaseCount = kFALSE;
+
+ UpdateShuttleStatus(AliShuttleStatus::kStarted, increaseCount);
cont = kTRUE;
}
- // Send mail to detector expert!
- AliInfo(Form("Sending mail to %s expert...", fCurrentDetector.Data()));
- if (!SendMail())
- Log("SHUTTLE", Form("ContinueProcessing - Could not send mail to %s expert",
- fCurrentDetector.Data()));
-
return cont;
}
// Returns kFALSE in case of error occured and kTRUE otherwise
//
- if(!entry) return kFALSE;
+ if (!entry) return kFALSE;
fLogbookEntry = entry;
- if (fLogbookEntry->IsDone())
- {
- Log("SHUTTLE","Process - Shuttle is already DONE. Updating logbook");
- UpdateShuttleLogbook("shuttle_done");
- fLogbookEntry = 0;
- return kTRUE;
- }
+ Log("SHUTTLE", Form("\t\t\t^*^*^*^*^*^*^*^*^*^*^*^* run %d: START ^*^*^*^*^*^*^*^*^*^*^*^*",
+ GetCurrentRun()));
// create ML instance that monitors this run
- fMonaLisa = new TMonaLisaWriter(Form("%d", GetCurrentRun()), "SHUTTLE", "aliendb1.cern.ch");
+ fMonaLisa = new TMonaLisaWriter(Form("%d", GetCurrentRun()), fConfig->GetMonitorTable(),
+ fConfig->GetMonitorHost());
// disable monitoring of other parameters that come e.g. from TFile
gMonitoringWriter = 0;
- AliInfo(Form("\n\n \t\t\t^*^*^*^*^*^*^*^*^*^*^*^* run %d: START ^*^*^*^*^*^*^*^*^*^*^*^* \n",
- GetCurrentRun()));
-
- // Set run type from run type logbook into current fLogbookEntry
- SetRunType();
-
// Send the information to ML
TMonaLisaText mlStatus("SHUTTLE_status", "Processing");
+ TMonaLisaText mlRunType("SHUTTLE_runtype", Form("%s (%s)", entry->GetRunType(), entry->GetRunParameter("log")));
TList mlList;
mlList.Add(&mlStatus);
+ mlList.Add(&mlRunType);
fMonaLisa->SendParameters(&mlList);
+ if (fLogbookEntry->IsDone())
+ {
+ Log("SHUTTLE","Process - Shuttle is already DONE. Updating logbook");
+ UpdateShuttleLogbook("shuttle_done");
+ fLogbookEntry = 0;
+ return kTRUE;
+ }
+
+ // read test mode if flag is set
+ if (fReadTestMode)
+ {
+ fTestMode = kNone;
+ TString logEntry(entry->GetRunParameter("log"));
+ //printf("log entry = %s\n", logEntry.Data());
+ TString searchStr("Testmode: ");
+ Int_t pos = logEntry.Index(searchStr.Data());
+ //printf("%d\n", pos);
+ if (pos >= 0)
+ {
+ TSubString subStr = logEntry(pos + searchStr.Length(), logEntry.Length());
+ //printf("%s\n", subStr.String().Data());
+ TString newStr(subStr.Data());
+ TObjArray* token = newStr.Tokenize(' ');
+ if (token)
+ {
+ //token->Print();
+ TObjString* tmpStr = dynamic_cast<TObjString*> (token->First());
+ if (tmpStr)
+ {
+ Int_t testMode = tmpStr->String().Atoi();
+ if (testMode > 0)
+ {
+ Log("SHUTTLE", Form("Process - Enabling test mode %d", testMode));
+ SetTestMode((TestMode) testMode);
+ }
+ }
+ delete token;
+ }
+ }
+ }
+
fLogbookEntry->Print("all");
// Initialization
Bool_t hasError = kFALSE;
+ // Set the CDB and Reference folders according to the year and LHC period
+ TString lhcPeriod(GetLHCPeriod());
+ if (lhcPeriod.Length() == 0)
+ {
+ Log("SHUTTLE","StoreRunMetaDataFile - LHCPeriod not found in logbook!");
+ return 0;
+ }
+
+ if (fgkMainCDB.Length() == 0)
+ fgkMainCDB = Form("alien://folder=/alice/data/%d/%s/OCDB?user=alidaq?cacheFold=/tmp/OCDBCache",
+ GetCurrentYear(), lhcPeriod.Data());
+
+ if (fgkMainRefStorage.Length() == 0)
+ fgkMainRefStorage = Form("alien://folder=/alice/data/%d/%s/Reference?user=alidaq?cacheFold=/tmp/OCDBCache",
+ GetCurrentYear(), lhcPeriod.Data());
+
AliCDBStorage *mainCDBSto = AliCDBManager::Instance()->GetStorage(fgkMainCDB);
if(mainCDBSto) mainCDBSto->QueryCDB(GetCurrentRun());
AliCDBStorage *mainRefSto = AliCDBManager::Instance()->GetStorage(fgkMainRefStorage);
if (ContinueProcessing() == kFALSE) continue;
- AliInfo(Form("\n\n \t\t\t****** run %d - %s: START ******",
+ Log("SHUTTLE", Form("\t\t\t****** run %d - %s: START ******",
GetCurrentRun(), aDetector->GetName()));
for(Int_t iSys=0;iSys<3;iSys++) fFXSCalled[iSys]=kFALSE;
- Log(fCurrentDetector.Data(), "Starting processing");
+ Log(fCurrentDetector.Data(), "Process - Starting processing");
Int_t pid = fork();
if (pid < 0)
{
- Log("SHUTTLE", "ERROR: Forking failed");
+ Log("SHUTTLE", "Process - ERROR: Forking failed");
}
else if (pid > 0)
{
// parent
- AliInfo(Form("In parent process of %d - %s: Starting monitoring",
+ Log("SHUTTLE", Form("Process - In parent process of %d - %s: Starting monitoring",
GetCurrentRun(), aDetector->GetName()));
Long_t begin = time(0);
if (expiredTime > fConfig->GetPPTimeOut())
{
- Log("SHUTTLE", Form("%s: Process time out. Run time: %d seconds. Killing...",
- fCurrentDetector.Data(), expiredTime));
+ TString tmp;
+ tmp.Form("Process - Process of %s time out. "
+ "Run time: %d seconds. Killing...",
+ fCurrentDetector.Data(), expiredTime);
+ Log("SHUTTLE", tmp);
+ Log(fCurrentDetector, tmp);
kill(pid, 9);
}
else
{
- if (expiredTime % 60 == 0)
- Log("SHUTTLE", Form("Checked process. Run time: %d seconds.",
- expiredTime));
gSystem->Sleep(1000);
+
+ TString checkStr;
+ checkStr.Form("ps -o vsize --pid %d | tail -n 1", pid);
+ FILE* pipe = gSystem->OpenPipe(checkStr, "r");
+ if (!pipe)
+ {
+ Log("SHUTTLE", Form("Process - Error: "
+ "Could not open pipe to %s", checkStr.Data()));
+ continue;
+ }
+
+ char buffer[100];
+ if (!fgets(buffer, 100, pipe))
+ {
+ Log("SHUTTLE", "Process - Error: ps did not return anything");
+ gSystem->ClosePipe(pipe);
+ continue;
+ }
+ gSystem->ClosePipe(pipe);
+
+ //Log("SHUTTLE", Form("ps returned %s", buffer));
+
+ Int_t mem = 0;
+ if ((sscanf(buffer, "%d\n", &mem) != 1) || !mem)
+ {
+ Log("SHUTTLE", "Process - Error: Could not parse output of ps");
+ continue;
+ }
+
+ if (expiredTime % 60 == 0)
+ Log("SHUTTLE", Form("Process - %s: Checking process. "
+ "Run time: %d seconds - Memory consumption: %d KB",
+ fCurrentDetector.Data(), expiredTime, mem));
+
+ if (mem > fConfig->GetPPMaxMem())
+ {
+ TString tmp;
+ tmp.Form("Process - Process exceeds maximum allowed memory "
+ "(%d KB > %d KB). Killing...",
+ mem, fConfig->GetPPMaxMem());
+ Log("SHUTTLE", tmp);
+ Log(fCurrentDetector, tmp);
+
+ kill(pid, 9);
+
+ UpdateShuttleStatus(AliShuttleStatus::kPPOutOfMemory);
+ hasError = kTRUE;
+
+ gSystem->Sleep(1000);
+ }
}
}
- AliInfo(Form("In parent process of %d - %s: Client has terminated.",
+ Log("SHUTTLE", Form("Process - In parent process of %d - %s: Client has terminated.",
GetCurrentRun(), aDetector->GetName()));
if (WIFEXITED(status))
{
Int_t returnCode = WEXITSTATUS(status);
- Log("SHUTTLE", Form("%s: the return code is %d", fCurrentDetector.Data(),
+ Log("SHUTTLE", Form("Process - %s: the return code is %d", fCurrentDetector.Data(),
returnCode));
- if (returnCode != 0) hasError = kTRUE;
+ if (returnCode == 0) hasError = kTRUE;
}
}
else if (pid == 0)
{
// client
- AliInfo(Form("In client process of %d - %s", GetCurrentRun(), aDetector->GetName()));
-
- UInt_t returnCode = ProcessCurrentDetector(); // 0 means success
+ Log("SHUTTLE", Form("Process - In client process of %d - %s", GetCurrentRun(),
+ aDetector->GetName()));
+ Log("SHUTTLE", Form("Process - Redirecting output to %s log",fCurrentDetector.Data()));
- if (returnCode > 0) // TODO: returnCode>0 means Preprocessor ERROR! Preprocessors should follow this!
+ if ((freopen(GetLogFileName(fCurrentDetector), "a", stdout)) == 0)
{
- if (returnCode == kDCSErrCode)
- {
- AliInfo(Form("\n \t\t\t****** run %d - %s: DCS ERROR ****** \n\n",
- GetCurrentRun(), aDetector->GetName()));
- } else {
-
- AliInfo(Form("\n \t\t\t****** run %d - %s: PREPROCESSOR ERROR ****** \n\n",
- GetCurrentRun(), aDetector->GetName()));
- }
- } else { // Preprocessor finished successfully!
-
+ Log("SHUTTLE", "Process - Could not freopen stdout");
+ }
+ else
+ {
+ fOutputRedirected = kTRUE;
+ if ((dup2(fileno(stdout), fileno(stderr))) < 0)
+ Log("SHUTTLE", "Process - Could not redirect stderr");
+
+ }
+
+ TString wd = gSystem->WorkingDirectory();
+ TString tmpDir = Form("%s/%s_%d_process", GetShuttleTempDir(),
+ fCurrentDetector.Data(), GetCurrentRun());
+
+ Int_t result = gSystem->GetPathInfo(tmpDir.Data(), 0, (Long64_t*) 0, 0, 0);
+ if (!result) // temp dir already exists!
+ {
+ Log(fCurrentDetector.Data(),
+ Form("Process - %s dir already exists! Removing...", tmpDir.Data()));
+ gSystem->Exec(Form("rm -rf %s",tmpDir.Data()));
+ }
+
+ if (gSystem->mkdir(tmpDir.Data(), 1))
+ {
+ Log(fCurrentDetector.Data(), "Process - could not make temp directory!!");
+ gSystem->Exit(1);
+ }
+
+ if (!gSystem->ChangeDirectory(tmpDir.Data()))
+ {
+ Log(fCurrentDetector.Data(), "Process - could not change directory!!");
+ gSystem->Exit(1);
+ }
+
+ Bool_t success = ProcessCurrentDetector();
+
+ gSystem->ChangeDirectory(wd.Data());
+
+ if (success) // Preprocessor finished successfully!
+ {
+ // remove temporary folder
+ gSystem->Exec(Form("rm -rf %s",tmpDir.Data()));
+
// Update time_processed field in FXS DB
if (UpdateTable() == kFALSE)
- Log("SHUTTLE", Form("Process - %s: Could not update FXS databases!"));
+ Log("SHUTTLE", Form("Process - %s: Could not update FXS databases!",
+ fCurrentDetector.Data()));
// Transfer the data from local storage to main storage (Grid)
UpdateShuttleStatus(AliShuttleStatus::kStoreStarted);
if (StoreOCDB() == kFALSE)
{
- AliInfo(Form("\n \t\t\t****** run %d - %s: STORAGE ERROR ****** \n\n",
+ Log("SHUTTLE",
+ Form("\t\t\t****** run %d - %s: STORAGE ERROR ******",
GetCurrentRun(), aDetector->GetName()));
UpdateShuttleStatus(AliShuttleStatus::kStoreError);
- returnCode = kStorErrCode;
+ success = kFALSE;
} else {
- AliInfo(Form("\n \t\t\t****** run %d - %s: DONE ****** \n\n",
+ Log("SHUTTLE",
+ Form("\t\t\t****** run %d - %s: DONE ******",
GetCurrentRun(), aDetector->GetName()));
UpdateShuttleStatus(AliShuttleStatus::kDone);
+ UpdateShuttleLogbook(fCurrentDetector, "DONE");
}
+ } else
+ {
+ Log("SHUTTLE",
+ Form("\t\t\t****** run %d - %s: PP ERROR ******",
+ GetCurrentRun(), aDetector->GetName()));
}
for (UInt_t iSys=0; iSys<3; iSys++)
if (fFXSCalled[iSys]) fFXSlist[iSys].Clear();
}
- AliInfo(Form("Client process of %d - %s is exiting now with %d.",
- GetCurrentRun(), aDetector->GetName(), returnCode));
+ Log("SHUTTLE", Form("Process - Client process of %d - %s is exiting now with %d.",
+ GetCurrentRun(), aDetector->GetName(), success));
// the client exits here
- gSystem->Exit(returnCode);
+ gSystem->Exit(success);
AliError("We should never get here!!!");
}
}
- AliInfo(Form("\n\n \t\t\t^*^*^*^*^*^*^*^*^*^*^*^* run %d: FINISH ^*^*^*^*^*^*^*^*^*^*^*^* \n",
+ Log("SHUTTLE", Form("\t\t\t^*^*^*^*^*^*^*^*^*^*^*^* run %d: FINISH ^*^*^*^*^*^*^*^*^*^*^*^*",
GetCurrentRun()));
//check if shuttle is done for this run, if so update logbook
}
//______________________________________________________________________________________________
-UInt_t AliShuttle::ProcessCurrentDetector()
+Bool_t AliShuttle::ProcessCurrentDetector()
{
//
// Makes data retrieval just for a specific detector (fCurrentDetector).
// Threre should be a configuration for this detector.
- AliInfo(Form("Retrieving values for %s, run %d", fCurrentDetector.Data(), GetCurrentRun()));
-
- TMap dcsMap;
- dcsMap.SetOwner(1);
+ Log("SHUTTLE", Form("ProcessCurrentDetector - Retrieving values for %s, run %d",
+ fCurrentDetector.Data(), GetCurrentRun()));
- Bool_t aDCSError = kFALSE;
+ TString wd = gSystem->WorkingDirectory();
+
+ if (!CleanReferenceStorage(fCurrentDetector.Data()))
+ return kFALSE;
+
+ gSystem->ChangeDirectory(wd.Data());
+
+ TMap* dcsMap = new TMap();
// call preprocessor
AliPreprocessor* aPreprocessor =
Bool_t processDCS = aPreprocessor->ProcessDCS();
- // TODO Test only... I've added a flag that allows to
- // exclude DCS archive DB query
- if (!processDCS || !fgkProcessDCS)
+ if (!processDCS)
+ {
+ Log(fCurrentDetector, "ProcessCurrentDetector -"
+ " The preprocessor requested to skip the retrieval of DCS values");
+ }
+ else if (fTestMode & kSkipDCS)
{
- AliInfo("Skipping DCS processing!");
- aDCSError = kFALSE;
+ Log(fCurrentDetector, "ProcessCurrentDetector - In TESTMODE: Skipping DCS processing");
+ }
+ else if (fTestMode & kErrorDCS)
+ {
+ Log(fCurrentDetector, "ProcessCurrentDetector - In TESTMODE: Simulating DCS error");
+ UpdateShuttleStatus(AliShuttleStatus::kDCSStarted);
+ UpdateShuttleStatus(AliShuttleStatus::kDCSError);
+ delete dcsMap;
+ return kFALSE;
} else {
UpdateShuttleStatus(AliShuttleStatus::kDCSStarted);
- TString host(fConfig->GetDCSHost(fCurrentDetector));
- Int_t port = fConfig->GetDCSPort(fCurrentDetector);
-
- // Retrieval of Aliases
- TObjString* anAlias = 0;
- Int_t iAlias = 1;
- Int_t nTotAliases= ((TMap*)fConfig->GetDCSAliases(fCurrentDetector))->GetEntries();
- TIter iterAliases(fConfig->GetDCSAliases(fCurrentDetector));
- while ((anAlias = (TObjString*) iterAliases.Next()))
+ // Query DCS archive
+ Int_t nServers = fConfig->GetNServers(fCurrentDetector);
+
+ for (int iServ=0; iServ<nServers; iServ++)
{
- TObjArray *valueSet = new TObjArray();
- valueSet->SetOwner(1);
-
- if (((iAlias-1) % 500) == 0 || iAlias == nTotAliases)
- AliInfo(Form("Querying DCS archive: alias %s (%d of %d)",
- anAlias->GetName(), iAlias++, nTotAliases));
- aDCSError = (GetValueSet(host, port, anAlias->String(), valueSet, kAlias) == 0);
-
- if(!aDCSError)
+
+ TString host(fConfig->GetDCSHost(fCurrentDetector, iServ));
+ Int_t port = fConfig->GetDCSPort(fCurrentDetector, iServ);
+ Int_t multiSplit = fConfig->GetMultiSplit(fCurrentDetector, iServ);
+
+ Log(fCurrentDetector, Form("ProcessCurrentDetector -"
+ " Querying DCS Amanda server %s:%d (%d of %d)",
+ host.Data(), port, iServ+1, nServers));
+
+ TMap* aliasMap = 0;
+ TMap* dpMap = 0;
+
+ if (fConfig->GetDCSAliases(fCurrentDetector, iServ)->GetEntries() > 0)
{
- dcsMap.Add(anAlias->Clone(), valueSet);
- } else {
- Log(fCurrentDetector,
- Form("ProcessCurrentDetector - Error while retrieving alias %s",
- anAlias->GetName()));
- UpdateShuttleStatus(AliShuttleStatus::kDCSError);
- dcsMap.DeleteAll();
- return kDCSErrCode;
+ aliasMap = GetValueSet(host, port,
+ fConfig->GetDCSAliases(fCurrentDetector, iServ),
+ kAlias, multiSplit);
+ if (!aliasMap)
+ {
+ Log(fCurrentDetector,
+ Form("ProcessCurrentDetector -"
+ " Error retrieving DCS aliases from server %s."
+ " Sending mail to DCS experts!", host.Data()));
+ UpdateShuttleStatus(AliShuttleStatus::kDCSError);
+
+ if (!SendMailToDCS())
+ Log("SHUTTLE", Form("ProcessCurrentDetector - Could not send mail to DCS experts!"));
+
+ delete dcsMap;
+ return kFALSE;
+ }
}
- }
-
- // Retrieval of Data Points
- TObjString* aDP = 0;
- Int_t iDP = 0;
- Int_t nTotDPs= ((TMap*)fConfig->GetDCSDataPoints(fCurrentDetector))->GetEntries();
- TIter iterDP(fConfig->GetDCSDataPoints(fCurrentDetector));
- while ((aDP = (TObjString*) iterDP.Next()))
- {
- TObjArray *valueSet = new TObjArray();
- valueSet->SetOwner(1);
- if (((iDP-1) % 500) == 0 || iDP == nTotDPs)
- AliInfo(Form("Querying DCS archive: DP %s (%d of %d)",
- aDP->GetName(), iDP++, nTotDPs));
- aDCSError = (GetValueSet(host, port, aDP->String(), valueSet, kDP) == 0);
-
- if(!aDCSError)
+
+ if (fConfig->GetDCSDataPoints(fCurrentDetector, iServ)->GetEntries() > 0)
{
- dcsMap.Add(aDP->Clone(), valueSet);
- } else {
- Log(fCurrentDetector,
- Form("ProcessCurrentDetector - Error while retrieving data point %s",
- aDP->GetName()));
- UpdateShuttleStatus(AliShuttleStatus::kDCSError);
- dcsMap.DeleteAll();
- return kDCSErrCode;
+ dpMap = GetValueSet(host, port,
+ fConfig->GetDCSDataPoints(fCurrentDetector, iServ),
+ kDP, multiSplit);
+ if (!dpMap)
+ {
+ Log(fCurrentDetector,
+ Form("ProcessCurrentDetector -"
+ " Error retrieving DCS data points from server %s."
+ " Sending mail to DCS experts!", host.Data()));
+ UpdateShuttleStatus(AliShuttleStatus::kDCSError);
+
+ if (!SendMailToDCS())
+ Log("SHUTTLE", Form("ProcessCurrentDetector - Could not send mail to DCS experts!"));
+
+ if (aliasMap) delete aliasMap;
+ delete dcsMap;
+ return kFALSE;
+ }
+ }
+
+ // merge aliasMap and dpMap into dcsMap
+ if(aliasMap) {
+ TIter iter(aliasMap);
+ TObjString* key = 0;
+ while ((key = (TObjString*) iter.Next()))
+ dcsMap->Add(key, aliasMap->GetValue(key->String()));
+
+ aliasMap->SetOwner(kFALSE);
+ delete aliasMap;
+ }
+
+ if(dpMap) {
+ TIter iter(dpMap);
+ TObjString* key = 0;
+ while ((key = (TObjString*) iter.Next()))
+ dcsMap->Add(key, dpMap->GetValue(key->String()));
+
+ dpMap->SetOwner(kFALSE);
+ delete dpMap;
}
}
}
-
+
+ // save map into file, to help debugging in case of preprocessor error
+ TFile* f = TFile::Open("DCSMap.root","recreate");
+ f->cd();
+ dcsMap->Write("DCSMap", TObject::kSingleKey);
+ f->Close();
+ delete f;
+
// DCS Archive DB processing successful. Call Preprocessor!
UpdateShuttleStatus(AliShuttleStatus::kPPStarted);
- UInt_t returnValue = aPreprocessor->Process(&dcsMap);
+ UInt_t returnValue = aPreprocessor->Process(dcsMap);
if (returnValue > 0) // Preprocessor error!
{
+ Log(fCurrentDetector, Form("ProcessCurrentDetector - "
+ "Preprocessor failed. Process returned %d.", returnValue));
UpdateShuttleStatus(AliShuttleStatus::kPPError);
+ dcsMap->DeleteAll();
+ delete dcsMap;
+ return kFALSE;
+ }
+
+ // preprocessor ok!
+ UpdateShuttleStatus(AliShuttleStatus::kPPDone);
+ Log(fCurrentDetector, Form("ProcessCurrentDetector - %s preprocessor returned success",
+ fCurrentDetector.Data()));
- } else { // preprocessor ok!
- UpdateShuttleStatus(AliShuttleStatus::kPPDone);
- UpdateShuttleLogbook(fCurrentDetector, "DONE");
- Log("SHUTTLE", Form("ProcessCurrentDetector - %s preprocessor returned success",
- fCurrentDetector.Data()));
- }
-
- dcsMap.DeleteAll();
+ dcsMap->DeleteAll();
+ delete dcsMap;
- return returnValue;
+ return kTRUE;
}
//______________________________________________________________________________________________
Bool_t AliShuttle::QueryShuttleLogbook(const char* whereClause,
TObjArray& entries)
{
-// Query DAQ's Shuttle logbook and fills detector status object.
-// Call QueryRunParameters to query DAQ logbook for run parameters.
+ // Query DAQ's Shuttle logbook and fills detector status object.
+ // Call QueryRunParameters to query DAQ logbook for run parameters.
+ //
entries.SetOwner(1);
AliDebug(2,Form("Query = %s", sqlQuery.Data()));
if(aResult->GetRowCount() == 0) {
-// if(sqlQuery.EndsWith("where shuttle_done=0 order by run")){
-// Log("SHUTTLE", "QueryShuttleLogbook - All runs in Shuttle Logbook are already DONE");
-// delete aResult;
-// return kTRUE;
-// } else {
- AliInfo("No entries in Shuttle Logbook match request");
- delete aResult;
- return kTRUE;
-// }
+ Log("SHUTTLE", "No entries in Shuttle Logbook match request");
+ delete aResult;
+ return kTRUE;
}
// TODO Check field count!
- const UInt_t nCols = 22;
+ const UInt_t nCols = 23;
if (aResult->GetFieldCount() != (Int_t) nCols) {
- AliError("Invalid SQL result field number!");
+ Log("SHUTTLE", "Invalid SQL result field number!");
delete aResult;
return kFALSE;
}
delete aRow;
}
-// if(sqlQuery.EndsWith("where shuttle_done=0 order by run"))
-// Log("SHUTTLE", Form("QueryShuttleLogbook - Found %d unprocessed runs in Shuttle Logbook",
-// entries.GetEntriesFast()));
delete aResult;
return kTRUE;
}
TSQLResult* aResult = fServer[3]->Query(sqlQuery);
if (!aResult) {
- AliError(Form("Can't execute query <%s>!", sqlQuery.Data()));
+ Log("SHUTTLE", Form("Can't execute query <%s>!", sqlQuery.Data()));
return 0;
}
}
if (aResult->GetRowCount() > 1) {
- AliError(Form("More than one entry in DAQ Logbook for run %d. Skipping", run));
+ Log("SHUTTLE", Form("QueryRunParameters - UNEXPECTED: "
+ "more than one entry in DAQ Logbook for run %d!", run));
delete aResult;
return 0;
}
TSQLRow* aRow = aResult->Next();
if (!aRow)
{
- AliError(Form("Could not retrieve row for run %d. Skipping", run));
+ Log("SHUTTLE", Form("QueryRunParameters - Could not retrieve row for run %d. Skipping", run));
delete aResult;
return 0;
}
}
//______________________________________________________________________________________________
-Bool_t AliShuttle::GetValueSet(const char* host, Int_t port, const char* entry,
- TObjArray* valueSet, DCSType type)
+TMap* AliShuttle::GetValueSet(const char* host, Int_t port, const TSeqCollection* entries,
+ DCSType type, Int_t multiSplit)
{
-// Retrieve all "entry" data points from the DCS server
-// host, port: TSocket connection parameters
-// entry: name of the alias or data point
-// valueSet: array of retrieved AliDCSValue's
-// type: kAlias or kDP
-
- AliDCSClient client(host, port, fTimeout, fRetries);
- if (!client.IsConnected())
- {
- return kFALSE;
- }
-
- Int_t result=0;
+ // Retrieve all "entry" data points from the DCS server
+ // host, port: TSocket connection parameters
+ // entries: list of name of the alias or data point
+ // type: kAlias or kDP
+ // returns TMap of values, 0 when failure
+
+ AliDCSClient client(host, port, fTimeout, fRetries, multiSplit);
+ TMap* result = 0;
if (type == kAlias)
{
- result = client.GetAliasValues(entry,
- GetCurrentStartTime(), GetCurrentEndTime(), valueSet);
- } else
- if (type == kDP)
+ result = client.GetAliasValues(entries, GetCurrentStartTime(),
+ GetCurrentEndTime());
+ }
+ else if (type == kDP)
{
- result = client.GetDPValues(entry,
- GetCurrentStartTime(), GetCurrentEndTime(), valueSet);
+ result = client.GetDPValues(entries, GetCurrentStartTime(),
+ GetCurrentEndTime());
}
- if (result < 0)
+ if (result == 0)
{
- Log(fCurrentDetector.Data(), Form("GetValueSet - Can't get '%s'! Reason: %s",
- entry, AliDCSClient::GetErrorString(result)));
-
- if (result == AliDCSClient::fgkServerError)
- {
- Log(fCurrentDetector.Data(), Form("GetValueSet - Server error: %s",
+ Log(fCurrentDetector.Data(), Form("GetValueSet - Can't get entries! Reason: %s",
+ client.GetErrorString(client.GetResultErrorCode())));
+ if (client.GetResultErrorCode() == AliDCSClient::fgkServerError)
+ Log(fCurrentDetector.Data(), Form("GetValueSet - Server error code: %s",
client.GetServerError().Data()));
- }
- return kFALSE;
+ return 0;
}
-
- return kTRUE;
+
+ return result;
}
//______________________________________________________________________________________________
const char* AliShuttle::GetFile(Int_t system, const char* detector,
const char* id, const char* source)
{
-// Get calibration file from file exchange servers
-// First queris the FXS database for the file name, using the run, detector, id and source info
-// then calls RetrieveFile(filename) for actual copy to local disk
-// run: current run being processed (given by Logbook entry fLogbookEntry)
-// detector: the Preprocessor name
-// id: provided as a parameter by the Preprocessor
-// source: provided by the Preprocessor through GetFileSources function
-
+ // Get calibration file from file exchange servers
+ // First queris the FXS database for the file name, using the run, detector, id and source info
+ // then calls RetrieveFile(filename) for actual copy to local disk
+ // run: current run being processed (given by Logbook entry fLogbookEntry)
+ // detector: the Preprocessor name
+ // id: provided as a parameter by the Preprocessor
+ // source: provided by the Preprocessor through GetFileSources function
+
+ // check if test mode should simulate a FXS error
+ if (fTestMode & kErrorFXSFiles)
+ {
+ Log(detector, Form("GetFile - In TESTMODE - Simulating error while connecting to %s FXS", GetSystemName(system)));
+ return 0;
+ }
+
// check connection, in case connect
if (!Connect(system))
{
filePath.Data(), fileSize.Data(), fileChecksum.Data()));
// retrieved file is renamed to make it unique
- TString localFileName = Form("%s_%s_%d_%s_%s.shuttle",
- GetSystemName(system), detector, GetCurrentRun(), id, sourceName.Data());
+ TString localFileName = Form("%s/%s_%d_process/%s_%s_%d_%s_%s.shuttle",
+ GetShuttleTempDir(), detector, GetCurrentRun(),
+ GetSystemName(system), detector, GetCurrentRun(),
+ id, sourceName.Data());
// file retrieval from FXS
Log(detector, Form("GetFileName - Copy of file %s from %s FXS failed",
filePath.Data(), GetSystemName(system)));
continue;
- } else {
- AliInfo(Form("File %s copied from %s FXS into %s/%s",
- filePath.Data(), GetSystemName(system),
- GetShuttleTempDir(), localFileName.Data()));
- }
+ }
if (fileChecksum.Length()>0)
{
// compare md5sum of local file with the one stored in the FXS DB
- Int_t md5Comp = gSystem->Exec(Form("md5sum %s/%s |grep %s 2>&1 > /dev/null",
- GetShuttleTempDir(), localFileName.Data(), fileChecksum.Data()));
+ Int_t md5Comp = gSystem->Exec(Form("md5sum %s |grep %s 2>&1 > /dev/null",
+ localFileName.Data(), fileChecksum.Data()));
if (md5Comp != 0)
{
fFXSCalled[system]=kTRUE;
TObjString *fileParams = new TObjString(Form("%s#!?!#%s", id, sourceName.Data()));
- fFXSlist[system].Add(fileParams);
-
- static TString fullLocalFileName;
- fullLocalFileName = TString::Format("%s/%s", GetShuttleTempDir(), localFileName.Data());
-
- AliInfo(Form("fullLocalFileName = %s", fullLocalFileName.Data()));
-
- return fullLocalFileName.Data();
+ fFXSlist[system].Add(fileParams);
+ static TString staticLocalFileName;
+ staticLocalFileName.Form("%s", localFileName.Data());
+
+ Log(fCurrentDetector, Form("GetFile - Retrieved file with id %s and "
+ "source %s from %s to %s", id, source,
+ GetSystemName(system), localFileName.Data()));
+
+ return staticLocalFileName.Data();
}
//______________________________________________________________________________________________
Bool_t AliShuttle::RetrieveFile(UInt_t system, const char* fxsFileName, const char* localFileName)
{
-// Copies file from FXS to local Shuttle machine
+ //
+ // Copies file from FXS to local Shuttle machine
+ //
// check temp directory: trying to cd to temp; if it does not exist, create it
- AliDebug(2, Form("Copy file %s from %s FXS into %s/%s",
- GetSystemName(system), fxsFileName, GetShuttleTempDir(), localFileName));
+ AliDebug(2, Form("Copy file %s from %s FXS into %s",
+ GetSystemName(system), fxsFileName, localFileName));
+
+ TString tmpDir(localFileName);
+
+ tmpDir = tmpDir(0,tmpDir.Last('/'));
- void* dir = gSystem->OpenDirectory(GetShuttleTempDir());
- if (dir == NULL) {
- if (gSystem->mkdir(GetShuttleTempDir(), kTRUE)) {
- AliError(Form("Can't open directory <%s>", GetShuttleTempDir()));
+ Int_t noDir = gSystem->GetPathInfo(tmpDir.Data(), 0, (Long64_t*) 0, 0, 0);
+ if (noDir) // temp dir does not exists!
+ {
+ if (gSystem->mkdir(tmpDir.Data(), 1))
+ {
+ Log(fCurrentDetector.Data(), "RetrieveFile - could not make temp directory!!");
return kFALSE;
}
-
- } else {
- gSystem->FreeDirectory(dir);
}
TString baseFXSFolder;
}
else if (system == kHLT)
{
- baseFXSFolder = "~/";
+ baseFXSFolder = "/opt/FXS/";
}
- TString command = Form("scp -oPort=%d -2 %s@%s:%s%s %s/%s",
+ TString command = Form("scp -oPort=%d -2 %s@%s:%s%s %s",
fConfig->GetFXSPort(system),
fConfig->GetFXSUser(system),
fConfig->GetFXSHost(system),
baseFXSFolder.Data(),
fxsFileName,
- GetShuttleTempDir(),
localFileName);
AliDebug(2, Form("%s",command.Data()));
//______________________________________________________________________________________________
TList* AliShuttle::GetFileSources(Int_t system, const char* detector, const char* id)
{
-// Get sources producing the condition file Id from file exchange servers
+ //
+ // Get sources producing the condition file Id from file exchange servers
+ // if id is NULL all sources are returned (distinct)
+ //
- if (system == kDCS)
+ Log(detector, Form("GetFileSources - Retrieving sources with id %s from %s", id, GetSystemName(system)));
+
+ // check if test mode should simulate a FXS error
+ if (fTestMode & kErrorFXSSources)
{
- AliError("DCS system has only one source of data!");
- return NULL;
+ Log(detector, Form("GetFileSources - In TESTMODE - Simulating error while connecting to %s FXS", GetSystemName(system)));
+ return 0;
+ }
+ if (system == kDCS)
+ {
+ Log(detector, "GetFileSources - WARNING: DCS system has only one source of data!");
+ TList *list = new TList();
+ list->SetOwner(1);
+ list->Add(new TObjString(" "));
+ return list;
}
// check connection, in case connect
if (!Connect(system))
{
- Log(detector, Form("GetFile - Couldn't connect to %s FXS database", GetSystemName(system)));
+ Log(detector, Form("GetFileSources - Couldn't connect to %s FXS database", GetSystemName(system)));
return NULL;
}
sourceName = "DDLnumbers";
}
- TString sqlQueryStart = Form("select %s from %s where", sourceName.Data(), fConfig->GetFXSdbTable(system));
- TString whereClause = Form("run=%d and detector=\"%s\" and fileId=\"%s\"",
- GetCurrentRun(), detector, id);
+ TString sqlQueryStart = Form("select distinct %s from %s where", sourceName.Data(), fConfig->GetFXSdbTable(system));
+ TString whereClause = Form("run=%d and detector=\"%s\"",
+ GetCurrentRun(), detector);
+ if (id)
+ whereClause += Form(" and fileId=\"%s\"", id);
TString sqlQuery = Form("%s %s", sqlQueryStart.Data(), whereClause.Data());
AliDebug(2, Form("SQL query: \n%s",sqlQuery.Data()));
return 0;
}
+ TList *list = new TList();
+ list->SetOwner(1);
+
if (aResult->GetRowCount() == 0)
{
Log(detector,
Form("GetFileSources - No entry in %s FXS table for id: %s", GetSystemName(system), id));
delete aResult;
- return 0;
+ return list;
}
- TSQLRow* aRow;
- TList *list = new TList();
- list->SetOwner(1);
+ Log(detector, Form("GetFileSources - Found %d sources", aResult->GetRowCount()));
+ TSQLRow* aRow;
while ((aRow = aResult->Next()))
{
return list;
}
+//______________________________________________________________________________________________
+TList* AliShuttle::GetFileIDs(Int_t system, const char* detector, const char* source)
+{
+ //
+ // Get all ids of condition files produced by a given source from file exchange servers
+ //
+
+ Log(detector, Form("GetFileIDs - Retrieving ids with source %s with %s", source, GetSystemName(system)));
+
+ // check if test mode should simulate a FXS error
+ if (fTestMode & kErrorFXSSources)
+ {
+ Log(detector, Form("GetFileIDs - In TESTMODE - Simulating error while connecting to %s FXS", GetSystemName(system)));
+ return 0;
+ }
+
+ // check connection, in case connect
+ if (!Connect(system))
+ {
+ Log(detector, Form("GetFileIDs - Couldn't connect to %s FXS database", GetSystemName(system)));
+ return NULL;
+ }
+
+ TString sourceName = 0;
+ if (system == kDAQ)
+ {
+ sourceName = "DAQsource";
+ } else if (system == kHLT)
+ {
+ sourceName = "DDLnumbers";
+ }
+
+ TString sqlQueryStart = Form("select fileId from %s where", fConfig->GetFXSdbTable(system));
+ TString whereClause = Form("run=%d and detector=\"%s\"",
+ GetCurrentRun(), detector);
+ if (sourceName.Length() > 0 && source)
+ whereClause += Form(" and %s=\"%s\"", sourceName.Data(), source);
+ TString sqlQuery = Form("%s %s", sqlQueryStart.Data(), whereClause.Data());
+
+ AliDebug(2, Form("SQL query: \n%s",sqlQuery.Data()));
+
+ // Query execution
+ TSQLResult* aResult;
+ aResult = fServer[system]->Query(sqlQuery);
+ if (!aResult) {
+ Log(detector, Form("GetFileIDs - Can't execute SQL query to %s database for source: %s",
+ GetSystemName(system), source));
+ return 0;
+ }
+
+ TList *list = new TList();
+ list->SetOwner(1);
+
+ if (aResult->GetRowCount() == 0)
+ {
+ Log(detector,
+ Form("GetFileIDs - No entry in %s FXS table for source: %s", GetSystemName(system), source));
+ delete aResult;
+ return list;
+ }
+
+ Log(detector, Form("GetFileIDs - Found %d ids", aResult->GetRowCount()));
+
+ TSQLRow* aRow;
+
+ while ((aRow = aResult->Next()))
+ {
+
+ TString id(aRow->GetField(0), aRow->GetFieldLength(0));
+ AliDebug(2, Form("fileId = %s", id.Data()));
+ list->Add(new TObjString(id));
+ delete aRow;
+ }
+
+ delete aResult;
+
+ return list;
+}
+
//______________________________________________________________________________________________
Bool_t AliShuttle::Connect(Int_t system)
{
-// Connect to MySQL Server of the system's FXS MySQL databases
-// DAQ Logbook, Shuttle Logbook and DAQ FXS db are on the same host
+ // Connect to MySQL Server of the system's FXS MySQL databases
+ // DAQ Logbook, Shuttle Logbook and DAQ FXS db are on the same host
+ //
// check connection: if already connected return
if(fServer[system] && fServer[system]->IsConnected()) return kTRUE;
//______________________________________________________________________________________________
Bool_t AliShuttle::UpdateTable()
{
-// Update FXS table filling time_processed field in all rows corresponding to current run and detector
+ //
+ // Update FXS table filling time_processed field in all rows corresponding to current run and detector
+ //
Bool_t result = kTRUE;
//______________________________________________________________________________________________
Bool_t AliShuttle::UpdateTableFailCase()
{
-// Update FXS table filling time_processed field in all rows corresponding to current run and detector
-// this is called in case the preprocessor is declared failed for the current run, because
-// the fields are updated only in case of success
+ // Update FXS table filling time_processed field in all rows corresponding to current run and detector
+ // this is called in case the preprocessor is declared failed for the current run, because
+ // the fields are updated only in case of success
Bool_t result = kTRUE;
//______________________________________________________________________________________________
Int_t AliShuttle::GetCurrentRun() const
{
-// Get current run from logbook entry
+ //
+ // Get current run from logbook entry
+ //
return fLogbookEntry ? fLogbookEntry->GetRun() : -1;
}
//______________________________________________________________________________________________
UInt_t AliShuttle::GetCurrentStartTime() const
{
-// get current start time
+ //
+ // get current start time
+ //
return fLogbookEntry ? fLogbookEntry->GetStartTime() : 0;
}
//______________________________________________________________________________________________
UInt_t AliShuttle::GetCurrentEndTime() const
{
-// get current end time from logbook entry
+ //
+ // get current end time from logbook entry
+ //
return fLogbookEntry ? fLogbookEntry->GetEndTime() : 0;
}
+//______________________________________________________________________________________________
+UInt_t AliShuttle::GetCurrentYear() const
+{
+ //
+ // Get current year from logbook entry
+ //
+
+ if (!fLogbookEntry) return 0;
+
+ TTimeStamp startTime(GetCurrentStartTime());
+ TString year = Form("%d",startTime.GetDate());
+ year = year(0,4);
+
+ return year.Atoi();
+}
+
+//______________________________________________________________________________________________
+const char* AliShuttle::GetLHCPeriod() const
+{
+ //
+ // Get current LHC period from logbook entry
+ //
+
+ if (!fLogbookEntry) return 0;
+
+ return fLogbookEntry->GetRunParameter("LHCperiod");
+}
+
//______________________________________________________________________________________________
void AliShuttle::Log(const char* detector, const char* message)
{
-// Fill log string with a message
+ //
+ // Fill log string with a message
+ //
void* dir = gSystem->OpenDirectory(GetShuttleLogDir());
if (dir == NULL) {
toLog += Form("%s", message);
AliInfo(toLog.Data());
+
+ // if we redirect the log output already to the file, leave here
+ if (fOutputRedirected && strcmp(detector, "SHUTTLE") != 0)
+ return;
- TString fileName;
- if (GetCurrentRun() >= 0)
- fileName.Form("%s/%s_%d.log", GetShuttleLogDir(), detector, GetCurrentRun());
- else
- fileName.Form("%s/%s.log", GetShuttleLogDir(), detector);
+ TString fileName = GetLogFileName(detector);
gSystem->ExpandPathName(fileName);
logFile.close();
}
+//______________________________________________________________________________________________
+TString AliShuttle::GetLogFileName(const char* detector) const
+{
+ //
+ // returns the name of the log file for a given sub detector
+ //
+
+ TString fileName;
+
+ if (GetCurrentRun() >= 0)
+ fileName.Form("%s/%s_%d.log", GetShuttleLogDir(), detector, GetCurrentRun());
+ else
+ fileName.Form("%s/%s.log", GetShuttleLogDir(), detector);
+
+ return fileName;
+}
+
//______________________________________________________________________________________________
Bool_t AliShuttle::Collect(Int_t run)
{
-//
-// Collects conditions data for all UNPROCESSED run written to DAQ LogBook in case of run = -1 (default)
-// If a dedicated run is given this run is processed
-//
-// In operational mode, this is the Shuttle function triggered by the EOR signal.
-//
+ //
+ // Collects conditions data for all UNPROCESSED run written to DAQ LogBook in case of run = -1 (default)
+ // If a dedicated run is given this run is processed
+ //
+ // In operational mode, this is the Shuttle function triggered by the EOR signal.
+ //
if (run == -1)
Log("SHUTTLE","Collect - Shuttle called. Collecting conditions data for unprocessed runs");
//______________________________________________________________________________________________
Bool_t AliShuttle::RetrieveConditionsData(const TObjArray& dateEntries)
{
-// Retrieve conditions data for all runs that aren't processed yet
+ //
+ // Retrieve conditions data for all runs that aren't processed yet
+ //
Bool_t hasError = kFALSE;
}
// clean SHUTTLE temp directory
- TString filename = Form("%s/*.shuttle", GetShuttleTempDir());
- RemoveFile(filename.Data());
+ //TString filename = Form("%s/*.shuttle", GetShuttleTempDir());
+ //RemoveFile(filename.Data());
}
return hasError == kFALSE;
//______________________________________________________________________________________________
ULong_t AliShuttle::GetTimeOfLastAction() const
{
+ //
+ // Gets time of last action
+ //
+
ULong_t tmp;
fMonitoringMutex->Lock();
//______________________________________________________________________________________________
const TString AliShuttle::GetLastAction() const
{
+ //
// returns a string description of the last action
+ //
TString tmp;
//______________________________________________________________________________________________
void AliShuttle::SetLastAction(const char* action)
{
+ //
// updates the monitoring variables
+ //
fMonitoringMutex->Lock();
//______________________________________________________________________________________________
const char* AliShuttle::GetRunParameter(const char* param)
{
-// returns run parameter read from DAQ logbook
+ //
+ // returns run parameter read from DAQ logbook
+ //
if(!fLogbookEntry) {
AliError("No logbook entry!");
}
//______________________________________________________________________________________________
-AliCDBEntry* AliShuttle::GetFromOCDB(const AliCDBPath& path)
+AliCDBEntry* AliShuttle::GetFromOCDB(const char* detector, const AliCDBPath& path)
{
-// returns obiect from OCDB valid for current run
+ //
+ // returns object from OCDB valid for current run
+ //
+ if (fTestMode & kErrorOCDB)
+ {
+ Log(detector, "GetFromOCDB - In TESTMODE - Simulating error with OCDB");
+ return 0;
+ }
+
AliCDBStorage *sto = AliCDBManager::Instance()->GetStorage(fgkMainCDB);
if (!sto)
{
- Log("SHUTTLE", "GetFromOCDB - Cannot activate main OCDB for query!");
+ Log(detector, "GetFromOCDB - Cannot activate main OCDB for query!");
return 0;
}
//______________________________________________________________________________________________
Bool_t AliShuttle::SendMail()
{
-// sends a mail to the subdetector expert in case of preprocessor error
+ //
+ // sends a mail to the subdetector expert in case of preprocessor error
+ //
+
+ if (fTestMode != kNone)
+ return kTRUE;
void* dir = gSystem->OpenDirectory(GetShuttleLogDir());
if (dir == NULL)
{
if (gSystem->mkdir(GetShuttleLogDir(), kTRUE))
{
- AliError(Form("Can't open directory <%s>", GetShuttleLogDir()));
+ Log("SHUTTLE", Form("SendMail - Can't open directory <%s>", GetShuttleLogDir()));
return kFALSE;
}
if (!mailBody.is_open())
{
- AliError(Form("Could not open mail body file %s", bodyFileName.Data()));
+ Log("SHUTTLE", Form("Could not open mail body file %s", bodyFileName.Data()));
return kFALSE;
}
to.Remove(to.Length()-1);
AliDebug(2, Form("to: %s",to.Data()));
- // TODO this will be removed...
- if (to.Contains("not_yet_set")) {
- AliInfo("List of detector responsibles not yet set!");
+ if (to.IsNull()) {
+ Log("SHUTTLE", "List of detector responsibles not yet set!");
return kFALSE;
}
TString cc="alberto.colla@cern.ch";
- TString subject = Form("%s Shuttle preprocessor error in run %d !",
+ TString subject = Form("%s Shuttle preprocessor FAILED in run %d !",
fCurrentDetector.Data(), GetCurrentRun());
AliDebug(2, Form("subject: %s", subject.Data()));
TString body = Form("Dear %s expert(s), \n\n", fCurrentDetector.Data());
body += Form("SHUTTLE just detected that your preprocessor "
- "exited with ERROR state in run %d!!\n\n", GetCurrentRun());
- body += Form("Please check %s status on the web page asap!\n\n", fCurrentDetector.Data());
+ "failed processing run %d!!\n\n", GetCurrentRun());
+ body += Form("Please check %s status on the SHUTTLE monitoring page: \n\n", fCurrentDetector.Data());
+ body += Form("\thttp://pcalimonitor.cern.ch:8889/shuttle.jsp?time=168 \n\n");
+ body += Form("Find the %s log for the current run on \n\n"
+ "\thttp://pcalishuttle01.cern.ch:8880/logs/%s_%d.log \n\n",
+ fCurrentDetector.Data(), fCurrentDetector.Data(), GetCurrentRun());
body += Form("The last 10 lines of %s log file are following:\n\n");
AliDebug(2, Form("Body begin: %s", body.Data()));
TString endBody = Form("------------------------------------------------------\n\n");
endBody += Form("In case of problems please contact the SHUTTLE core team.\n\n");
endBody += "Please do not answer this message directly, it is automatically generated.\n\n";
- endBody += "Sincerely yours,\n\n \t\t\tthe SHUTTLE\n";
+ endBody += "Greetings,\n\n \t\t\tthe SHUTTLE\n";
AliDebug(2, Form("Body end: %s", endBody.Data()));
}
//______________________________________________________________________________________________
-void AliShuttle::SetRunType()
+Bool_t AliShuttle::SendMailToDCS()
{
-// Gets run type from logbook and fills current Shuttle logbook entry
+ //
+ // sends a mail to the DCS experts in case of DCS error
+ //
+
+ if (fTestMode != kNone)
+ return kTRUE;
- // check connection, in case connect
- if(!Connect(3)){
- Log("SHUTTLE", "SetRunType - Couldn't connect to DAQ Logbook.");
- return;
+ void* dir = gSystem->OpenDirectory(GetShuttleLogDir());
+ if (dir == NULL)
+ {
+ if (gSystem->mkdir(GetShuttleLogDir(), kTRUE))
+ {
+ Log("SHUTTLE", Form("SendMailToDCS - Can't open directory <%s>", GetShuttleLogDir()));
+ return kFALSE;
+ }
+
+ } else {
+ gSystem->FreeDirectory(dir);
}
- TString sqlQuery = Form("select detector,run_type from %s where run_number=%d",
- fConfig->GetRunTypelbTable(), GetCurrentRun());
+ TString bodyFileName;
+ bodyFileName.Form("%s/mail.body", GetShuttleLogDir());
+ gSystem->ExpandPathName(bodyFileName);
- AliDebug(2, Form("SQL query: \n%s",sqlQuery.Data()));
+ ofstream mailBody;
+ mailBody.open(bodyFileName, ofstream::out);
- // Query execution
- TSQLResult* aResult;
- aResult = dynamic_cast<TSQLResult*> (fServer[3]->Query(sqlQuery));
- if (!aResult) {
- Log("SHUTTLE", Form("SetRunType - Can't execute query <%s>", sqlQuery.Data()));
- return;
+ if (!mailBody.is_open())
+ {
+ Log("SHUTTLE", Form("SendMailToDCS - Could not open mail body file %s", bodyFileName.Data()));
+ return kFALSE;
+ }
+
+ TString to="Vladimir.Fekete@cern.ch, Svetozar.Kapusta@cern.ch";
+ //TString to="alberto.colla@cern.ch";
+ AliDebug(2, Form("to: %s",to.Data()));
+
+ if (to.IsNull()) {
+ Log("SHUTTLE", "List of detector responsibles not yet set!");
+ return kFALSE;
}
- TSQLRow* aRow;
- while ((aRow = aResult->Next())) {
- TString det(aRow->GetField(0), aRow->GetFieldLength(0));
- TString runType(aRow->GetField(1), aRow->GetFieldLength(1));
+ TString cc="alberto.colla@cern.ch";
- fLogbookEntry->SetRunType(det, runType);
- delete aRow;
+ TString subject = Form("Retrieval of data points for %s FAILED in run %d !",
+ fCurrentDetector.Data(), GetCurrentRun());
+ AliDebug(2, Form("subject: %s", subject.Data()));
+
+ TString body = Form("Dear DCS experts, \n\n");
+ body += Form("SHUTTLE couldn\'t retrieve the data points for detector %s "
+ "in run %d!!\n\n", fCurrentDetector.Data(), GetCurrentRun());
+ body += Form("Please check %s status on the SHUTTLE monitoring page: \n\n", fCurrentDetector.Data());
+ body += Form("\thttp://pcalimonitor.cern.ch:8889/shuttle.jsp?time=168 \n\n");
+ body += Form("Find the %s log for the current run on \n\n"
+ "\thttp://pcalishuttle01.cern.ch:8880/logs/%s_%d.log \n\n",
+ fCurrentDetector.Data(), fCurrentDetector.Data(), GetCurrentRun());
+ body += Form("The last 10 lines of %s log file are following:\n\n");
+
+ AliDebug(2, Form("Body begin: %s", body.Data()));
+
+ mailBody << body.Data();
+ mailBody.close();
+ mailBody.open(bodyFileName, ofstream::out | ofstream::app);
+
+ TString logFileName = Form("%s/%s_%d.log", GetShuttleLogDir(), fCurrentDetector.Data(), GetCurrentRun());
+ TString tailCommand = Form("tail -n 10 %s >> %s", logFileName.Data(), bodyFileName.Data());
+ if (gSystem->Exec(tailCommand.Data()))
+ {
+ mailBody << Form("%s log file not found ...\n\n", fCurrentDetector.Data());
}
- delete aResult;
+ TString endBody = Form("------------------------------------------------------\n\n");
+ endBody += Form("In case of problems please contact the SHUTTLE core team.\n\n");
+ endBody += "Please do not answer this message directly, it is automatically generated.\n\n";
+ endBody += "Greetings,\n\n \t\t\tthe SHUTTLE\n";
+
+ AliDebug(2, Form("Body end: %s", endBody.Data()));
- return;
+ mailBody << endBody.Data();
+
+ mailBody.close();
+
+ // send mail!
+ TString mailCommand = Form("mail -s \"%s\" -c %s %s < %s",
+ subject.Data(),
+ cc.Data(),
+ to.Data(),
+ bodyFileName.Data());
+ AliDebug(2, Form("mail command: %s", mailCommand.Data()));
+
+ Bool_t result = gSystem->Exec(mailCommand.Data());
+
+ return result == 0;
+}
+//______________________________________________________________________________________________
+const char* AliShuttle::GetRunType()
+{
+ //
+ // returns run type read from "run type" logbook
+ //
+
+ if(!fLogbookEntry) {
+ AliError("No logbook entry!");
+ return 0;
+ }
+
+ return fLogbookEntry->GetRunType();
}
//______________________________________________________________________________________________
-const char* AliShuttle::GetRunType(const char* detCode)
+Bool_t AliShuttle::GetHLTStatus()
{
-// returns run type read from "run type" logbook
+ // Return HLT status (ON=1 OFF=0)
+ // Converts the HLT status from the status string read in the run logbook (not just a bool)
if(!fLogbookEntry) {
AliError("No logbook entry!");
return 0;
}
- return fLogbookEntry->GetRunType(detCode);
+ // TODO implement when HLTStatus is inserted in run logbook
+ //TString hltStatus = fLogbookEntry->GetRunParameter("HLTStatus");
+ //if(hltStatus == "OFF") {return kFALSE};
+
+ return kTRUE;
}
//______________________________________________________________________________________________
void AliShuttle::SetShuttleTempDir(const char* tmpDir)
{
-// sets Shuttle temp directory
+ //
+ // sets Shuttle temp directory
+ //
fgkShuttleTempDir = gSystem->ExpandPathName(tmpDir);
}
//______________________________________________________________________________________________
void AliShuttle::SetShuttleLogDir(const char* logDir)
{
-// sets Shuttle log directory
+ //
+ // sets Shuttle log directory
+ //
fgkShuttleLogDir = gSystem->ExpandPathName(logDir);
}
git://git.uio.no / u / mrichter / AliRoot.git / blobdiff