Table of contents
The silicon D-mode calibration analysis software is a loosely coupled system of executables and scripts. The advantage of the loose organization is that it is very easy to develop new functional units, modify existing ones, and change the analysis flow without disturbing the rest of the setup. The disadvantage is the overhead incurred by every program due to data loading from disk into memory. The calibration consumer scheme used for other CDF subsystems can not work for silicon because the sheer amount of data needed to calibrate the silicon system with 722,432 channels and the resulting processing time effectively precludes any quasi-real-time analysis.
The software is distributed across several nodes. The histograms with
pulse height distributions are built on fcdfsgi2.fnal.gov (fcdfsgi2) which
can run several histogramming jobs simultaneously on multiple CPUs. The
main data analysis and job status monitoring is performed on pcdaq.lbl.gov
(pcdaq) which is a fast PC with large amount of memory. The job status
web page and CGI scripts are hosted by phyweb.lbl.gov (phyweb).
Support Executables on fcdfsgi2
Calibration
Scripts on fcdfsgi2
The calibration scripts provide a convenient interface to the core executables, automate data processing and job tracking, and glue various calibration tasks together. Most scripts are written in tcl, while a few simple ones are executed by tcsh. All scripts are in the same directory, currently ~igv/Calib_scripts. The main scripts are:
Viewing Silicon Calibration Tables on fcdfsgi2
It is possible to view and compare calibration table contents on fcdfsgi2. There is a general tool for accessing calibration database tables called DBANA which has the ability to fetch SICHIPPED and SISTRIPDH tables and display their contents with various detector granularity. However, other programs also had to be developed in order to deal with specific tasks related to the analysis of silicon D-mode calibrations. This is what available on fcdfsgi2:
The table-viewing programs organize and display the data by half-ladder side (in the calibration database it is stored by chip). The naming convention adopted for silicon half-ladders follows the scheme originally proposed in the CDF note 4103:
C 5 6 -B PHI -- This is an example name for the
phi side of the SVX sensor
| | | | |
in the center barrel, east half-barrel, layer 5, wedge 6.
| | | | `----- Sensor side. May be Z or
PHI.
| | | `-------- Ladder segment. A, B, and C stand
for west, east, and center, respectively.
| | `----------- Phi wedge
| `------------- Layer
`--------------- Barrel. Letters W, C, E, and S mean
west, center, east, and spare, respectively.
When necessary, these sensor designators can be related to their DAQ
names using the SVX
chip terminology converter.
The calibration scripts require a recent tcl version. Currently, we use Tcl/Tk 8.3.2 installed in the directory /cdf/data05/s1/hardware/svxii/Analysis (the latest tcl version in FNAL KITS is 8.0.2 -- too old). The latest stable tcl release is usually available directly from ActiveState. In addition to the core tcl, the following add-on tcl packages are installed:
source ~cdfsoft/products/setups/setups.csh
setup oracle_client
setenv ANALYSIS_DIR /cdf/data05/s1/hardware/svxii/Analysis
setenv HISTO_DIR $ANALYSIS_DIR/Histoscope
setenv MYSQL_DIR $ANALYSIS_DIR/mysql
setenv LD_LIBRARY_PATH $ANALYSIS_DIR/lib:$MYSQL_DIR/lib/mysql:$LD_LIBRARY_PATH
set path = ($ANALYSIS_DIR/bin $HISTO_DIR/bin
$MYSQL_DIR/bin $path)
It is convenient to load the "rdl" package and to use its command-line enhancing features automatically for interactive tcl sessions. A simple rdl initialization can be accomplished by placing the following code in your .tclshrc and .wishrc files:
if {[catch {package require Tcl 8.3}] == 0} {
if {$tcl_interactive} {
package require rdl
namespace import rdl::ls
rdl::rm
rdl::use_xtermcd
rdl::add_namespace ::
rdl::add_path /bin /usr/bin
$env(HOME)/bin
# The next command never
returns
rdl::input_loop
}
}
The pulse height distribution histograms are analyzed on pcdaq.lbl.gov. This is a PC with 1 GHz Pentium III CPU and 2 GB of physical memory. So far, this amount of memory has been sufficient for simultaneous loading of two sets of pulse height distribution histograms for all CDF silicon sensors (two sets are needed for DPS-on pedestal and noise calculations). Most analysis tasks are performed by tcl scripts which use the hs extension for histogram manipulation. Histo-Scope/tcl was chosen over root/C++ because Histo-Scope is more mature and reliable than root and because of Histo-Scope's superior interactive response in viewing 2d histograms. In addition, tcl makes much better macro language than C++ because tcl is typeless, it has automatic garbage collection and consistent error handling, and because it can be used very efficiently for building GUIs with the Tk toolkit. The basic pedestal and noise analysis functionality had to be implemented and used very early during the development of the system, and the advantages of the Histo-Scope/tcl combination for rapid coding and analysis clearly outweighted a few drawbacks such as the necessity of data file conversions from root to Histo-Scope format or the inability to compile the final analysis code.
Currently, all major calibration data analysis scrips are in the directory ~svxii/Scripts. This directory contains the following files:
The hscompare script is available on pcdaq at ~svxii/bin/hscompare. There is also another useful generic script called "calibwish". It can be used to load a set of Histo-Scope files like this:
~svxii/bin/calibwish file1.hs file2.hs ...
Calibwish loads the hs package and reads all files into memory. It also starts the Histo-Scope client and sources calibration-related definitions and utilities, thus getting ready for the interactive analysis of the data.
All environmental variables necessary for calibration data analysis
(related to ROOT, Histo-Scope, tcl, MySQL, etc.) are set in the
~svxii/.cshrc file.
Job
Tracking Database on pcdaq
The progress of calibration analysis jobs is tracked using a MySQL database installed on pcdaq.lbl.gov. The table used for this purpose is named CALIB_JOBS. It was created with the following SQL command:
create table calib_jobs (
id INT UNSIGNED AUTO_INCREMENT NOT
NULL,
caltype VARCHAR(32) NOT NULL,
dps_off_run INT UNSIGNED NOT NULL,
dps_on_run INT UNSIGNED NOT NULL,
name VARCHAR(127) NOT NULL,
email VARCHAR(255) NOT NULL,
submission_time DATETIME NOT NULL,
processing_stage VARCHAR(64),
run_period_tag VARCHAR(32),
main_pid INT,
completion_time DATETIME,
completion_status SET('SUCCESS','FAILURE'),
failure_reason TEXT,
update_date TIMESTAMP,
comment VARCHAR(255),
primary key(id)
);
The columns in this table have the following meaning:
Other tables in the database are used to remember the sets of biased ladders extracted by the svx_config_server.tcl script (table BIASED_LADDERS) and to save all dissimilarity coefficients calculated by the compare_calibrations.tcl script (table DISSIM). The SQL commands used to create these tables are given below:
create table biased_ladders (
run_number INT UNSIGNED NOT NULL,
hdi_list TEXT NOT NULL,
primary key(run_number)
);
create table dissim (
calib_run_off INT UNSIGNED NOT NULL,
calib_run_on INT UNSIGNED
NOT NULL,
ref_run_off INT UNSIGNED
NOT NULL,
ref_run_on INT
UNSIGNED NOT NULL,
diss_name
VARCHAR(16) NOT NULL,
diss_type
SET('abs','rel') NOT NULL,
detector
VARCHAR(16) NOT NULL,
diss_value FLOAT
NOT NULL,
primary key(
calib_run_off,
calib_run_on,
ref_run_off,
ref_run_on,
diss_name,
diss_type,
detector
)
);
The scripts are currently in the directory ~webmaster/cgi-bin-CDF/cdf_si_calibrations. This directory contains the following files related to the job tracking:
The typical calibration processing time is 4 to 6 hours when the data files are already available on fcdfsgi2 by the time the job request is made. The bulk of this time is used by the SvxPulseHeights executables on fcdfsgi2 while they accumulate pulse height distribution histograms. The analysis of the histograms on pcdaq takes about 45 minutes. About 10 minutes are needed for the submission of calibration tables into the CDF database using a2db.
The amount of data transferred between fcdfsgi2 and pcdaq over the network is about 150 MBytes per calibration. The internet connection between FNAL and LBNL is usually good enough to transfer data at the speeds of 500 kBytes/second or higher, and so far the file transfer time has never been exceedingly long.
Compared to the items just described, other tasks consume a negligible
amount of time. The overall processing time will become longer as the number
of ladders integrated into the silicon system increases.
Due to the presence of the dynamic pedestal subtraction (DPS) circuitry in the readout chips, a standard silicon calibration includes two read-all runs: one with DPS turned off, and one with DPS on. The analysis of data from these runs starts with accumulation of the ADC pulse height distributions for every strip in the silicon system in three different modes: DPS-off, DPS-on, and simulated DPS. In each of these modes, the SvxPulseHeights program builds one 2-d pulse height histogram for each silicon half-ladder side (phi and z). These histograms have ADC counts along the x axis, strip numbers along the y axis, and number of readouts along the z axis.
In hope to provide a better explanation of the algorithms used by D-mode
to determine the DPS-on pedestals and noise, a basic description of the
DPS functionality is included in the next section. Please consult CDF notes
4851,
4852
if you need more details.
The SVX3D silicon readout chips use Wilkinson ADCs for signal digitization. All 128 ADC channels use a common ramp and a common clock counter. In each channel, silicon signals are compared with the ramp, and the value of the counter is latched when the ramp becomes larger than the signal. Normally, the clock counter starts counting at a fixed point in the digitization sequence -- at the same moment when the ramp generator is turned on or a little bit earlier. The idea of the DPS circuit is to shift the counter start time to the moment when the ramp crosses the pedestal level in most of the channels. Then any kind of a common pick up across all channels will not affect the ADC values read out by individual signal channels (which are above the pedestal level). This feature provides an effective hardware comon mode noise suppression which could be accomplished with older silicon chips only by reading out all channels every event and peforming a similar pedestal subtraction in DSPs or software -- a prohibitively slow operation at high trigger rates.
The principal desing of the DPS circuit is illustrated below (with only 3 channels shown out of 128):
The number of ADC comparators which must switch before the comparator Cdps turns on the counter, f_dps, is set with the external voltage Vthreshold. The comparator Cdps introduces a small propagation delay, t_d, and the counter begins counting a little bit later than most ADC comparators switch. Therefore, the effective pedestal values in this scheme are often negative, and most latches register 0 even before the counter starts. The effect of this circuit on the pulse heigh distribution from 128 chip channels in one calibration event is illustrated in Fig. 1 (the delay t_d is exaggerated in this picture). The blue histogram shows pulse height distribution with DPS off, the red one shows the pulse height distribution with DPS on, and the green one shows how the DPS on distribution would look like if we were able to see negative ADC counts. It is convenient to use ADC counts as units for t_d, and we will continue doing for the rest of this note (1 ADC count = 9.4 nsec for the fixed 106 MHz ADC conversion frequency).
The pulse height distributions from one silicon strip in DPS-off and
DPS-on calibration runs are illustrated in Fig.
2. As before, the DPS-off distribution is shown in blue, DPS-on is
red, and the green histogram shows how the DPS on distribution would look
like if we were able to see negative ADC counts. The purpose of DPS-on
calibration is to measure the location and width of the green curve. From
the shape of these histograms it is obvious that the DPS-on pulse height
distributions alone are not sufficient for estimation of the DPS-on pedestals
and noise, and to deduce these pedestals the D-mode software needs to simulate
the effect of the DPS circuit using DPS-off data.
If we know the f_dps and t_d parameters of the DPS circuit for each chip then we can simulate its behavior in software. On the event-by-event basis, we can find the point, X_dps, where the DPS circuit starts the counter (see Fig. 1). Then for each strip we can calculate the difference of its ADC count and X_dps, D_strip. The width of the D_strip distribution should be identical to the width of the DPS-on distribution in Fig. 2 (the green histogram). Finally, by subtracting t_d from D_strip we should obtain a distribution which is identical to the DPS-on distribution but without mapping all negative ADC counts into 0.
The f_dps parameters may differ from chip to chip, and they should be measured in a dedicated calibration run. In this run the number of channels with charge injection should be changed (by changing calibration mask bits) within the range from about 60 to about 110, with chips operating in DPS-on mode. We look at the average pedestal for several control channels. For these control channels the charge injection is always on. When the number of channels with charge injection becomes large, the common mode noise suppression circuit stops working (it is tricked into believing that the charge injection itself is the common mode noise), and it suppresses the control channels. The curve which shows the dependence of the average pedestal in the control channels on the number of channels without charge injection has an S-like shape. The location of the steepest part of this curve defines the f_dps parameter.
The f_dps measurements were performed for each chip during hybrid testing at LBNL. The resulting f_dps distribution is shown on this plot. These measurements should also be performed on the complete silicon system at B0 as soon as the silicon DAQ is capable of running this kind of calibration.
When f_dps values are known, the t_d parameters are estimated naturally as a part of the DPS-on pedestal calculation.
With read-all silicon data taken in DPS-off mode, the SvxPulseHeights
executable can build the D_strip distributions instead of just the raw
pulse height distributions. This feature can be turned on or off in the
configuration file by modifying the boolean parameter "suppressCommonMode"
in the SvxPulseHeightModule "talk-to". At the time of this writing, the
program uses the peak position of the LBNL f_dps distribution for every
chip in the system.
The DPS-off pedestals are determined for each strip as the 50th percentile of the corresponding pulse height distribution. Due to technical reasons, the histograms produced by the SvxPulseHeights executables have integers at bin edges (instead of bin centers). For example, the bin which corresponds to 5 ADC counts has its left edge at 5 and its right edge at 6 ADC counts. Because of that, 0.5 ADC counts are subtracted from the 50th percentile in order to obtain a result which, for symmetrical distributions, is consistent with simple average of the original ADC counts. Before this result is written into the database, it is multiplied by 2 and rounded to the nearest integer because silicon pedestals are stored with the precision of 1/2 ADC counts.
The DPS-off noise is defined as a half of the difference between 84.13th and 15.87th percentiles. For Gaussian distribution, this definition coincides with the standard deviation. The noise values are stored with the precision of 1/8 ADC counts, so they are multiplied by 8 and rounded to the nearest integer for writing into the database. Both pedestals and noise are stored with full floating point precision in the intermediate Histo-Scope files which are subsequently used for inspection of calibration results.
The robust location and scale estimators used to calculate silicon pedestal and noise are virtually insensitive to low-rate data bit errors in the optical transmission of silicon data and to outlier events caused by extreme common mode noise fluctuations. The estimation of the DPS-off pedestals and noise are performed by the robust_sichipped.tcl script.
Dnoise values for each strip are estimated by the SvxPulseHeights executable
from DPS-off, read-all data. Dnoise is defined for channel i as the standard
deviation of the difference (ADC i - ADC
i-1) divided by the square root of 2. For channel number 0 on each
chip dnoise is copied from channel 1. Dnoise values are not used in the
silicon clustering, so it is not important to have a robust estimate in
this case.
The DPS-on pedestals and noise are estimated using two pulse height distributions: the actual hardware DPS distribution and a simulated DPS distribution created using DPS-off data (D_strip distribution). The noise is defined as a half of the difference between 84.13th and 15.87th percentiles for the simulated DPS distribution. In order to find the pedestals we need to know the shift between the hardware DPS distribution and the D_strip distribution. To find this shift we perform tail matching: the fraction of the hardware DPS distribution with 1 or more ADC counts is determined, and then the corresponding percentile is found in the simulated DPS distribution. The shift equals the difference between this percentile and 1. When the DPS-on area above 0 ADC counts is larger than 50%, the shift is determined instead as the difference between 50th percentiles of these distributions.
Since the shift equals t_d, it is supposed to be the same for all channels in a given chip. With this constraint, the shift estimate is improved by calculating t_d as the median shift over all channels in the chip. The DPS-on pedestals are then determined for each strip as the difference between the 50th percentile of the D_strip distribution and t_d. As in the case of DPS-off pedestals, 0.5 ADC counts are subtracted from this difference to obtain the final result. The estimation of the DPS-on pedestals and noise are performed by the dcms_sichipped.tcl script.
When f_dps and t_d are known for each chip, we can rely even more on the DPS simulation and take calibration runs with hardware DPS only infrequently. It is not yet clear how f_dps and t_d change with time, temperature, and radiation, but the change rate is expected to be rather slow.
As a crosscheck of this procedure, it will be interesting to calculate
the DPS-on pedestals by measuring the pedestal shifts between the two DPS
modes using charge injection. For that, we need to take calibration runs
with charge injection into a small fraction (e.g., 10%) of the chip
channels in both DPS modes. The charge injection is necessary in order
to see the complete DPS-on pulse height distribution in at least a few
channels. These channels alone can be used for determining the pedestal
shift for the whole chip. This method relies on the assumption that the
pedestal shift due to DPS is the same for all channels. This assumption
works well if the pedestals and noise values are reasonably flat accross
the chip.
There is no way to restore the data from channels tagged for suppression at the FIB level. Therefore, it is important to tag pinholes and very noisy channels while keeping away from defects which can be potentially fixed in software, such as "flapping wings" or isolated bad pipeline cells. The "fibkill" algorithm described below has been designed with these considerations in mind.
For the seed strips:
The algorithm which defines bad channels for clustering purposes makes the assumption that most channels on each ladder are good and searches for various deviations from "normal". This assumption allows us to perform bad channel tagging in a manner independent from absolute values of the strip noise. This simplifies the task quite a bit because the strip noise is different for various silicon subsystems and it is different for phi and z sensor sides. In its current form, the algorithm consists of the following steps:
The choice of the chip thresholds represents a compromise between the need to keep the silicon clustering/tracking efficiency high and the desire to minimize the noise occupancy (reduces the event size) and readout time (improves SVT trigger rate).
The threshold selection algorithm implemented right now in the make_sichipthresh.tcl script is very simple. It takes the desired target noise occupancy and finds the smallest ADC value so that the fraction of the pulse height distribution at this ADC value and above is less than the target occupancy. This ADC value is taken to be the preliminary chip threshold. Then the expected occupancy is calculated for each half-ladder side. An attempt is made to lower individual chip thresholds by one count so that the half-ladder occupancy still stays within the desired limit (the chips with the highest threshold values are tried first). When there is no chip left whose threshold can be lowered, the procedure is complete. Bad channels are ignored throughout this whole calculation.
The desired target occupancies can be specified separately for each layer on the script's command line. The occupancy calculation does not take the neighbor readout into account. When neighbor readout is in use, the occupancy argument given to the script should be equal to the desired noise occupancy divided by three.
In the future, the threshold calculation will also use an efficiency
estimate from silicon clustering/tracking.
At the time of this writing, the silicon D-mode calibration software can distinguish two calibration types: "standard" and "nonstandard". The calibration type is specified by the user when the data analysis request is made either on the job status web page or by running the submit_job.tcl script. The "standard" mode is intended for calibrations taken with default silicon settings, while the "nonstandard" mode should be specified if silicon settings or conditions are in any way different from the default ones.
The calibration quality is verified for the "standard" calibrations only. The compare_calibrations.tcl script checks the resulting SICHIPPED and SISTRIPDH tables against the set of tables produced by a reference calibration. It is assumed that the reference tables are known to be correct (e.g., verified by thorough visual inspection). For each half-ladder side, the comparison script calculates a set of dissimilarity coefficients which quantify the differences between the calibration tables. These coefficients are defined as follows. Let || Q || be the norm of some quantity of interest, such as pedestal or noise, for a given half-ladder side:
|| Q || = Sum over all strips (abs(Qi))
Then the absolute dissimilarity coefficient Dabs between two tables A and B is defined as
||
A
- B ||
Dabs = -----------------,
number of strips
which is an average absolute difference. For our purposes this coefficient is more useful than chi-square because it puts less emphasis on deviations due to outliers and depends more on the common trend.
The relative dissimilariry coefficient Drel is defined as follows:
|| A - B
||
Drel = ----------.
|| A ||
+ || B ||
This coefficient is useful for comparison of the quantities which are naturally non-negative, such as noise. Its value can not be less than 0 or more than 1.
The same definition of relative and absolute dissimilarity coefficients is used by the hscompare script.
Sensors with similar properties are grouped into ladder sets. Right
now, the script defines five such sets: SVX_PHI, SVX_Z, ISL_PHI, ISL_Z,
and L00. The composition of the sets is obvious from the set names. Only
sensors present in both reference and current calibration are included
in these sets. "Grassy" ladders and ladders with readout problems are excluded
from all sets. For the calibration quality check purposes, the ladder is
considered to be "grassy" if the number of pinhole-like defects found by
the pinhole_finder program (using reference
tables only) exceeds 2 per chip. For pedestals and noise, the dissimilarity
coefficients for each ladder set are defined as the median dissimilarity
coefficients over all individual half-ladders in the set. For the number
of bad channels, the set coefficients are averaged over included half-ladders
with weights proportional to the number of strips on each half-ladder.
For each ladder set, the dissimilarity coefficients are required to be
less than certain limits defined in the comparison script. The table below
shows the limit values used at the time of this writing:
|
|
|
|
|
|
|
|
| DPS-off pedestals |
|
|
|
|
|
|
| DPS-on pedestals |
|
|
|
|
|
|
| DPS-off noise |
|
|
|
|
|
|
| DPS-on noise |
|
|
|
|
|
|
| Dnoise |
|
|
|
|
|
|
| # of bad channels |
|
|
|
|
|
|
| # of bad channels |
|
|
|
|
|
|
The limits are determined by looking at the distributions of dissimilarity coefficients obtained from previous comparisons. The calibration is automatically marked as a successful one only if all dissimilarity coefficients are below the limits. If one or more coefficients fall outside the allowed range then a failure report is written into the job tracking database and the calibration has to be analyzed by an expert.
Several additional requirements are imposed on the tables in order to
ensure that a bad calibration will not get through. The detector coverage
in the reference calibration must not be very different from the current
calibration (at least 50% of the ladders must be common to both calibrations).
The chip settings must coincide in at least 75% of the ladders which are
common to both calibrations. Also, the total fraction of bad strips inside
each ladder set must be within a certain range, as shown in the table below:
|
|
|
|
| SVX_PHI |
|
|
| SVX_Z |
|
|
| ISL_PHI |
|
|
| ISL_Z |
|
|
| L00 |
|
|
The half-ladder dissimilarity coefficients and fractions of bad channels
obtained in the "standard" calibrations are written into the DISSIM
table of the job tracking database. The dissimilarity coefficient for the
ladder sets are also stored in a dedicated ntuple which allows for quick
lookup of their distributions in order to help with cut selection.
Associating Data Runs with Calibrations
The silicon calibration tables are needed for the following purposes:
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
The data runs are associated with calibrations through the valid sets/used sets mechanism described in the CDF note 5088. For most CDF subsystems, the assignment of the calibrations to the data runs is trivial: they just use the latest calibration available at the run start time. For silicon this gets more complicated because the silicon configuration (ladder sets, chip settings, presence or absence of the sensor bias) so far has been very volatile -- it changes more often than the calibrations are taken. In addition, various external conditions, such as beam losses or presence of the magnetic field, can affect calibration results as well. It is often not easy to compare the conditions during data and calibration runs because the information is not readily available.
The current algorithm which associates data runs with calibrations focuses on getting the constants right for production. For each data run, it tries to find the "best matching calibration", starting with the most recent calibration first. The match is determined by finding how many ladders in the data run are calibrated correctly by the given calibration. The chip settings must coincide, and unbiased ladders are ignored. The absence of calibration for one ladder makes a different impact on the detector performance depending on the location of the ladder -- for example, it is obvious that an L00 ladder is usually more important than an ISL ladder. Therefore, in the calculation of the match all ladders are assigned weights approximately proportional to the phi angles subtended. Each SVX half-ladder side is assigned weight 6, ISL layer 6 half-ladder sides are weighted by 3, ISL layer 7 by 2, L00 narrow segments are weighted by 4, and L00 wide segments are weighted by 8. Although this weighting scheme is pretty naive, it reflects the relative importance of the ladders better than a uniform treatment. The calibration which provides the best match for a given data run is assigned to this data run.
The actual writing of valid/used sets into the database is performed by the used_sets_maker.tcl script. This script normally runs in the background on fcdfsgi2. It works as a server, so that the run_calib_analysis.tcl script can tell it when a new calibration is ready. Every 20 min used_sets_maker.tcl checks the run conditions database for a presence of new physics and cosmics runs with 1000 events or more. If the silicon was not integrated or biased in a new run, the script writes a dummy PROD_PHYSICS_SVX used set for this run. The dummy set is needed in order to let the silicon clustering know that this run should be skipped during production. Both pedestal and bad channel tables in such a set have status "BAD". For a data run with silicon, the script writes the normal used set immediately if the latest calibration provides a perfect configuration match for this run. If the match is not perfect then the script waits until an appropriate calibration is performed. Silicon calibrations are usually taken at the end of each Tevatron store, so there is a fair chance that the next calibration will have a better match than the current one. If there is still no calibration with perfect configuration match after 48 hours (the waiting period is tunable), the script determines the best calibration as described in the previous paragraph. All calibrations taken within one month of the data run start time are considered as candidates. The PROD_PHYSICS_SVX used set is written automatically if the weighted mismatch in the configuration is less than 5%. If the mismatch is larger, the script sends an e-mail to an expert who is supposed to decide what to do with this run. When the used sets are created, the DPS-on and DPS-off tables are mixed and written into the database as necessary, and the relevant valid sets are also made automatically.
The used_sets_maker.tcl script can also write a new FIB_PHYSICS_SVX valid set for use by the FIBs and the online programs. This can happen if the most recent D-mode calibration provides a better configuration match to the last run with silicon than the calibration in the last FIB_PHYSICS_SVX valid set. Before writing the new valid set, the script checks that the last set does not use tables with the same or higher run numbers, so that newer X-mode calibrations are not overwritten, and the same set of D-mode tables is not used several times.