1. Usefulness of the Monte-Carlo method to calculate uncertainties

At present there are two approaches defined by the Guide to the expression of uncertainty in measurement (in summary GUM) [1, 2] to calculate uncertainties:

• a method consisting in propagating variances , which was first introduced in 1995;
• and more recently a method consisting in propagating distributions by means of the Monte-Carlo method .

The first method is often called "analytical GUM" in opposition to the second which is mainly a numerical method.

Note: there is a third method based on interlaboratory comparisons . This method is motivated by the difficulty in some cases to find the model of the measurement. Thus, it suffers from two major handicaps: it is not built to ensure a continuum of traceability (traceability is introduced through a correction added at the end of the calculation, that is to say a failure), it requires to organize inter-laboratory comparisons (which is very cumbersome to organize, often heavier than spending a little time to build the measurement model so far...). And on top of that, the time spent to build the model of measurement is a fixed cost. In other words, in the case of the GUM approach a modification of the parameters of the measurement results in most cases by a change in values assigned to variables. In the case of a 5725 approach, we must proceed with new campaigns of interlaboratory comparisons: from an economic point of view it quickly becomes disastrous. For these reasons, the 5725 approach should be avoided.

So far in the industry, uncertainties were mainly determined by the method of the GUM by propagating variances. This method has been widely recognized in many publications, it might be legitimate to question the usefulness of introducing a new method. In response, this method poses a number of assumptions - in particular the linearity of the model (see demos) - that make its application can be mistaken for a calculation of uncertainty given. To validate the application of the GUM to a calculation of uncertainty, the easiest way is to compare it to a calculation by another method. The Monte-Carlo is quite appropriate for this comparison because it poses no assumptions on the model of the measurement.

Conversely one could now question the usefulness of retaining the GUM method if the Monte-Carlo did not have the drawbacks of the GUM. In response, the implementation of the Monte-Carlo method (detailed in references ,  et ) is computationally extremely heavy and requires special software. For this reason, it is preferable to retain as much as possible the method of the GUM.

Many normative references (such as 17025 ) used as part of company quality systems require that the methods are validated. In order to comply with these requirements, the simplest is to design a first step the calculation of uncertainties using the method of the GUM. Then in a second time to validate the Monte-Carlo. Having done so, the application of the GUM method is validated for the model considered so far, it will be possible eventually to use only the method of the GUM. In conclusion, the Monte-Carlo method is a tool, necessary for use during the design calculations of uncertainties, in addition to the method of the GUM.

The implementation of the Monte-Carlo method presented in ,  et  leads to the use of software running in command line mode ("Pack Monte-Carlo). This modular approach allows great flexibility and can work with higher sample sizes (several million values). However usability is unsatisfactory. For this reason the reader will find in the lines that follow the software-Ed MC (Monte-Carlo Editor) using the flexibility of the Windows interface while keeping the computational power of the Monte-Carlo Pack.

2. MC-Ed software: main features and functionalities

MC-Ed software is primarily a graphical interface that allows to control the Pack Monte-Carlo software using the Windows environment. Its main features and functionalities are:

An intuitive interface...

 All the quantities involved in the measurement model are presented in a concise and visually in a table. The software lets you juggle four screens corresponding to the four main steps in the calculation of uncertainties: seizure model, the definition of variables and generation of samples for input variables and the output; construction of intermediate data needed for calculations and graphs (statistical distribution functions and classes); calculations (mean, standard deviation and expanded intervals); graphical representations. The use of nested tabs provides access to different screens each size without resorting to many dialog boxes. Fig. 1. - MC-Ed main interface. (Click on the image to magnify it.)

Definition of input variables using wizards...

 Fig. 2. - Definition of the distributions of the input quantities. (Click on the image to magnify it.) After entering the model measuring throught a PFS-AC equation , the software prompts the user to define the laws of the input quantities. MC-Ed can use the following laws: gaussian; uniform; derived from arcsine; isosceles triangle; V (symmetric and asymmetric, and by extension all laws triangles).

Perform various calculations on each of the input quantities...

 For each input and the output, it is possible to calculate the mean, standard deviation and a wider range.   To ensure greater accuracy of the calculations, the mean and standard deviation are calculated directly on the samples using a novel technique for reducing samples  and not on statistics classes. Depending on the distributions, it is possible to calculate an extended interval using dedicated algorithms for probability densities symmetrical or asymmetrical. The flexibility of software that lets you juggle between different screens can have access during calculations to graphs to select the appropriate algorithm.   Icons indicate permanently the user if the previous steps have been met to continue the calculations. Fig. 3. - Calculations on the samples. (Click on the image to magnify it.)

View results for each quantity...

 Fig. 4. - Example of chart. (Click on the image to magnify it.) For each input and the output MC-Ed allows you to draw: the probability density directly from the sample; distribution function; frequency distribution table or histogram of the distribution. This can include mean value end expanded interval calculated at the previous step. It is also possible to include custom values.   The different curves can be annotated, legends, resolutions and scales can be changed. Graphics can be exported in the form of images by copy / paste in most desktop applications.

Save an entire computation (including samples)...

MC-Ed saves all data in a compressed format. This feature ensures the traceability of a document validation carried out within a quality system .

Add new calculations modules (for experienced users)...

 MC-ED makes it possible to modify the parameter setting of the software of the Monte-Carlo Pack, to even control your own computation softwares which can substitute or supplement the software of the Monte-Carlo Pack.   For each subroutine the command line can include some useful variables required by the program (source file, file destination, sample size...). The relative paths are also supported. This allows a portable use of MC-ED with an USB key for example. Finally the temporary directory for calculations can be modified. It however is recommended to proceed to all the backups necessary before resorting to these options which can assign to the integrity of MC-ED (in the worst of the cases it will be necessary to uninstall MC-ED and to reinstall it). Fig. 5. - Parameter Setting of the Monte-Carlo Pack. (Click on the image to magnify it.)

4. Example: calibration of a micropipette

Micropipettes are calibrated by gravimetric method by using the water, which one knows the density as a reference material . This method consists in pouring the contents of the pipette to be calibrated in a container on a balance and to measure the variation of mass. The volume delivered by the micropipette is determined by the relation:

 $$\begin{eqnarray}V_{20} = M \cdot \frac{ 1 }{ \rho_{\text{W}} - \rho_{\text{A}} } \cdot \left( 1 - \frac{\rho_{\text{A}}}{\rho_{\text{B}}} \right) \cdot \bigl[ 1 - \gamma \cdot (t - 20) \bigr] \end{eqnarray}$$  , (1)

in which:

 $$V_{20}$$ : is the volume delivered by the pipette at the temperature of 20 °C (in µl); $$M$$ : is the variation of mass indicated by the balance (in mg); $$t$$ : is the temperature of water during measurement (in °C); $$\rho_{\text{W}}$$ : is the density of water (in mg/µl); $$\rho_{\text{A}}$$ : is the density of the air during measurement (in mg/µl); $$\rho_{\text{B}}$$ : is the density of the masses used to calibrate the balance (in mg/µl); $$\gamma$$ : is the thermal dilation coefficient of the micropipette (in l/K). This One is generally given by the manufacturer.

In reality, the measured mass $$M$$ is affected of an error due to the resolution of the balance ($$\delta m_{\text{res}}$$) and of an error due to the calibration of the balance ($$\delta m_{\text{cal}}$$). In the same way the temperature is affected of an error due to the calibration of the thermometer ($$\delta t_{\text{cal}}$$). The taking into account of these errors makes it possible to deduce the measurement model:

 $$\begin{eqnarray}V_{20} = \left( M + \delta m_{\text{res}} + \delta m_{\text{cal}} \right) \cdot \frac{ 1 }{ \rho_{\text{W}} - \rho_{\text{A}} } \cdot \left( 1 - \frac{\rho_{\text{A}}}{\rho_{\text{B}}} \right) \cdot \bigl[ 1 - \gamma \cdot (t + \delta t_{\text{cal}} - 20) \bigr] \end{eqnarray}$$  . (2)

This equation was programmed with PFS-AC . It can be downloaded with the following link (first decompress the zip archive to use the file modele_pipette.fct).

Within the framework of this example, we will use the data resulting from the reference  reproduced in table 1.

 Variable Estimate Distribution Standard deviation $$M$$ $$5,033~\text{µg}$$ Gaussian $$8,8 \times 10^{-3}~\text {µg}$$ $$t$$ $$19,5~\text{°C}$$ Rectangular $$0 \text{°C}$$ $$\rho_{\text{W}}$$ $$0,998~3~\text{µg/µl}$$ Rectangular $$1,16 \times 10^{-5}~\text{µg/µl}$$ $$\rho_{\text{A}}$$ $$1,2 \times 10^{-3}~\text{µg/µl}$$ Rectangular $$2,89 \times 10^{-7}~\text{µg/µl}$$ $$\rho_{\text{B}}$$ $$7,96~\text{µg/µl}$$ Rectangular $$0,0346~\text{µg/µl}$$ $$\gamma$$ $$2,40 \times 10^{-4}~\text{l/K}$$ Rectangular $$2,89 \times 10^{-6}~\text{l/K}$$ $$\delta m_{\text{res}}$$ $$0~\text{µg}$$ Rectangular $$0,001~\text{µg}$$ $$\delta m_{\text{cal}}$$ $$0~\text{µg}$$ Gaussian $$0,005~\text{µg}$$ $$\delta t_{\text{cal}}$$ $$0~\text{°C}$$ Gaussian $$0,05~\text{°C}$$
Table 1
Calibration data of the micropipette.

4.1. GUM method

First of all, the sensitivity coefficients have been calculated using the derivation function of PFS-AC  applied to each input quantity of the model. These values are reproduced in table 2.

 Variable Sensitivity coefficient $$M$$ $$1,002~9$$ $$t$$ $$-1,211~3 \times 10^{-3}$$ $$\rho_{\text{W}}$$ $$-5,062~2$$ $$\rho_{\text{A}}$$ $$4,428~0$$ $$\rho_{\text{B}}$$ $$9,560~8 \times 10^{-5}$$ $$\gamma$$ $$2,523~4$$ $$\delta m_{\text{res}}$$ $$1,002~9$$ $$\delta m_{\text{cal}}$$ $$1,002~9$$ $$\delta t_{\text{cal}}$$ $$-1,211~3 \times 10^{-3}$$
Table 2
Values of the sensitivity coefficients.

The data of tables 1 and 2 were used to carry out the calculation of uncertainty by using the law of propagation of the variances (GUM method) with the software Gumy . The file of calculation is downloadable in the following link (uncompress zip archive to use the file etalonnage_pipette.gmy).

The detail of the calculations produced by Gumy is deferred in table 3. The volume calculated by this method with a coverage factor $$k = 2$$ equal to:

 $$V_{20} = (5,047 ± 0,021)~µl$$ Table 3
Calculation made with Gumy .

We can notice by the way that this method has the advantage of providing the main components of uncertainty. Indeed, according to table 3 it appears clearly that the leading cause of uncertainty is the measurement of the variation of mass (repeatability) and to a lesser extent the calibration of the balance. This information is not accessible simply by the Monte-Carlo method.

4.2. Propagation of the distributions with MC-ED

Since rectangular distributions are being defined by their half interval on MC-ED, it is necessary to transform the data of table 1 to make them suitable (the formulas of conversion are recalled in the distribution assistant one MC-ED). The transformed data are provided in table 4.

 Variable Estimate Distribution Standard uncertainty Half width* $$M$$ $$5,033~\text{µg}$$ Gaussian $$8,8 \times 10^{-3}~\text{µg}$$ / $$t$$ $$19,5~\text{°C}$$ Rectangular / $$0~\text{°C}$$ $$\rho_{\text{W}}$$ $$0,998~3~\text{µg/µl}$$ Rectangular / $$2,01 \times 10^{-5}~\text{µg/µl}$$ $$\rho_{\text{A}}$$ $$1,2 \times 10^{-3}~\text{µg/µl}$$ Rectangular / $$5,01 \times 10^{-7}~\text{µg/µl}$$ $$\rho_{\text{B}}$$ $$7,96~\text{µg/µl}$$ Rectangular / $$0,059~9~\text{µg/µl}$$ $$\gamma$$ $$2,40 \times 10^{-4}~\text{l/K}$$ Rectangular / $$5,01 \times 10^{-6}~\text{l/K}$$ $$\delta m_{\text{res}}$$ $$0$$ Rectangular / $$0,001~73~\text{µg}$$ $$\delta m_{\text{cal}}$$ $$0$$ Gaussian $$0,005~\text{µg}$$ / $$\delta t_{\text{cal}}$$ $$0$$ Gaussian $$0,05~\text{°C}$$ /
Table 4
Adaptation of the data for the Monte-Carlo method.
* in the case of the rectangular distribution, the half interval is equal to: $$\text{standard uncertainty} \times \sqrt{3}$$

The calculation carried out with MC-ED gives for a sample of 100 000 values:

 $$\left\{ \begin{array}{l} \text{Mean}~V_{20} = 5,047~\text{µl}~; \\ \text{Standard deviation} = 0,010~2~\text{µl}~; \\ \text{Expanded interval :}~\bigl[5,026~\text{µl}~;~5,067~\text{µl}\bigr]. \end{array} \right.$$   .

The frequency histogram with the mean value and the expanded interval obtained by MC-ED is represented on figure 6. Fig. 6. - Statistique par classes de l'échantillon de la grandeur de sortie.

Table 5 presents the results obtained by the method of the GUM and the Monte-Carlo method. The results are very close, which makes it possible to conclude that the application of the law of propagation of the variances (method GUM) is validated for the model of measurement (locally at the point of measurement considered).

 Quantity GUM method Monte-Carlo method $$V_{20}$$ $$5,047~\text{µl}$$ $$5,047~\text{µl}$$ Standard deviation $$0,010~3~\text{µl}$$ $$0,010~2~\text{µl}$$ Expanded interval(µl) $$[5,026~;~5,068]$$($$k = 2$$) $$[5,026~;~5,067]$$(95 %)
Table 5
GUM method vs Monte-Carlo method.

References

  JCGM, "Evaluation of measurement data - Guide to the expression of uncertainty in measurement", BIPM, JCGM 100:2008, September 2008, www.bipm.org.  JCGM, "Evaluation of measurement data - Supplement 1 to the "Guide to the expression of uncertainty in measurement" - Propagation of distributions using a Monte-Carlo method", BIPM, JCGM 101:2008, 2008, www.bipm.org.  AFNOR, « Application de la statistique - Exactitude (justesse et fidelité) des résultats et méthodes de mesure », NF ISO 5725-1 à 6, 1994.  PLATEL F., « Génération de nombres aléatoires pour la méthode de Monte-Carlo », MetGen, Metrology article No 21.  PLATEL F., « Quelques générateurs de nombres aléatoires pour la méthode de Monte-Carlo correspondant à des lois utilisées en métrologie », MetGen, Metrology article No 23.  PLATEL F., « Outils complémentaires pour la méthode de Monte-Carlo. Le "Pack Monte-Carlo". », MetGen, Metrology article No 24.  CEN/CENELEC, "General requirements for the competence of testing and calibration laboratories", EN ISO/IEC 17025, May 2005.  PLATEL F., « Calcul symbolique sur ordinateur - Projet PFS-Algebraic Calculator », MetGen, Miscellaneous Article No 2.  PLATEL F., "Calculation of the density of water - DensiCal Project", MetGen, Metrology article No 18.  BATISTA E., PINTO L., FILIPE E. and VAN DER VEEN A.M.H., "Calibration of micropipettes: Test methods and uncertainty analysis", Measurement, 40, 338-342, 2007.  PLATEL F., « Calcul d'incertitudes avec Gumy », MetGen, Metrology article No 4.