Reliability is defined as the degree to which a measurement technique can ensure consistent results when repeatedly measured on the same objects, either by multiple spleeners or by testing by an observer at different times. There are two distinct types of reliability that need to be distinguished; Absolute consistency or consent. Thus, three advisors independently evaluate the applications of twenty students for a scholarship on a scale of zero to 100. The first advisor is particularly hard and the third is particularly forgiving, but each advisor scores substantial points. There must be differences between the actual scores given by the three advisors. If the objective is to rank the candidates and select five students, the difference between the counsellors cannot yield significantly different results if the “coherence” has been maintained throughout the evaluation process. However, if the goal is to select students rated above or below an absolute default score, the scores of the three advisors must be absolutely similar at the mathematical level. While we want a uniform assessment in the first case, we want to reach an “absolute agreement” in a later case. The differences in objective are reflected in the procedure for calculating honorability.

A flow diagram showing the process of selecting the ICC form based on the experimental design of a reliability study. The procedure includes the choice of the appropriate model (i.e., random one-way effects, two-way random effects or two-way fixed effects), type (i.e., i.e., i.e. individual/measured craters or the average value of debtors/measures K) and the definition of the relationship considered important (i.e., consistency or absolute coherence). A statement on fixed distortions is therefore not relevant to the calculation of the ICC, but can be considered as information on how the researcher intends to proceed [6]. Fixed distortions should mean that the researcher intends to continue with the same method, the same staff and the same experimental structure, so that distortions (if any) can be expected to remain the same. Of course, we should not expect absolute agreement between trials with different prejudices. In addition, the CCI estimate from a reliability study is only an expected value of the true ICC. It makes sense to determine the degree of reliability (i.e. mediocre, moderate, good and excellent) by testing whether the value of the ICC obtained is significantly higher, using statistical conclusions, than the values proposed above. This type of analysis can be easily implemented using SPSS or other statistical software.

As part of the reliability analysis, SPSS calculates not only an ICC value, but also its 95% confidence interval. Table 4 shows an example of a background analysis of SPSS. In this hypothetical example, the CCI obtained was calculated by a single scoring model with absolute match and 2-way random effects with 3 advisors in 30 subjects. Although the ICC value obtained is 0.932 (which indicates excellent reliability), its 95% confidence interval is between 0.879 and 0.965, which means that the probability that the real ICC will be at each point between 0.879 and 0.965 is 95%.