Method details

Observer3



Abstract:
This is one of the three manual annotation from which the reference standard is created. On the About page of our website, we describe how the observer results are combined into the reference standard. Observers are ranked with respect to the reference standard in the same way the other methods are ranked.

Detection confusion tables

Calc. cat.QCA (per segment)CTA (per lesion)
 TPFPFNTP+FPTP+FNTPFPFNTP+FPTP+FN
All  18  24  10  42  28  31  21  16  52  47 
The results of this method are based on the following centerlines: own or none.

Detection (QCA per segment / CTA per lesion)

Calc. cat.QCA
Sens.
QCA
P.P.V.
CTA
Sens.
CTA
P.P.V.
Avg. rank
 %rank%rank%rank%rank 
0 (0 - 0.1)  40.0  8.0  28.6  5.0  71.4  3.0  35.7  5.0  5.2 
1 (0.1 - 10)  100.0  1.0  33.3  3.0  60.0  6.0  100.0  1.0  2.8 
2 (11 - 100)  71.4  4.0  55.6  3.0  72.7  4.0  72.7  2.0  3.2 
3 (101 - 400)  80.0  3.0  42.1  5.0  73.7  2.0  63.6  3.0  3.2 
4 (400+)  40.0  12.0  50.0  5.0  20.0  17.0  50.0  5.0  9.8 
All  64.3  5.0  42.9  4.0  66.0  4.0  59.6  4.0  4.2 
For ranking  64.3  5.0  42.9  4.0  66.0  4.0  59.6  4.0  4.2 
These results are based on 30 datasets and 19 submissions.


Quantification

Calc. cat.QCA
Avg. Abs. diff.
QCA
R.M.S. diff.
CTA
Weigthed Kappa
Avg. rank
 %rank%rankΚrank 
0 (0 - 0.1)  32.5  8.0  39.9  8.0  0.42  4.0  6.0 
1 (0.1 - 10)  27.4  4.0  31.3  4.0  0.63  4.0  4.0 
2 (11 - 100)  28.9  2.0  35.6  3.0  0.88  2.0  2.2 
3 (101 - 400)  32.2  5.0  39.0  5.0  0.76  3.0  4.0 
4 (400+)  29.2  4.0  33.4  3.0  0.81  3.0  3.2 
All  30.6  4.0  36.9  6.0  0.73  4.0  4.5 
For ranking  30.6  4.0  36.9  6.0  0.73  4.0  4.5