Method details

Observer2



Abstract:
This is one of the three manual annotation from which the reference standard is created. On the About page of our website, we describe how the observer results are combined into the reference standard. Observers are ranked with respect to the reference standard in the same way the other methods are ranked.

Detection confusion tables

Calc. cat.QCA (per segment)CTA (per lesion)
 TPFPFNTP+FPTP+FNTPFPFNTP+FPTP+FN
All  21  20  41  28  33  14  41  47 
The results of this method are based on the following centerlines: own or none.

Detection (QCA per segment / CTA per lesion)

Calc. cat.QCA
Sens.
QCA
P.P.V.
CTA
Sens.
CTA
P.P.V.
Avg. rank
 %rank%rank%rank%rank 
0 (0 - 0.1)  60.0  4.0  50.0  1.0  57.1  5.0  66.7  2.0  3.0 
1 (0.1 - 10)  100.0  1.0  20.0  6.0  100.0  1.0  83.3  7.0  3.8 
2 (11 - 100)  71.4  4.0  62.5  1.0  72.7  4.0  72.7  2.0  2.8 
3 (101 - 400)  80.0  3.0  50.0  2.0  57.9  4.0  84.6  2.0  2.8 
4 (400+)  80.0  1.0  66.7  2.0  100.0  1.0  100.0  1.0  1.2 
All  75.0  3.0  51.2  2.0  70.2  3.0  80.5  2.0  2.5 
For ranking  75.0  3.0  51.2  2.0  70.2  3.0  80.5  2.0  2.5 
These results are based on 30 datasets and 19 submissions.


Quantification

Calc. cat.QCA
Avg. Abs. diff.
QCA
R.M.S. diff.
CTA
Weigthed Kappa
Avg. rank
 %rank%rankΚrank 
0 (0 - 0.1)  31.1  6.0  35.8  6.0  0.69  3.0  4.5 
1 (0.1 - 10)  34.1  7.0  36.1  7.0  0.70  3.0  5.0 
2 (11 - 100)  31.6  5.0  37.3  5.0  0.70  3.0  4.0 
3 (101 - 400)  31.2  4.0  37.2  4.0  0.77  2.0  3.0 
4 (400+)  27.1  2.0  34.0  4.0  0.98  2.0  2.5 
All  31.1  7.0  36.5  4.0  0.77  2.0  3.8 
For ranking  31.1  7.0  36.5  4.0  0.77  2.0  3.8