Method details

Observer2

Manual


Abstract:
This is one of the three manual annotation from which the reference standard is created. On the About page of our website, we describe how the observer results are combined into the reference standard. Observers are ranked with respect to the reference standard in the same way the other methods are ranked.

Detection confusion tables

Calc. cat.QCA (per segment)CTA (per lesion)
 TPFPFNTP+FPTP+FNTPFPFNTP+FPTP+FN
All  21  20  7  41  28  33  8  14  41  47 
The results of this method are based on the following centerlines: own or none.

Detection (QCA per segment / CTA per lesion)

Calc. cat.QCA
Sens.		QCA
P.P.V.		CTA
Sens.		CTA
P.P.V.		Avg. rank
 %rank%rank%rank%rank 
0 (0 - 0.1)  60.0  4.0  50.0  1.0  57.1  5.0  66.7  2.0  3.0 
1 (0.1 - 10)  100.0  1.0  20.0  6.0  100.0  1.0  83.3  7.0  3.8 
2 (11 - 100)  71.4  4.0  62.5  1.0  72.7  4.0  72.7  2.0  2.8 
3 (101 - 400)  80.0  3.0  50.0  2.0  57.9  4.0  84.6  2.0  2.8 
4 (400+)  80.0  1.0  66.7  2.0  100.0  1.0  100.0  1.0  1.2 
All  75.0  3.0  51.2  2.0  70.2  3.0  80.5  2.0  2.5 
For ranking  75.0  3.0  51.2  2.0  70.2  3.0  80.5  2.0  2.5 
These results are based on 30 datasets and 19 submissions.


Quantification

Calc. cat.QCA
Avg. Abs. diff.		QCA
R.M.S. diff.		CTA
Weigthed Kappa		Avg. rank
 %rank%rankΚrank 
0 (0 - 0.1)  31.1  6.0  35.8  6.0  0.69  3.0  4.5 
1 (0.1 - 10)  34.1  7.0  36.1  7.0  0.70  3.0  5.0 
2 (11 - 100)  31.6  5.0  37.3  5.0  0.70  3.0  4.0 
3 (101 - 400)  31.2  4.0  37.2  4.0  0.77  2.0  3.0 
4 (400+)  27.1  2.0  34.0  4.0  0.98  2.0  2.5 
All  31.1  7.0  36.5  4.0  0.77  2.0  3.8 
For ranking  31.1  7.0  36.5  4.0  0.77  2.0  3.8