![]() |
![]() |
Method details
Observer2
Abstract:
This is one of the three manual annotation from which the reference standard is created. On the About page of our website, we describe how the observer results are combined into the reference standard. Observers are ranked with respect to the reference standard in the same way the other methods are ranked.
Detection confusion tables
Calc. cat. | QCA (per segment) | CTA (per lesion) | ||||||||
---|---|---|---|---|---|---|---|---|---|---|
  | TP | FP | FN | TP+FP | TP+FN | TP | FP | FN | TP+FP | TP+FN |
All | 21 | 20 | 7 | 41 | 28 | 33 | 8 | 14 | 41 | 47 |
The results of this method are based on the following centerlines: own or none.
Detection (QCA per segment / CTA per lesion)
Calc. cat. | QCA Sens. | QCA P.P.V. | CTA Sens. | CTA P.P.V. | Avg. rank | ||||
---|---|---|---|---|---|---|---|---|---|
  | % | rank | % | rank | % | rank | % | rank |   |
0 (0 - 0.1) | 60.0 | 4.0 | 50.0 | 1.0 | 57.1 | 5.0 | 66.7 | 2.0 | 3.0 |
1 (0.1 - 10) | 100.0 | 1.0 | 20.0 | 6.0 | 100.0 | 1.0 | 83.3 | 7.0 | 3.8 |
2 (11 - 100) | 71.4 | 4.0 | 62.5 | 1.0 | 72.7 | 4.0 | 72.7 | 2.0 | 2.8 |
3 (101 - 400) | 80.0 | 3.0 | 50.0 | 2.0 | 57.9 | 4.0 | 84.6 | 2.0 | 2.8 |
4 (400+) | 80.0 | 1.0 | 66.7 | 2.0 | 100.0 | 1.0 | 100.0 | 1.0 | 1.2 |
All | 75.0 | 3.0 | 51.2 | 2.0 | 70.2 | 3.0 | 80.5 | 2.0 | 2.5 |
For ranking | 75.0 | 3.0 | 51.2 | 2.0 | 70.2 | 3.0 | 80.5 | 2.0 | 2.5 |
These results are based on 30 datasets and 19 submissions.
Quantification
Calc. cat. | QCA Avg. Abs. diff. | QCA R.M.S. diff. | CTA Weigthed Kappa | Avg. rank | |||
---|---|---|---|---|---|---|---|
  | % | rank | % | rank | Κ | rank |   |
0 (0 - 0.1) | 31.1 | 6.0 | 35.8 | 6.0 | 0.69 | 3.0 | 4.5 |
1 (0.1 - 10) | 34.1 | 7.0 | 36.1 | 7.0 | 0.70 | 3.0 | 5.0 |
2 (11 - 100) | 31.6 | 5.0 | 37.3 | 5.0 | 0.70 | 3.0 | 4.0 |
3 (101 - 400) | 31.2 | 4.0 | 37.2 | 4.0 | 0.77 | 2.0 | 3.0 |
4 (400+) | 27.1 | 2.0 | 34.0 | 4.0 | 0.98 | 2.0 | 2.5 |
All | 31.1 | 7.0 | 36.5 | 4.0 | 0.77 | 2.0 | 3.8 |
For ranking | 31.1 | 7.0 | 36.5 | 4.0 | 0.77 | 2.0 | 3.8 |