7705:
3985:
3273:
7691:
1659:
3763:
7729:
7717:
895:
returns a value between â1 and +1. A coefficient of +1 represents a perfect prediction, 0 no better than random prediction and â1 indicates total disagreement between prediction and observation. However, if MCC equals neither â1, 0, or +1, it is not a reliable indicator of how similar a predictor is to random guessing because MCC is dependent on the dataset. MCC is closely related to the
1184:
1398:
3980:{\displaystyle {\text{MCC}}={\frac {(6+3)\times {\color {green}12}\;-\;{\color {blue}5}\times {\color {brown}4}\;-\;{\color {purple}7}\times {\color {maroon}8}}{{\sqrt {{\color {green}12}^{2}-{\color {blue}5}^{2}-{\color {purple}7}^{2}}}{\sqrt {{\color {green}12}^{2}-{\color {brown}4}^{2}-{\color {maroon}8}^{2}}}}}={\frac {32}{\sqrt {4480}}}\approx 0.478}
2902:
1383:
1221:. If exactly one of the four sums in the denominator is zero, the denominator can be arbitrarily set to one; this results in a Matthews correlation coefficient of zero, which can be shown to be the correct limiting value. In case two or more sums are zero (e.g. both labels and model predictions are all positive or negative), the limit does not exist.
991:
4404:
On the other hand, checking the
Matthews correlation coefficient would be pivotal once again. In this example, the value of the MCC would be 0.14 (Equation 3), indicating that the algorithm is performing similarly to random guessing. Acting as an alarm, the MCC would be able to inform the data mining
4400:
In this example, the classifier has performed well in classifying positive instances, but was not able to correctly recognize negative data elements. Again, the resulting F1 score and accuracy scores would be extremely high: accuracy = 91%, and F1 score = 95.24%. Similarly to the previous case, if a
1803:
In this confusion matrix, of the 8 cat pictures, the system judged that 2 were dogs, and of the 4 dog pictures, it predicted that 1 was a cat. All correct predictions are located in the diagonal of the table (highlighted in bold), so it is easy to visually inspect the table for prediction errors, as
4210:
Suppose, for example, you have a very imbalanced validation set made of 100 elements, 95 of which are positive elements, and only 5 are negative elements (as explained in Tip 5). And suppose also you made some mistakes in designing and training your machine learning classifier, and now you have an
894:
The coefficient takes into account true and false positives and negatives and is generally regarded as a balanced measure which can be used even if the classes are of very different sizes. The MCC is in essence a correlation coefficient between the observed and predicted binary classifications; it
3659:
1743:
assume that a classifier that distinguishes between cats and dogs is trained, and we take the 12 pictures and run them through the classifier, and the classifier makes 9 accurate predictions and misses 3: 2 cats wrongly predicted as dogs (first 2 predictions) and 1 dog wrongly predicted as a cat
4388:
In the example above, the MCC score would be undefined (since TN and FN would be 0, therefore the denominator of
Equation 3 would be 0). By checking this value, instead of accuracy and F1 score, you would then be able to notice that your classifier is going in the wrong direction, and you would
4030:, 2020), the Matthews correlation coefficient is more informative than F1 score and accuracy in evaluating binary classification problems, because it takes into account the balance ratios of the four confusion matrix categories (true positives, true negatives, false positives, false negatives).
874:
Although computationally the
Pearson correlation coefficient reduces to the phi coefficient in the 2Ă2 case, they are not in general the same. The Pearson correlation coefficient ranges from â1 to +1, where ±1 indicates perfect agreement or disagreement, and 0 indicates no relationship. The phi
2006:
1654:{\displaystyle {\begin{aligned}N&={\mathit {TN}}+{\mathit {TP}}+{\mathit {FN}}+{\mathit {FP}}\\S&={\frac {{\mathit {TP}}+{\mathit {FN}}}{N}}\\P&={\frac {{\mathit {TP}}+{\mathit {FP}}}{N}}\\{\text{MCC}}&={\frac {{\mathit {TP}}/N-S\times P}{\sqrt {PS(1-S)(1-P)}}}\end{aligned}}}
3268:{\displaystyle {\text{MCC}}={\frac {\sum _{k}\sum _{l}\sum _{m}C_{kk}C_{lm}-C_{kl}C_{mk}}{{\sqrt {\sum _{k}\left(\sum _{l}C_{kl}\right)\left(\sum _{k'|k'\neq k}\sum _{l'}C_{k'l'}\right)}}{\sqrt {\sum _{k}\left(\sum _{l}C_{lk}\right)\left(\sum _{k'|k'\neq k}\sum _{l'}C_{l'k'}\right)}}}}}
4222:
These values lead to the following performance scores: accuracy = 95%, and F1 score = 97.44%. By reading these over-optimistic scores, then you will be very happy and will think that your machine learning algorithm is doing an excellent job. Obviously, you would be on the wrong track.
1230:
864:
4424:
Note that the F1 score depends on which class is defined as the positive class. In the first example above, the F1 score is high because the majority class is defined as the positive class. Inverting the positive and negative classes results in the following confusion matrix:
588:
1179:{\displaystyle {\text{MCC}}={\frac {{\mathit {TP}}\times {\mathit {TN}}-{\mathit {FP}}\times {\mathit {FN}}}{\sqrt {({\mathit {TP}}+{\mathit {FP}})({\mathit {TP}}+{\mathit {FN}})({\mathit {TN}}+{\mathit {FP}})({\mathit {TN}}+{\mathit {FN}})}}}}
4377:
973:
of true and false positives and negatives by a single number, the
Matthews correlation coefficient is generally regarded as being one of the best such measures. Other measures, such as the proportion of correct predictions (also termed
103:
Two binary variables are considered positively associated if most of the data falls along the diagonal cells. In contrast, two binary variables are considered negatively associated if most of the data falls off the diagonal.
3507:
1388:
using the positive predictive value, the true positive rate, the true negative rate, the negative predictive value, the false discovery rate, the false negative rate, the false positive rate, and the false omission rate.
1867:
978:), are not useful when the two classes are of very different sizes. For example, assigning every object to the larger set achieves a high proportion of correct predictions, but is not generally a useful classification.
4206:
However, even if accuracy and F1 score are widely employed in statistics, both can be misleading, since they do not fully consider the size of the four classes of the confusion matrix in their final score computation.
875:
coefficient has a maximum value that is determined by the distribution of the two variables if one or both variables can take on more than two values. See
Davenport and El-Sanhury (1991) for a thorough discussion.
4001:
Several generalizations of the
Matthews Correlation Coefficient to more than two classes along with new Multivariate Correlation Metrics for multinary classification have been presented by P Stoica and P Babu .
1378:{\displaystyle {\text{MCC}}={\sqrt {{\mathit {PPV}}\times {\mathit {TPR}}\times {\mathit {TNR}}\times {\mathit {NPV}}}}-{\sqrt {{\mathit {FDR}}\times {\mathit {FNR}}\times {\mathit {FPR}}\times {\mathit {FOR}}}}}
4401:
researcher analyzed only these two score indicators, without considering the MCC, they would wrongly think the algorithm is performing quite well in its task, and would have the illusion of being successful.
4408:
For these reasons, we strongly encourage to evaluate each test performance through the
Matthews correlation coefficient (MCC), instead of the accuracy and the F1 score, for any binary classification problem.
717:
4123:
957:
3278:
When there are more than two labels the MCC will no longer range between â1 and +1. Instead the minimum value will be between â1 and 0 depending on the true distribution. The maximum value is always +1.
4198:
1403:
464:
1727:
Some scientists claim the
Matthews correlation coefficient to be the most informative single score to establish the quality of a binary classifier prediction in a confusion matrix context.
4385:
By considering the proportion of each class of the confusion matrix in its formula, its score is high only if your classifier is doing well on both the negative and the positive elements.
3498:
3392:
3337:
3440:
4421:
Chicco's passage might be read as endorsing the MCC score in cases with imbalanced data sets. This, however, is contested; in particular, Zhu (2020) offers a strong rebuttal.
3990:
An alternative generalization of the
Matthews Correlation Coefficient to more than two classes was given by Powers by the definition of Correlation as the geometric mean of
7767:
709:
679:
390:
359:
323:
228:
4232:
2874:
4226:
On the contrary, to avoid these dangerous misleading illusions, there is another performance score that you can exploit: the
Matthews correlation coefficient (MCC).
1722:
5138:"The Matthews correlation coefficient (MCC) is more reliable than balanced accuracy, bookmaker informedness, and markedness in two-class confusion matrix evaluation"
4721:"The Matthews correlation coefficient (MCC) is more reliable than balanced accuracy, bookmaker informedness, and markedness in two-class confusion matrix evaluation"
891:
in 1912. Despite these antecedents which predate Matthews's use by several decades, the term MCC is widely used in the field of bioinformatics and machine learning.
649:
292:
264:
197:
169:
2848:
43:
8097:
2894:
622:
411:
5298:
Stoica P and Babu P (2024), PearsonâMatthews correlation coefficients for binary and multinary classification, Elsevier Signal Processing, 222, 109511, doi =
3757:
Using above formula to compute MCC measure for the dog and cat example discussed above, where the confusion matrix is treated as a 2 Ă Multiclass example:
3654:{\displaystyle {\text{MCC}}={\frac {cs-{\vec {t}}\cdot {\vec {p}}}{{\sqrt {s^{2}-{\vec {p}}\cdot {\vec {p}}}}{\sqrt {s^{2}-{\vec {t}}\cdot {\vec {t}}}}}}}
4041:
In order to have an overall understanding of your prediction, you decide to take advantage of common statistical scores, such as accuracy, and F1 score.
8192:
4436:
The MCC doesn't depend on which class is the positive one, which has the advantage over the F1 score to avoid incorrectly defining the positive class.
7760:
4614:
Paper presented at the annual meeting of the Florida Educational Research Association, Orlando, FL. (ERIC Document Reproduction Service No. ED433353)
2001:{\displaystyle {\text{MCC}}={\frac {6\times 3-1\times 2}{\sqrt {(6+1)\times (6+2)\times (3+1)\times (3+2)}}}={\frac {16}{\sqrt {1120}}}\approx 0.478}
2280:
1752:
With these two labelled sets (actual and predictions) we can create a confusion matrix that will summarize the results of testing the classifier:
4392:
Consider this other example. You ran a classification on the same dataset which led to the following values for the confusion matrix categories:
6826:
4214:
By applying your only-positive predictor to your imbalanced validation set, therefore, you obtain values for the confusion matrix categories:
7331:
4777:
Powers, David M. W. (10 October 2020). "Evaluation: from precision, recall and F-measure to ROC, informedness, markedness and correlation".
7753:
7896:
7481:
859:{\displaystyle \phi ={\frac {nn_{11}-n_{1\bullet }n_{\bullet 1}}{\sqrt {n_{1\bullet }n_{\bullet 1}(n-n_{1\bullet })(n-n_{\bullet 1})}}}.}
7105:
5746:
2728:
5059:
Brooks, Harold; Brown, Barb; Ebert, Beth; Ferro, Chris; Jolliffe, Ian; Koh, Tieh-Yong; Roebber, Paul; Stephenson, David (2015-01-26).
8172:
4047:
7803:
6879:
908:
4893:"The Matthews correlation coefficient (MCC) should replace the ROC AUC as the standard metric for assessing binary classification"
7911:
7318:
4134:
594:
5316:"The advantages of the Matthews correlation coefficient (MCC) over F1 score and accuracy in binary classification evaluation"
5087:"The advantages of the Matthews correlation coefficient (MCC) over F1 score and accuracy in binary classification evaluation"
5043:
4024:"The advantages of the Matthews correlation coefficient (MCC) over F1 score and accuracy in binary classification evaluation"
5741:
5441:
8182:
6345:
5493:
583:{\displaystyle \phi ={\frac {n_{11}n_{00}-n_{10}n_{01}}{\sqrt {n_{1\bullet }n_{0\bullet }n_{\bullet 0}n_{\bullet 1}}}}.}
7901:
7128:
7020:
4541:
4474:(subtype: Tetrachoric correlation), when variables are seen as dichotomized versions of (latent) continuous variables
2386:
1858:
where P = Positive; N = Negative; TP = True Positive; FP = False Positive; TN = True Negative; FN = False Negative.
1724:
p statistics, while their geometric mean generalizes the Matthews Correlation Coefficient to more than two classes.
8167:
7733:
7306:
7180:
2702:
5281:
8123:
7984:
7364:
7025:
6770:
6141:
5731:
2823:
The Matthews correlation coefficient has been generalized to the multiclass case. The generalization called the
2494:
1735:
Given a sample of 12 pictures, 8 of cats and 4 of dogs, where cats belong to class 1 and dogs belong to class 0,
97:
85:
3447:
7808:
7415:
6627:
6434:
6323:
6281:
2166:
6355:
7818:
7658:
6617:
5520:
4798:
Perruchet, P.; Peereman, R. (2004). "The exploitation of distributional information in syllable processing".
4495:
Matthews, B. W. (1975). "Comparison of the predicted and observed secondary structure of T4 phage lysozyme".
8074:
4455:
7833:
7209:
7158:
7143:
7133:
7002:
6874:
6841:
6667:
6622:
6452:
3344:
3289:
896:
8187:
8040:
7969:
7906:
7721:
7553:
7354:
7278:
6579:
6333:
6002:
5466:
66:
7853:
7828:
7813:
7438:
7410:
7405:
7153:
6912:
6818:
6798:
6706:
6417:
6235:
5718:
5590:
3399:
2808:
Type I error: A test result which wrongly indicates that a particular condition or attribute is present
2781:
Type II error: A test result which wrongly indicates that a particular condition or attribute is absent
1665:
5366:
5220:
Gorodkin, Jan (2004). "Comparing two K-category assignments by a K-category correlation coefficient".
8177:
7843:
7838:
7170:
6938:
6659:
6584:
6513:
6442:
6362:
6350:
6220:
6208:
6201:
5909:
5630:
5005:"Evaluation: From Precision, Recall and F-Measure to ROC, Informedness, Markedness & Correlation"
4466:
2686:
2547:
2467:
2442:
2380:
4372:{\displaystyle {\text{MCC}}={\frac {TP\times TN-FP\times FN}{\sqrt {(TP+FP)(TP+FN)(TN+FP)(TN+FN)}}}}
8162:
7939:
7653:
7420:
7283:
6968:
6933:
6897:
6682:
6124:
6033:
5992:
5904:
5595:
5434:
684:
654:
365:
334:
298:
203:
8157:
8030:
7934:
7929:
7562:
7175:
7115:
7052:
6690:
6674:
6412:
6274:
6264:
6114:
6028:
2080:
1701:
1685:
7871:
7600:
7530:
7323:
7260:
7015:
6902:
5899:
5796:
5703:
5582:
5481:
4986:"Data Science for Business: What You Need to Know about Data Mining and Data-Analytic Thinking"
4612:
Equating r-based and d-based effect-size indices: Problems with a commonly recommended formula.
4471:
2853:
2061:
1673:
7625:
7567:
7510:
7336:
7229:
7138:
6864:
6748:
6607:
6599:
6489:
6481:
6296:
6192:
6170:
6129:
6094:
6061:
6007:
5982:
5937:
5876:
5836:
5638:
5461:
4414:
4013:
2588:
1707:
62:
627:
270:
242:
175:
147:
8092:
7992:
7881:
7876:
7548:
7123:
7072:
7048:
7010:
6928:
6907:
6859:
6738:
6716:
6685:
6594:
6471:
6422:
6340:
6313:
6269:
6225:
5987:
5763:
5643:
5378:
4673:
2826:
2519:
2093:
450:, the total number of observations. The phi coefficient that describes the association of
8:
7798:
7785:
7695:
7620:
7543:
7224:
6988:
6981:
6943:
6851:
6831:
6803:
6536:
6402:
6397:
6387:
6379:
6197:
6158:
6048:
6038:
5947:
5726:
5682:
5600:
5525:
5427:
2414:
2322:
2273:
2204:
2170:
5382:
4677:
4211:
algorithm which always predicts positive. Imagine that you are not aware of this issue.
8118:
8084:
7964:
7793:
7709:
7520:
7374:
7270:
7219:
7095:
6992:
6976:
6953:
6730:
6464:
6447:
6407:
6318:
6213:
6175:
6146:
6106:
6066:
6012:
5929:
5615:
5610:
5402:
5342:
5315:
5233:
5164:
5137:
5113:
5086:
4966:
4919:
4892:
4868:
4841:
4815:
4778:
4747:
4720:
4696:
4661:
4580:
4544:
2879:
2316:
2162:
1677:
607:
396:
7745:
4811:
4662:"Optimal classifier for imbalanced data using Matthews Correlation Coefficient metric"
8113:
8002:
7949:
7704:
7615:
7585:
7577:
7397:
7388:
7313:
7244:
7100:
7085:
7060:
6948:
6889:
6755:
6743:
6369:
6286:
6230:
6153:
5997:
5919:
5698:
5572:
5406:
5394:
5367:"On the performance of Matthews correlation coefficient (MCC) for imbalanced dataset"
5347:
5237:
5169:
5118:
5039:
5004:
4985:
4924:
4873:
4752:
4701:
4537:
4512:
4508:
4450:
2176:
2032:
900:
4819:
2772:
A test result that correctly indicates the presence of a condition or characteristic
2030:
negative instances for some condition. The four outcomes can be formulated in a 2Ă2
1680:. The component regression coefficients of the Matthews correlation coefficient are
8136:
7921:
7886:
7823:
7776:
7640:
7595:
7359:
7346:
7239:
7214:
7148:
7080:
6958:
6566:
6459:
6392:
6305:
6252:
6071:
5942:
5736:
5620:
5535:
5502:
5386:
5337:
5327:
5229:
5200:
5159:
5149:
5108:
5098:
5031:
4970:
4958:
4914:
4904:
4863:
4853:
4807:
4742:
4732:
4691:
4681:
4633:
4572:
4504:
4445:
2799:
A test result that correctly indicates the absence of a condition or characteristic
2038:
2017:
982:
970:
54:
47:
5060:
8069:
8012:
7557:
7301:
7163:
7090:
6765:
6639:
6612:
6589:
6558:
6185:
6180:
6134:
5864:
5515:
5390:
4962:
4943:
4686:
7047:
7506:
7501:
5964:
5894:
5540:
5299:
5154:
4909:
4737:
4019:
3283:
This formula can be more easily understood by defining intermediate variables:
2250:
2151:
1669:
1218:
1210:
5332:
5205:
5188:
5103:
5035:
4858:
8151:
7954:
7663:
7630:
7493:
7454:
7265:
7234:
6698:
6652:
6257:
5959:
5786:
5550:
5545:
5398:
4637:
2262:
2214:
2139:
1202:
1194:
7605:
7538:
7515:
7430:
6760:
6056:
5954:
5889:
5831:
5816:
5753:
5708:
5351:
5256:
5241:
5173:
5122:
4928:
4877:
4756:
4705:
4027:
3991:
2288:
1697:
1689:
884:
73:
4624:
Davenport, E.; El-Sanhury, N. (1991). "Phi/Phimax: Review and Synthesis".
4516:
601:
and estimates the extent of the relationship between two variables (2Ă2).
7648:
7610:
7293:
7194:
7056:
6869:
6836:
6328:
6245:
6240:
5884:
5841:
5821:
5801:
5791:
5560:
4389:
become aware that there are issues you ought to solve before proceeding.
3500:
the total number of samples. This allows the formula to be expressed as:
8048:
7959:
7944:
6494:
5974:
5674:
5605:
5555:
5530:
5450:
4584:
3995:
2575:
2355:
1693:
1681:
20:
5309:
5307:
1700:
correspond to different directions of information flow and generalize
446:, are non-negative counts of numbers of observations that sum to
7866:
6647:
6499:
6119:
5914:
5826:
5811:
5806:
5771:
4536:. Princeton: Princeton University Press, p. 282 (second paragraph).
888:
100:
estimated for two binary variables will return the phi coefficient.
81:
4576:
4560:
6163:
5781:
5658:
5653:
5648:
5304:
4783:
4611:
4461:
2640:
975:
887:, also known as the Yule phi coefficient from its introduction by
7974:
7861:
7668:
7369:
5061:"WWRP/WGNE Joint Working Group on Forecast Verification Research"
5026:
Ting, Kai Ming (2011). Sammut, Claude; Webb, Geoffrey I. (eds.).
4610:
Aaron, B., Kromrey, J. D., & Ferron, J. M. (1998, November).
883:
The MCC is defined identically to phi coefficient, introduced by
61:(MCC) and used as a measure of the quality of binary (two-class)
4561:"On the Methods of Measuring Association Between Two Attributes"
7590:
6571:
6545:
6525:
5776:
5567:
4842:"Ten quick tips for machine learning in computational biology"
4405:
practitioner that the statistical model is performing poorly.
4118:{\displaystyle {\text{accuracy}}={\frac {TP+TN}{TP+TN+FP+FN}}}
2850:
statistic (for K different classes) was defined in terms of a
5419:
4458:, a similar measure of association between nominal variables.
952:{\displaystyle |{\text{MCC}}|={\sqrt {\frac {\chi ^{2}}{n}}}}
8053:
8025:
8020:
7997:
5510:
16:
Statistical measure of association for two binary variables
4193:{\displaystyle {\text{F1 score}}={\frac {2TP}{2TP+FP+FN}}}
5065:
Collaboration for Australian Weather and Climate Research
1804:
they will be represented by values outside the diagonal.
1581:
1546:
1533:
1500:
1487:
1460:
1447:
1434:
1421:
1368:
1365:
1352:
1349:
1336:
1333:
1320:
1317:
1300:
1297:
1284:
1281:
1268:
1265:
1252:
1249:
1165:
1152:
1136:
1123:
1107:
1094:
1078:
1065:
1050:
1037:
1024:
1011:
4528:
4526:
4203:(Equation 2, F1 score: worst value = 0; best value = 1)
4128:(Equation 1, accuracy: worst value = 0; best value = 1)
4005:
7775:
4497:
Biochimica et Biophysica Acta (BBA) - Protein Structure
1807:
In abstract terms, the confusion matrix is as follows:
4413:
Davide Chicco, Ten quick tips for machine learning in
4382:(Equation 3, MCC: worst value = â1; best value = +1).
5213:
5058:
4523:
4235:
4137:
4050:
3766:
3510:
3450:
3402:
3347:
3292:
2905:
2882:
2856:
2829:
1870:
1710:
1401:
1233:
994:
911:
720:
687:
657:
630:
610:
604:
The phi coefficient can also be expressed using only
467:
399:
368:
337:
301:
273:
245:
206:
178:
150:
7332:
Autoregressive conditional heteroskedasticity (ARCH)
4545:
https://archive.org/details/in.ernet.dli.2015.223699
5135:
4623:
4554:
4552:
2175:probability of detection, hit rate,
6794:
4983:
4718:
4371:
4192:
4117:
3979:
3653:
3492:
3434:
3386:
3331:
3267:
2888:
2868:
2842:
2000:
1716:
1653:
1377:
1178:
951:
858:
703:
673:
643:
616:
582:
405:
384:
353:
317:
286:
258:
222:
191:
163:
4797:
2727:Threat score (TS), critical success index (CSI),
8149:
4549:
3442:the total number of samples correctly predicted,
969:While there is no perfect way of describing the
107:If we have a 2Ă2 table for two random variables
6880:Multivariate adaptive regression splines (MARS)
5136:Chicco D, Toetsch N, Jurman G (February 2021).
1664:This is equal to the formula given above. As a
1392:The original formula as given by Matthews was:
5248:
4490:
4488:
1668:, the Matthews correlation coefficient is the
7761:
5435:
4835:
4833:
4831:
4829:
4791:
4772:
4770:
4768:
4766:
4653:
4650:Date unclear, but prior to his death in 1936.
2790:the number of real negative cases in the data
2763:the number of real positive cases in the data
5313:
5300:https://doi.org/10.1016/j.sigpro.2024.109511
5084:
4890:
1224:The MCC can be calculated with the formula:
981:The MCC can be calculated directly from the
4984:Provost, Foster; Tom Fawcett (2013-08-01).
4719:Chicco, D.; Tötsch, N.; Jurman, G. (2021).
4485:
4010:As explained by Davide Chicco in his paper
3339:the number of times class k truly occurred,
2279:probability of false alarm,
7768:
7754:
5480:
5442:
5428:
5186:
4826:
4763:
4659:
4601:. New York: McGrawâHill Book Company, Inc.
3829:
3825:
3807:
3803:
3493:{\displaystyle s=\sum _{i}\sum _{j}C_{ij}}
3394:the number of times class k was predicted,
8193:Summary statistics for contingency tables
6093:
5341:
5331:
5204:
5163:
5153:
5112:
5102:
4918:
4908:
4867:
4857:
4782:
4746:
4736:
4712:
4695:
4685:
4626:Educational and Psychological Measurement
5219:
5009:Journal of Machine Learning Technologies
4565:Journal of the Royal Statistical Society
4494:
4012:"Ten quick tips for machine learning in
4941:
1861:Plugging the numbers from the formula:
84:in 1912 this measure is similar to the
8150:
7406:KaplanâMeier estimator (product limit)
5002:
4839:
4776:
595:point-biserial correlation coefficient
7749:
7479:
7046:
6793:
6092:
5862:
5479:
5423:
4006:Advantages over accuracy and F1 score
3387:{\displaystyle p_{k}=\sum _{i}C_{ki}}
3332:{\displaystyle t_{k}=\sum _{i}C_{ik}}
966:is the total number of observations.
7716:
7416:Accelerated failure time (AFT) model
5254:
5025:
4891:Chicco D, Jurman G (February 2023).
4558:
3939:
3887:
3841:
3831:
7728:
7011:Analysis of variance (ANOVA, anova)
5863:
5364:
5314:Chicco D, Jurman G (January 2020).
5222:Computational Biology and Chemistry
5189:"Classification assessment methods"
5085:Chicco D, Jurman G (January 2020).
5067:. World Meteorological Organisation
4604:
3922:
3905:
3853:
3819:
3797:
2011:
878:
29:mean square contingency coefficient
13:
7106:CochranâMantelâHaenszel statistics
5732:Pearson product-moment correlation
5234:10.1016/j.compbiolchem.2004.09.006
4534:Mathematical Methods of Statistics
3870:
3809:
2818:
1578:
1543:
1530:
1497:
1484:
1457:
1444:
1431:
1418:
1362:
1346:
1330:
1314:
1294:
1278:
1262:
1246:
1162:
1149:
1133:
1120:
1104:
1091:
1075:
1062:
1047:
1034:
1021:
1008:
14:
8204:
5282:"Matthew Correlation Coefficient"
5193:Applied Computing and Informatics
4944:"An Introduction to ROC Analysis"
4033:The former article explains, for
3435:{\displaystyle c=\sum _{k}C_{kk}}
2022:Let us define an experiment from
869:
8173:Information retrieval evaluation
7727:
7715:
7703:
7690:
7689:
7480:
5028:Encyclopedia of machine learning
4396:TP = 90, FP = 4; TN = 1, FN = 5.
4218:TP = 95, FP = 5; TN = 0, FN = 0.
2703:Matthews correlation coefficient
59:Matthews correlation coefficient
8124:Pearson correlation coefficient
7365:Least-squares spectral analysis
5358:
5292:
5274:
5180:
5129:
5078:
5052:
5019:
4996:
4977:
4935:
4884:
4429:TP = 0, FP = 0; TN = 5, FN = 95
2802:
2793:
2784:
2775:
2766:
2757:
2044:
98:Pearson correlation coefficient
86:Pearson correlation coefficient
6346:Mean-unbiased minimum-variance
5449:
4644:
4617:
4591:
4363:
4345:
4342:
4324:
4321:
4303:
4300:
4282:
3790:
3778:
3640:
3625:
3596:
3581:
3552:
3537:
3197:
3068:
1971:
1959:
1953:
1941:
1935:
1923:
1917:
1905:
1641:
1629:
1626:
1614:
1170:
1144:
1141:
1115:
1112:
1086:
1083:
1057:
923:
913:
847:
825:
822:
800:
1:
8063:Deep Learning Related Metrics
7659:Geographic information system
6875:Simultaneous equations models
4812:10.1016/s0911-6044(03)00059-9
4478:
4433:This gives an F1 score = 0%.
704:{\displaystyle n_{\bullet 1}}
674:{\displaystyle n_{1\bullet }}
385:{\displaystyle n_{\bullet 0}}
354:{\displaystyle n_{\bullet 1}}
318:{\displaystyle n_{0\bullet }}
223:{\displaystyle n_{1\bullet }}
91:
6842:Coefficient of determination
6453:Uniformly most powerful test
5391:10.1016/j.patrec.2020.03.030
5003:Powers, David M. W. (2011).
4963:10.1016/j.patrec.2005.10.010
4687:10.1371/journal.pone.0177678
4509:10.1016/0005-2795(75)90109-9
7:
7907:Sensitivity and specificity
7411:Proportional hazards models
7355:Spectral density estimation
7337:Vector autoregression (VAR)
6771:Maximum posterior estimator
6003:Randomized controlled trial
5371:Pattern Recognition Letters
5365:Zhu, Qiuming (2020-08-01).
4951:Pattern Recognition Letters
4439:
2257:false alarm, overestimation
2085:bookmaker informedness (BM)
65:, introduced by biochemist
10:
8209:
8183:Statistical classification
7171:Multivariate distributions
5591:Average absolute deviation
5187:Tharwat A. (August 2018).
5155:10.1186/s13040-021-00244-z
4910:10.1186/s13040-023-00322-4
4840:Chicco D (December 2017).
4738:10.1186/s13040-021-00244-z
2015:
1730:
8132:
8106:
8083:
8062:
8039:
8011:
7983:
7920:
7852:
7784:
7685:
7639:
7576:
7529:
7492:
7488:
7475:
7447:
7429:
7396:
7387:
7345:
7292:
7253:
7202:
7193:
7159:Structural equation model
7114:
7071:
7067:
7042:
7001:
6967:
6921:
6888:
6850:
6817:
6813:
6789:
6729:
6638:
6557:
6521:
6512:
6495:Score/Lagrange multiplier
6480:
6433:
6378:
6304:
6295:
6105:
6101:
6088:
6047:
6021:
5973:
5928:
5910:Sample size determination
5875:
5871:
5858:
5762:
5717:
5691:
5673:
5629:
5581:
5501:
5492:
5488:
5475:
5457:
5333:10.1186/s12864-019-6413-7
5206:10.1016/j.aci.2018.08.003
5104:10.1186/s12864-019-6413-7
5036:10.1007/978-0-387-30164-8
4859:10.1186/s13040-017-0155-3
3682:
2869:{\displaystyle K\times K}
2726:
2548:Negative predictive value
2468:Negative likelihood ratio
2443:Positive likelihood ratio
2381:Positive predictive value
2352:
2125:
2056:
2051:
2047:
80:from its introduction by
7654:Environmental statistics
7176:Elliptical distributions
6969:Generalized linear model
6898:Simple linear regression
6668:HodgesâLehmann estimator
6125:Probability distribution
6034:Stochastic approximation
5596:Coefficient of variation
4660:Boughorbel, S.B (2017).
4638:10.1177/0013164491051004
88:in its interpretation.
76:, and also known as the
8168:Computational chemistry
7935:Calinski-Harabasz index
7314:Cross-correlation (XCF)
6922:Non-standard predictors
6356:LehmannâScheffĂ© theorem
6029:Adaptive clinical trial
2618:Balanced accuracy (BA)
2215:type II error
2076:Predicted negative (PN)
2071:Predicted positive (PP)
2026:positive instances and
1717:{\displaystyle \delta }
1676:of the problem and its
1674:regression coefficients
1666:correlation coefficient
7710:Mathematics portal
7531:Engineering statistics
7439:NelsonâAalen estimator
7016:Analysis of covariance
6903:Ordinary least squares
6827:Pearson product-moment
6231:Statistical functional
6142:Empirical distribution
5975:Controlled experiments
5704:Frequency distribution
5482:Descriptive statistics
4559:Yule, G. Udny (1912).
4472:Polychoric correlation
4419:
4373:
4194:
4119:
3981:
3655:
3494:
3436:
3388:
3333:
3269:
2890:
2870:
2844:
2289:type I error
2002:
1718:
1655:
1379:
1180:
953:
860:
705:
675:
645:
644:{\displaystyle n_{11}}
618:
593:Phi is related to the
584:
407:
386:
355:
319:
288:
287:{\displaystyle n_{00}}
260:
259:{\displaystyle n_{01}}
224:
193:
192:{\displaystyle n_{10}}
165:
164:{\displaystyle n_{11}}
44:measure of association
8098:Intra-list Similarity
7626:Population statistics
7568:System identification
7302:Autocorrelation (ACF)
7230:Exponential smoothing
7144:Discriminant analysis
7139:Canonical correlation
7003:Partition of variance
6865:Regression validation
6709:(JonckheereâTerpstra)
6608:Likelihood-ratio test
6297:Frequentist inference
6209:Locationâscale family
6130:Sampling distribution
6095:Statistical inference
6062:Cross-sectional study
6049:Observational studies
6008:Randomized experiment
5837:Stem-and-leaf display
5639:Central limit theorem
4942:Fawcett, Tom (2006).
4597:Guilford, J. (1936).
4467:FowlkesâMallows index
4415:computational biology
4374:
4195:
4120:
4039:
4014:computational biology
3982:
3656:
3495:
3437:
3389:
3334:
3270:
2891:
2871:
2845:
2843:{\displaystyle R_{K}}
2721:FNR Ă FPR Ă FOR Ă FDR
2712:TPR Ă TNR Ă PPV Ă NPV
2687:FowlkesâMallows index
2158:miss, underestimation
2003:
1719:
1656:
1380:
1181:
954:
861:
706:
676:
646:
619:
585:
408:
387:
356:
320:
289:
261:
225:
194:
166:
57:, it is known as the
7549:Probabilistic design
7134:Principal components
6977:Exponential families
6929:Nonlinear regression
6908:General linear model
6870:Mixed effects models
6860:Errors and residuals
6837:Confounding variable
6739:Bayesian probability
6717:Van der Waerden test
6707:Ordered alternative
6472:Multiple comparisons
6351:RaoâBlackwellization
6314:Estimating equations
6270:Statistical distance
5988:Factorial experiment
5521:Arithmetic-Geometric
4599:Psychometric Methods
4532:Cramer, H. (1946).
4233:
4135:
4048:
3764:
3508:
3448:
3400:
3345:
3290:
2903:
2880:
2854:
2827:
2520:False discovery rate
2094:Prevalence threshold
1868:
1708:
1702:Youden's J statistic
1686:Youden's J statistic
1399:
1231:
992:
909:
897:chi-square statistic
718:
685:
655:
628:
608:
465:
397:
366:
335:
299:
271:
243:
204:
176:
148:
78:Yule phi coefficient
7621:Official statistics
7544:Methods engineering
7225:Seasonal adjustment
6993:Poisson regressions
6913:Bayesian regression
6852:Regression analysis
6832:Partial correlation
6804:Regression analysis
6403:Prediction interval
6398:Likelihood interval
6388:Confidence interval
6380:Interval estimation
6341:Unbiased estimators
6159:Model specification
6039:Up-and-down designs
5727:Partial correlation
5683:Index of dispersion
5601:Interquartile range
5383:2020PaReL.136...71Z
4990:O'Reilly Media, Inc
4800:J. Neurolinguistics
4678:2017PLoSO..1277678B
2415:False omission rate
2274:False positive rate
2205:False negative rate
2053:Predicted condition
1744:(last prediction).
985:using the formula:
8188:Statistical ratios
8119:Euclidean distance
8085:Recommender system
7965:Similarity measure
7779:evaluation metrics
7641:Spatial statistics
7521:Medical statistics
7421:First hitting time
7375:Whittle likelihood
7026:Degrees of freedom
7021:Multivariate ANOVA
6954:Heteroscedasticity
6766:Bayesian estimator
6731:Bayesian inference
6580:KolmogorovâSmirnov
6465:Randomization test
6435:Testing hypotheses
6408:Tolerance interval
6319:Maximum likelihood
6214:Exponential family
6147:Density estimation
6107:Statistical theory
6067:Natural experiment
6013:Scientific control
5930:Survey methodology
5616:Standard deviation
4369:
4190:
4115:
3977:
3943:
3926:
3909:
3891:
3874:
3857:
3845:
3835:
3823:
3813:
3801:
3651:
3490:
3476:
3466:
3432:
3418:
3384:
3370:
3329:
3315:
3265:
3231:
3216:
3159:
3144:
3102:
3087:
3030:
3015:
2946:
2936:
2926:
2886:
2866:
2840:
2325:(SPC), selectivity
2317:True negative rate
2163:True positive rate
1998:
1714:
1651:
1649:
1375:
1189:In this equation,
1176:
949:
856:
701:
671:
641:
614:
580:
403:
382:
351:
315:
284:
256:
220:
189:
161:
8145:
8144:
8114:Cosine similarity
7950:Hopkins statistic
7743:
7742:
7681:
7680:
7677:
7676:
7616:National accounts
7586:Actuarial science
7578:Social statistics
7471:
7470:
7467:
7466:
7463:
7462:
7398:Survival function
7383:
7382:
7245:Granger causality
7086:Contingency table
7061:Survival analysis
7038:
7037:
7034:
7033:
6890:Linear regression
6785:
6784:
6781:
6780:
6756:Credible interval
6725:
6724:
6508:
6507:
6324:Method of moments
6193:Parametric family
6154:Statistical model
6084:
6083:
6080:
6079:
5998:Random assignment
5920:Statistical power
5854:
5853:
5850:
5849:
5699:Contingency table
5669:
5668:
5536:Generalized/power
5045:978-0-387-30164-8
4451:Contingency table
4367:
4366:
4239:
4188:
4141:
4113:
4054:
3969:
3968:
3954:
3951:
3899:
3770:
3755:
3754:
3649:
3646:
3643:
3628:
3602:
3599:
3584:
3555:
3540:
3514:
3467:
3457:
3409:
3361:
3306:
3263:
3260:
3217:
3183:
3150:
3135:
3131:
3088:
3054:
3021:
3006:
2937:
2927:
2917:
2909:
2889:{\displaystyle C}
2876:confusion matrix
2753:
2752:
2269:correct rejection
2033:contingency table
1990:
1989:
1975:
1974:
1874:
1856:
1855:
1801:
1800:
1645:
1644:
1564:
1555:
1509:
1373:
1305:
1237:
1193:is the number of
1174:
1173:
998:
947:
946:
920:
901:contingency table
851:
850:
617:{\displaystyle n}
575:
574:
416:
415:
406:{\displaystyle n}
67:Brian W. Matthews
8200:
8178:Machine learning
8137:Confusion matrix
7912:Logarithmic Loss
7777:Machine learning
7770:
7763:
7756:
7747:
7746:
7731:
7730:
7719:
7718:
7708:
7707:
7693:
7692:
7596:Crime statistics
7490:
7489:
7477:
7476:
7394:
7393:
7360:Fourier analysis
7347:Frequency domain
7327:
7274:
7240:Structural break
7200:
7199:
7149:Cluster analysis
7096:Log-linear model
7069:
7068:
7044:
7043:
6985:
6959:Homoscedasticity
6815:
6814:
6791:
6790:
6710:
6702:
6694:
6693:(KruskalâWallis)
6678:
6663:
6618:Cross validation
6603:
6585:AndersonâDarling
6532:
6519:
6518:
6490:Likelihood-ratio
6482:Parametric tests
6460:Permutation test
6443:1- & 2-tails
6334:Minimum distance
6306:Point estimation
6302:
6301:
6253:Optimal decision
6204:
6103:
6102:
6090:
6089:
6072:Quasi-experiment
6022:Adaptive designs
5873:
5872:
5860:
5859:
5737:Rank correlation
5499:
5498:
5490:
5489:
5477:
5476:
5444:
5437:
5430:
5421:
5420:
5411:
5410:
5362:
5356:
5355:
5345:
5335:
5311:
5302:
5296:
5290:
5289:
5286:scikit-learn.org
5278:
5272:
5271:
5269:
5267:
5252:
5246:
5245:
5217:
5211:
5210:
5208:
5184:
5178:
5177:
5167:
5157:
5133:
5127:
5126:
5116:
5106:
5082:
5076:
5075:
5073:
5072:
5056:
5050:
5049:
5023:
5017:
5016:
5000:
4994:
4993:
4981:
4975:
4974:
4948:
4939:
4933:
4932:
4922:
4912:
4888:
4882:
4881:
4871:
4861:
4837:
4824:
4823:
4795:
4789:
4788:
4786:
4774:
4761:
4760:
4750:
4740:
4716:
4710:
4709:
4699:
4689:
4657:
4651:
4648:
4642:
4641:
4621:
4615:
4608:
4602:
4595:
4589:
4588:
4556:
4547:
4530:
4521:
4520:
4492:
4417:
4378:
4376:
4375:
4370:
4368:
4281:
4280:
4245:
4240:
4237:
4199:
4197:
4196:
4191:
4189:
4187:
4158:
4147:
4142:
4139:
4124:
4122:
4121:
4116:
4114:
4112:
4077:
4060:
4055:
4052:
3986:
3984:
3983:
3978:
3970:
3964:
3960:
3955:
3953:
3952:
3950:
3949:
3944:
3933:
3932:
3927:
3916:
3915:
3910:
3902:
3900:
3898:
3897:
3892:
3881:
3880:
3875:
3864:
3863:
3858:
3850:
3847:
3846:
3836:
3824:
3814:
3802:
3776:
3771:
3768:
3751:
3744:
3737:
3725:
3705:
3664:
3663:
3660:
3658:
3657:
3652:
3650:
3648:
3647:
3645:
3644:
3636:
3630:
3629:
3621:
3615:
3614:
3605:
3603:
3601:
3600:
3592:
3586:
3585:
3577:
3571:
3570:
3561:
3558:
3557:
3556:
3548:
3542:
3541:
3533:
3520:
3515:
3512:
3499:
3497:
3496:
3491:
3489:
3488:
3475:
3465:
3441:
3439:
3438:
3433:
3431:
3430:
3417:
3393:
3391:
3390:
3385:
3383:
3382:
3369:
3357:
3356:
3338:
3336:
3335:
3330:
3328:
3327:
3314:
3302:
3301:
3274:
3272:
3271:
3266:
3264:
3262:
3261:
3259:
3255:
3254:
3253:
3252:
3244:
3230:
3229:
3215:
3208:
3200:
3195:
3177:
3173:
3172:
3171:
3158:
3143:
3134:
3132:
3130:
3126:
3125:
3124:
3123:
3115:
3101:
3100:
3086:
3079:
3071:
3066:
3048:
3044:
3043:
3042:
3029:
3014:
3005:
3002:
3001:
3000:
2988:
2987:
2972:
2971:
2959:
2958:
2945:
2935:
2925:
2915:
2910:
2907:
2895:
2893:
2892:
2887:
2875:
2873:
2872:
2867:
2849:
2847:
2846:
2841:
2839:
2838:
2809:
2806:
2800:
2797:
2791:
2788:
2782:
2779:
2773:
2770:
2764:
2761:
2749:
2748:
2746:
2745:
2742:
2739:
2724:
2723:
2722:
2715:
2714:
2713:
2699:
2698:
2697:
2683:
2682:
2680:
2679:
2676:
2673:
2665:
2664:
2662:
2661:
2658:
2655:
2637:
2636:
2634:
2633:
2630:
2627:
2613:
2612:
2610:
2609:
2606:
2603:
2592:
2585:
2581:
2580:deltaP (Δp)
2572:
2569:
2568:
2566:
2565:
2562:
2559:
2544:
2541:
2540:
2538:
2537:
2534:
2531:
2516:
2515:
2513:
2512:
2509:
2506:
2489:
2488:
2486:
2485:
2482:
2479:
2464:
2463:
2461:
2460:
2457:
2454:
2439:
2436:
2435:
2433:
2432:
2429:
2426:
2411:
2408:
2407:
2405:
2404:
2401:
2398:
2389:
2384:
2376:
2375:
2373:
2372:
2369:
2366:
2348:
2345:
2344:
2342:
2341:
2338:
2335:
2326:
2313:
2310:
2309:
2307:
2306:
2303:
2300:
2291:
2285:
2283:
2270:
2258:
2239:
2236:
2235:
2233:
2232:
2229:
2226:
2217:
2211:
2201:
2198:
2197:
2195:
2194:
2191:
2188:
2179:
2159:
2147:
2128:Actual condition
2121:
2120:
2118:
2117:
2114:
2111:
2109:
2108:
2090:
2086:
2067:
2062:Total population
2045:
2039:confusion matrix
2018:Confusion matrix
2012:Confusion matrix
2007:
2005:
2004:
1999:
1991:
1985:
1981:
1976:
1904:
1903:
1880:
1875:
1872:
1810:
1809:
1755:
1754:
1723:
1721:
1720:
1715:
1660:
1658:
1657:
1652:
1650:
1646:
1607:
1606:
1590:
1585:
1584:
1574:
1565:
1562:
1556:
1551:
1550:
1549:
1537:
1536:
1526:
1510:
1505:
1504:
1503:
1491:
1490:
1480:
1464:
1463:
1451:
1450:
1438:
1437:
1425:
1424:
1384:
1382:
1381:
1376:
1374:
1372:
1371:
1356:
1355:
1340:
1339:
1324:
1323:
1311:
1306:
1304:
1303:
1288:
1287:
1272:
1271:
1256:
1255:
1243:
1238:
1235:
1185:
1183:
1182:
1177:
1175:
1169:
1168:
1156:
1155:
1140:
1139:
1127:
1126:
1111:
1110:
1098:
1097:
1082:
1081:
1069:
1068:
1056:
1055:
1054:
1053:
1041:
1040:
1028:
1027:
1015:
1014:
1004:
999:
996:
983:confusion matrix
971:confusion matrix
958:
956:
955:
950:
948:
942:
941:
932:
931:
926:
921:
918:
916:
879:Machine learning
865:
863:
862:
857:
852:
846:
845:
821:
820:
799:
798:
786:
785:
773:
772:
771:
770:
758:
757:
742:
741:
728:
710:
708:
707:
702:
700:
699:
680:
678:
677:
672:
670:
669:
650:
648:
647:
642:
640:
639:
623:
621:
620:
615:
589:
587:
586:
581:
576:
573:
572:
560:
559:
547:
546:
534:
533:
521:
520:
519:
518:
509:
508:
496:
495:
486:
485:
475:
412:
410:
409:
404:
391:
389:
388:
383:
381:
380:
360:
358:
357:
352:
350:
349:
324:
322:
321:
316:
314:
313:
293:
291:
290:
285:
283:
282:
265:
263:
262:
257:
255:
254:
229:
227:
226:
221:
219:
218:
198:
196:
195:
190:
188:
187:
170:
168:
167:
162:
160:
159:
117:
116:
55:machine learning
48:binary variables
8208:
8207:
8203:
8202:
8201:
8199:
8198:
8197:
8163:Cheminformatics
8148:
8147:
8146:
8141:
8128:
8102:
8079:
8070:Inception score
8058:
8035:
8013:Computer Vision
8007:
7979:
7916:
7848:
7780:
7774:
7744:
7739:
7702:
7673:
7635:
7572:
7558:quality control
7525:
7507:Clinical trials
7484:
7459:
7443:
7431:Hazard function
7425:
7379:
7341:
7325:
7288:
7284:BreuschâGodfrey
7272:
7249:
7189:
7164:Factor analysis
7110:
7091:Graphical model
7063:
7030:
6997:
6983:
6963:
6917:
6884:
6846:
6809:
6808:
6777:
6721:
6708:
6700:
6692:
6676:
6661:
6640:Rank statistics
6634:
6613:Model selection
6601:
6559:Goodness of fit
6553:
6530:
6504:
6476:
6429:
6374:
6363:Median unbiased
6291:
6202:
6135:Order statistic
6097:
6076:
6043:
6017:
5969:
5924:
5867:
5865:Data collection
5846:
5758:
5713:
5687:
5665:
5625:
5577:
5494:Continuous data
5484:
5471:
5453:
5448:
5417:
5415:
5414:
5363:
5359:
5326:(1): 6-1â6-13.
5312:
5305:
5297:
5293:
5280:
5279:
5275:
5265:
5263:
5255:Gorodkin, Jan.
5253:
5249:
5218:
5214:
5185:
5181:
5134:
5130:
5097:(1): 6-1â6-13.
5083:
5079:
5070:
5068:
5057:
5053:
5046:
5024:
5020:
5001:
4997:
4982:
4978:
4946:
4940:
4936:
4889:
4885:
4838:
4827:
4806:(2â3): 97â119.
4796:
4792:
4775:
4764:
4717:
4713:
4672:(6): e0177678.
4658:
4654:
4649:
4645:
4622:
4618:
4609:
4605:
4596:
4592:
4577:10.2307/2340126
4557:
4550:
4531:
4524:
4493:
4486:
4481:
4442:
4418:
4412:
4246:
4244:
4236:
4234:
4231:
4230:
4159:
4148:
4146:
4138:
4136:
4133:
4132:
4078:
4061:
4059:
4051:
4049:
4046:
4045:
4008:
3959:
3945:
3938:
3937:
3928:
3921:
3920:
3911:
3904:
3903:
3901:
3893:
3886:
3885:
3876:
3869:
3868:
3859:
3852:
3851:
3849:
3848:
3840:
3830:
3818:
3808:
3796:
3777:
3775:
3767:
3765:
3762:
3761:
3747:
3740:
3733:
3721:
3701:
3674:
3671:
3669:
3635:
3634:
3620:
3619:
3610:
3606:
3604:
3591:
3590:
3576:
3575:
3566:
3562:
3560:
3559:
3547:
3546:
3532:
3531:
3521:
3519:
3511:
3509:
3506:
3505:
3481:
3477:
3471:
3461:
3449:
3446:
3445:
3423:
3419:
3413:
3401:
3398:
3397:
3375:
3371:
3365:
3352:
3348:
3346:
3343:
3342:
3320:
3316:
3310:
3297:
3293:
3291:
3288:
3287:
3282:
3245:
3237:
3236:
3232:
3222:
3221:
3201:
3196:
3188:
3187:
3182:
3178:
3164:
3160:
3154:
3149:
3145:
3139:
3133:
3116:
3108:
3107:
3103:
3093:
3092:
3072:
3067:
3059:
3058:
3053:
3049:
3035:
3031:
3025:
3020:
3016:
3010:
3004:
3003:
2993:
2989:
2980:
2976:
2964:
2960:
2951:
2947:
2941:
2931:
2921:
2916:
2914:
2906:
2904:
2901:
2900:
2881:
2878:
2877:
2855:
2852:
2851:
2834:
2830:
2828:
2825:
2824:
2821:
2819:Multiclass case
2815:
2813:
2812:
2807:
2803:
2798:
2794:
2789:
2785:
2780:
2776:
2771:
2767:
2762:
2758:
2743:
2740:
2737:
2736:
2734:
2732:
2731:
2720:
2718:
2716:
2711:
2709:
2707:
2706:
2695:
2693:
2691:
2690:
2677:
2674:
2671:
2670:
2668:
2666:
2659:
2656:
2653:
2652:
2650:
2648:
2647:
2644:
2631:
2628:
2625:
2624:
2622:
2620:
2619:
2607:
2604:
2601:
2600:
2598:
2596:
2595:
2590:
2584:= PPV + NPV â 1
2583:
2582:
2579:
2570:
2563:
2560:
2557:
2556:
2554:
2552:
2551:
2542:
2535:
2532:
2529:
2528:
2526:
2524:
2523:
2510:
2507:
2504:
2503:
2501:
2499:
2498:
2483:
2480:
2477:
2476:
2474:
2472:
2471:
2458:
2455:
2452:
2451:
2449:
2447:
2446:
2437:
2430:
2427:
2424:
2423:
2421:
2419:
2418:
2409:
2402:
2399:
2396:
2395:
2393:
2391:
2390:
2385:
2379:
2370:
2367:
2364:
2363:
2361:
2359:
2358:
2346:
2339:
2336:
2333:
2332:
2330:
2328:
2327:
2321:
2320:
2311:
2304:
2301:
2298:
2297:
2295:
2293:
2292:
2287:
2286:
2281:
2278:
2277:
2268:
2267:
2256:
2255:
2237:
2230:
2227:
2224:
2223:
2221:
2219:
2218:
2213:
2212:
2209:
2208:
2199:
2192:
2189:
2186:
2185:
2183:
2181:
2180:
2174:
2157:
2156:
2145:
2144:
2130:
2115:
2112:
2106:
2104:
2103:
2102:
2100:
2098:
2097:
2089:= TPR + TNR â 1
2088:
2087:
2084:
2065:
2064:
2020:
2014:
1980:
1881:
1879:
1871:
1869:
1866:
1865:
1820:
1817:
1815:
1765:
1762:
1760:
1733:
1709:
1706:
1705:
1648:
1647:
1586:
1577:
1576:
1575:
1573:
1566:
1561:
1558:
1557:
1542:
1541:
1529:
1528:
1527:
1525:
1518:
1512:
1511:
1496:
1495:
1483:
1482:
1481:
1479:
1472:
1466:
1465:
1456:
1455:
1443:
1442:
1430:
1429:
1417:
1416:
1409:
1402:
1400:
1397:
1396:
1361:
1360:
1345:
1344:
1329:
1328:
1313:
1312:
1310:
1293:
1292:
1277:
1276:
1261:
1260:
1245:
1244:
1242:
1234:
1232:
1229:
1228:
1219:false negatives
1211:false positives
1161:
1160:
1148:
1147:
1132:
1131:
1119:
1118:
1103:
1102:
1090:
1089:
1074:
1073:
1061:
1060:
1046:
1045:
1033:
1032:
1020:
1019:
1007:
1006:
1005:
1003:
995:
993:
990:
989:
937:
933:
930:
922:
917:
912:
910:
907:
906:
881:
872:
838:
834:
813:
809:
791:
787:
778:
774:
763:
759:
750:
746:
737:
733:
729:
727:
719:
716:
715:
692:
688:
686:
683:
682:
662:
658:
656:
653:
652:
635:
631:
629:
626:
625:
609:
606:
605:
565:
561:
552:
548:
539:
535:
526:
522:
514:
510:
504:
500:
491:
487:
481:
477:
476:
474:
466:
463:
462:
445:
438:
431:
424:
398:
395:
394:
373:
369:
367:
364:
363:
342:
338:
336:
333:
332:
306:
302:
300:
297:
296:
278:
274:
272:
269:
268:
250:
246:
244:
241:
240:
211:
207:
205:
202:
201:
183:
179:
177:
174:
173:
155:
151:
149:
146:
145:
94:
63:classifications
40:
31:and denoted by
25:phi coefficient
17:
12:
11:
5:
8206:
8196:
8195:
8190:
8185:
8180:
8175:
8170:
8165:
8160:
8158:Bioinformatics
8143:
8142:
8140:
8139:
8133:
8130:
8129:
8127:
8126:
8121:
8116:
8110:
8108:
8104:
8103:
8101:
8100:
8095:
8089:
8087:
8081:
8080:
8078:
8077:
8072:
8066:
8064:
8060:
8059:
8057:
8056:
8051:
8045:
8043:
8037:
8036:
8034:
8033:
8028:
8023:
8017:
8015:
8009:
8008:
8006:
8005:
8000:
7995:
7989:
7987:
7981:
7980:
7978:
7977:
7972:
7967:
7962:
7957:
7952:
7947:
7942:
7940:Davies-Bouldin
7937:
7932:
7926:
7924:
7918:
7917:
7915:
7914:
7909:
7904:
7899:
7894:
7889:
7884:
7879:
7874:
7869:
7864:
7858:
7856:
7854:Classification
7850:
7849:
7847:
7846:
7841:
7836:
7831:
7826:
7821:
7816:
7811:
7806:
7801:
7796:
7790:
7788:
7782:
7781:
7773:
7772:
7765:
7758:
7750:
7741:
7740:
7738:
7737:
7725:
7713:
7699:
7686:
7683:
7682:
7679:
7678:
7675:
7674:
7672:
7671:
7666:
7661:
7656:
7651:
7645:
7643:
7637:
7636:
7634:
7633:
7628:
7623:
7618:
7613:
7608:
7603:
7598:
7593:
7588:
7582:
7580:
7574:
7573:
7571:
7570:
7565:
7560:
7551:
7546:
7541:
7535:
7533:
7527:
7526:
7524:
7523:
7518:
7513:
7504:
7502:Bioinformatics
7498:
7496:
7486:
7485:
7473:
7472:
7469:
7468:
7465:
7464:
7461:
7460:
7458:
7457:
7451:
7449:
7445:
7444:
7442:
7441:
7435:
7433:
7427:
7426:
7424:
7423:
7418:
7413:
7408:
7402:
7400:
7391:
7385:
7384:
7381:
7380:
7378:
7377:
7372:
7367:
7362:
7357:
7351:
7349:
7343:
7342:
7340:
7339:
7334:
7329:
7321:
7316:
7311:
7310:
7309:
7307:partial (PACF)
7298:
7296:
7290:
7289:
7287:
7286:
7281:
7276:
7268:
7263:
7257:
7255:
7254:Specific tests
7251:
7250:
7248:
7247:
7242:
7237:
7232:
7227:
7222:
7217:
7212:
7206:
7204:
7197:
7191:
7190:
7188:
7187:
7186:
7185:
7184:
7183:
7168:
7167:
7166:
7156:
7154:Classification
7151:
7146:
7141:
7136:
7131:
7126:
7120:
7118:
7112:
7111:
7109:
7108:
7103:
7101:McNemar's test
7098:
7093:
7088:
7083:
7077:
7075:
7065:
7064:
7040:
7039:
7036:
7035:
7032:
7031:
7029:
7028:
7023:
7018:
7013:
7007:
7005:
6999:
6998:
6996:
6995:
6979:
6973:
6971:
6965:
6964:
6962:
6961:
6956:
6951:
6946:
6941:
6939:Semiparametric
6936:
6931:
6925:
6923:
6919:
6918:
6916:
6915:
6910:
6905:
6900:
6894:
6892:
6886:
6885:
6883:
6882:
6877:
6872:
6867:
6862:
6856:
6854:
6848:
6847:
6845:
6844:
6839:
6834:
6829:
6823:
6821:
6811:
6810:
6807:
6806:
6801:
6795:
6787:
6786:
6783:
6782:
6779:
6778:
6776:
6775:
6774:
6773:
6763:
6758:
6753:
6752:
6751:
6746:
6735:
6733:
6727:
6726:
6723:
6722:
6720:
6719:
6714:
6713:
6712:
6704:
6696:
6680:
6677:(MannâWhitney)
6672:
6671:
6670:
6657:
6656:
6655:
6644:
6642:
6636:
6635:
6633:
6632:
6631:
6630:
6625:
6620:
6610:
6605:
6602:(ShapiroâWilk)
6597:
6592:
6587:
6582:
6577:
6569:
6563:
6561:
6555:
6554:
6552:
6551:
6543:
6534:
6522:
6516:
6514:Specific tests
6510:
6509:
6506:
6505:
6503:
6502:
6497:
6492:
6486:
6484:
6478:
6477:
6475:
6474:
6469:
6468:
6467:
6457:
6456:
6455:
6445:
6439:
6437:
6431:
6430:
6428:
6427:
6426:
6425:
6420:
6410:
6405:
6400:
6395:
6390:
6384:
6382:
6376:
6375:
6373:
6372:
6367:
6366:
6365:
6360:
6359:
6358:
6353:
6338:
6337:
6336:
6331:
6326:
6321:
6310:
6308:
6299:
6293:
6292:
6290:
6289:
6284:
6279:
6278:
6277:
6267:
6262:
6261:
6260:
6250:
6249:
6248:
6243:
6238:
6228:
6223:
6218:
6217:
6216:
6211:
6206:
6190:
6189:
6188:
6183:
6178:
6168:
6167:
6166:
6161:
6151:
6150:
6149:
6139:
6138:
6137:
6127:
6122:
6117:
6111:
6109:
6099:
6098:
6086:
6085:
6082:
6081:
6078:
6077:
6075:
6074:
6069:
6064:
6059:
6053:
6051:
6045:
6044:
6042:
6041:
6036:
6031:
6025:
6023:
6019:
6018:
6016:
6015:
6010:
6005:
6000:
5995:
5990:
5985:
5979:
5977:
5971:
5970:
5968:
5967:
5965:Standard error
5962:
5957:
5952:
5951:
5950:
5945:
5934:
5932:
5926:
5925:
5923:
5922:
5917:
5912:
5907:
5902:
5897:
5895:Optimal design
5892:
5887:
5881:
5879:
5869:
5868:
5856:
5855:
5852:
5851:
5848:
5847:
5845:
5844:
5839:
5834:
5829:
5824:
5819:
5814:
5809:
5804:
5799:
5794:
5789:
5784:
5779:
5774:
5768:
5766:
5760:
5759:
5757:
5756:
5751:
5750:
5749:
5744:
5734:
5729:
5723:
5721:
5715:
5714:
5712:
5711:
5706:
5701:
5695:
5693:
5692:Summary tables
5689:
5688:
5686:
5685:
5679:
5677:
5671:
5670:
5667:
5666:
5664:
5663:
5662:
5661:
5656:
5651:
5641:
5635:
5633:
5627:
5626:
5624:
5623:
5618:
5613:
5608:
5603:
5598:
5593:
5587:
5585:
5579:
5578:
5576:
5575:
5570:
5565:
5564:
5563:
5558:
5553:
5548:
5543:
5538:
5533:
5528:
5526:Contraharmonic
5523:
5518:
5507:
5505:
5496:
5486:
5485:
5473:
5472:
5470:
5469:
5464:
5458:
5455:
5454:
5447:
5446:
5439:
5432:
5424:
5413:
5412:
5357:
5303:
5291:
5273:
5247:
5228:(5): 367â374.
5212:
5179:
5142:BioData Mining
5128:
5077:
5051:
5044:
5018:
4995:
4976:
4957:(8): 861â874.
4934:
4883:
4846:BioData Mining
4825:
4790:
4762:
4725:BioData Mining
4711:
4652:
4643:
4616:
4603:
4590:
4571:(6): 579â652.
4548:
4522:
4503:(2): 442â451.
4483:
4482:
4480:
4477:
4476:
4475:
4469:
4464:
4459:
4453:
4448:
4441:
4438:
4431:
4430:
4410:
4398:
4397:
4380:
4379:
4365:
4362:
4359:
4356:
4353:
4350:
4347:
4344:
4341:
4338:
4335:
4332:
4329:
4326:
4323:
4320:
4317:
4314:
4311:
4308:
4305:
4302:
4299:
4296:
4293:
4290:
4287:
4284:
4279:
4276:
4273:
4270:
4267:
4264:
4261:
4258:
4255:
4252:
4249:
4243:
4220:
4219:
4201:
4200:
4186:
4183:
4180:
4177:
4174:
4171:
4168:
4165:
4162:
4157:
4154:
4151:
4145:
4126:
4125:
4111:
4108:
4105:
4102:
4099:
4096:
4093:
4090:
4087:
4084:
4081:
4076:
4073:
4070:
4067:
4064:
4058:
4020:BioData Mining
4007:
4004:
3988:
3987:
3976:
3973:
3967:
3963:
3958:
3948:
3942:
3936:
3931:
3925:
3919:
3914:
3908:
3896:
3890:
3884:
3879:
3873:
3867:
3862:
3856:
3844:
3839:
3834:
3828:
3822:
3817:
3812:
3806:
3800:
3795:
3792:
3789:
3786:
3783:
3780:
3774:
3753:
3752:
3745:
3738:
3731:
3727:
3726:
3719:
3714:
3711:
3707:
3706:
3699:
3696:
3691:
3687:
3686:
3683:
3681:
3678:
3675:
3672:
3667:
3662:
3661:
3642:
3639:
3633:
3627:
3624:
3618:
3613:
3609:
3598:
3595:
3589:
3583:
3580:
3574:
3569:
3565:
3554:
3551:
3545:
3539:
3536:
3530:
3527:
3524:
3518:
3502:
3501:
3487:
3484:
3480:
3474:
3470:
3464:
3460:
3456:
3453:
3443:
3429:
3426:
3422:
3416:
3412:
3408:
3405:
3395:
3381:
3378:
3374:
3368:
3364:
3360:
3355:
3351:
3340:
3326:
3323:
3319:
3313:
3309:
3305:
3300:
3296:
3276:
3275:
3258:
3251:
3248:
3243:
3240:
3235:
3228:
3225:
3220:
3214:
3211:
3207:
3204:
3199:
3194:
3191:
3186:
3181:
3176:
3170:
3167:
3163:
3157:
3153:
3148:
3142:
3138:
3129:
3122:
3119:
3114:
3111:
3106:
3099:
3096:
3091:
3085:
3082:
3078:
3075:
3070:
3065:
3062:
3057:
3052:
3047:
3041:
3038:
3034:
3028:
3024:
3019:
3013:
3009:
2999:
2996:
2992:
2986:
2983:
2979:
2975:
2970:
2967:
2963:
2957:
2954:
2950:
2944:
2940:
2934:
2930:
2924:
2920:
2913:
2885:
2865:
2862:
2859:
2837:
2833:
2820:
2817:
2811:
2810:
2801:
2792:
2783:
2774:
2765:
2755:
2754:
2751:
2750:
2725:
2700:
2684:
2678:2 TP + FP + FN
2642:
2638:
2615:
2614:
2586:
2573:
2545:
2517:
2491:
2490:
2465:
2440:
2412:
2377:
2353:
2350:
2349:
2314:
2271:
2259:
2251:False positive
2247:
2241:
2240:
2210:miss rate
2202:
2160:
2152:False negative
2148:
2136:
2131:
2126:
2123:
2122:
2091:
2078:
2073:
2068:
2058:
2057:
2055:
2050:
2048:
2042:, as follows:
2016:Main article:
2013:
2010:
2009:
2008:
1997:
1994:
1988:
1984:
1979:
1973:
1970:
1967:
1964:
1961:
1958:
1955:
1952:
1949:
1946:
1943:
1940:
1937:
1934:
1931:
1928:
1925:
1922:
1919:
1916:
1913:
1910:
1907:
1902:
1899:
1896:
1893:
1890:
1887:
1884:
1878:
1854:
1853:
1848:
1845:
1841:
1840:
1837:
1832:
1828:
1827:
1824:
1821:
1818:
1813:
1799:
1798:
1793:
1790:
1786:
1785:
1782:
1777:
1773:
1772:
1769:
1766:
1763:
1758:
1750:
1749:
1741:
1740:
1732:
1729:
1713:
1670:geometric mean
1662:
1661:
1643:
1640:
1637:
1634:
1631:
1628:
1625:
1622:
1619:
1616:
1613:
1610:
1605:
1602:
1599:
1596:
1593:
1589:
1583:
1580:
1572:
1569:
1567:
1560:
1559:
1554:
1548:
1545:
1540:
1535:
1532:
1524:
1521:
1519:
1517:
1514:
1513:
1508:
1502:
1499:
1494:
1489:
1486:
1478:
1475:
1473:
1471:
1468:
1467:
1462:
1459:
1454:
1449:
1446:
1441:
1436:
1433:
1428:
1423:
1420:
1415:
1412:
1410:
1408:
1405:
1404:
1386:
1385:
1370:
1367:
1364:
1359:
1354:
1351:
1348:
1343:
1338:
1335:
1332:
1327:
1322:
1319:
1316:
1309:
1302:
1299:
1296:
1291:
1286:
1283:
1280:
1275:
1270:
1267:
1264:
1259:
1254:
1251:
1248:
1241:
1217:the number of
1209:the number of
1203:true negatives
1201:the number of
1195:true positives
1187:
1186:
1172:
1167:
1164:
1159:
1154:
1151:
1146:
1143:
1138:
1135:
1130:
1125:
1122:
1117:
1114:
1109:
1106:
1101:
1096:
1093:
1088:
1085:
1080:
1077:
1072:
1067:
1064:
1059:
1052:
1049:
1044:
1039:
1036:
1031:
1026:
1023:
1018:
1013:
1010:
1002:
960:
959:
945:
940:
936:
929:
925:
915:
880:
877:
871:
870:Maximum values
868:
867:
866:
855:
849:
844:
841:
837:
833:
830:
827:
824:
819:
816:
812:
808:
805:
802:
797:
794:
790:
784:
781:
777:
769:
766:
762:
756:
753:
749:
745:
740:
736:
732:
726:
723:
698:
695:
691:
668:
665:
661:
638:
634:
613:
591:
590:
579:
571:
568:
564:
558:
555:
551:
545:
542:
538:
532:
529:
525:
517:
513:
507:
503:
499:
494:
490:
484:
480:
473:
470:
443:
436:
429:
422:
414:
413:
402:
392:
379:
376:
372:
361:
348:
345:
341:
330:
326:
325:
312:
309:
305:
294:
281:
277:
266:
253:
249:
238:
231:
230:
217:
214:
210:
199:
186:
182:
171:
158:
154:
143:
136:
135:
132:
126:
120:
93:
90:
72:Introduced by
38:
15:
9:
6:
4:
3:
2:
8205:
8194:
8191:
8189:
8186:
8184:
8181:
8179:
8176:
8174:
8171:
8169:
8166:
8164:
8161:
8159:
8156:
8155:
8153:
8138:
8135:
8134:
8131:
8125:
8122:
8120:
8117:
8115:
8112:
8111:
8109:
8105:
8099:
8096:
8094:
8091:
8090:
8088:
8086:
8082:
8076:
8073:
8071:
8068:
8067:
8065:
8061:
8055:
8052:
8050:
8047:
8046:
8044:
8042:
8038:
8032:
8029:
8027:
8024:
8022:
8019:
8018:
8016:
8014:
8010:
8004:
8001:
7999:
7996:
7994:
7991:
7990:
7988:
7986:
7982:
7976:
7973:
7971:
7968:
7966:
7963:
7961:
7958:
7956:
7955:Jaccard index
7953:
7951:
7948:
7946:
7943:
7941:
7938:
7936:
7933:
7931:
7928:
7927:
7925:
7923:
7919:
7913:
7910:
7908:
7905:
7903:
7900:
7898:
7895:
7893:
7890:
7888:
7885:
7883:
7880:
7878:
7875:
7873:
7870:
7868:
7865:
7863:
7860:
7859:
7857:
7855:
7851:
7845:
7842:
7840:
7837:
7835:
7832:
7830:
7827:
7825:
7822:
7820:
7817:
7815:
7812:
7810:
7807:
7805:
7802:
7800:
7797:
7795:
7792:
7791:
7789:
7787:
7783:
7778:
7771:
7766:
7764:
7759:
7757:
7752:
7751:
7748:
7736:
7735:
7726:
7724:
7723:
7714:
7712:
7711:
7706:
7700:
7698:
7697:
7688:
7687:
7684:
7670:
7667:
7665:
7664:Geostatistics
7662:
7660:
7657:
7655:
7652:
7650:
7647:
7646:
7644:
7642:
7638:
7632:
7631:Psychometrics
7629:
7627:
7624:
7622:
7619:
7617:
7614:
7612:
7609:
7607:
7604:
7602:
7599:
7597:
7594:
7592:
7589:
7587:
7584:
7583:
7581:
7579:
7575:
7569:
7566:
7564:
7561:
7559:
7555:
7552:
7550:
7547:
7545:
7542:
7540:
7537:
7536:
7534:
7532:
7528:
7522:
7519:
7517:
7514:
7512:
7508:
7505:
7503:
7500:
7499:
7497:
7495:
7494:Biostatistics
7491:
7487:
7483:
7478:
7474:
7456:
7455:Log-rank test
7453:
7452:
7450:
7446:
7440:
7437:
7436:
7434:
7432:
7428:
7422:
7419:
7417:
7414:
7412:
7409:
7407:
7404:
7403:
7401:
7399:
7395:
7392:
7390:
7386:
7376:
7373:
7371:
7368:
7366:
7363:
7361:
7358:
7356:
7353:
7352:
7350:
7348:
7344:
7338:
7335:
7333:
7330:
7328:
7326:(BoxâJenkins)
7322:
7320:
7317:
7315:
7312:
7308:
7305:
7304:
7303:
7300:
7299:
7297:
7295:
7291:
7285:
7282:
7280:
7279:DurbinâWatson
7277:
7275:
7269:
7267:
7264:
7262:
7261:DickeyâFuller
7259:
7258:
7256:
7252:
7246:
7243:
7241:
7238:
7236:
7235:Cointegration
7233:
7231:
7228:
7226:
7223:
7221:
7218:
7216:
7213:
7211:
7210:Decomposition
7208:
7207:
7205:
7201:
7198:
7196:
7192:
7182:
7179:
7178:
7177:
7174:
7173:
7172:
7169:
7165:
7162:
7161:
7160:
7157:
7155:
7152:
7150:
7147:
7145:
7142:
7140:
7137:
7135:
7132:
7130:
7127:
7125:
7122:
7121:
7119:
7117:
7113:
7107:
7104:
7102:
7099:
7097:
7094:
7092:
7089:
7087:
7084:
7082:
7081:Cohen's kappa
7079:
7078:
7076:
7074:
7070:
7066:
7062:
7058:
7054:
7050:
7045:
7041:
7027:
7024:
7022:
7019:
7017:
7014:
7012:
7009:
7008:
7006:
7004:
7000:
6994:
6990:
6986:
6980:
6978:
6975:
6974:
6972:
6970:
6966:
6960:
6957:
6955:
6952:
6950:
6947:
6945:
6942:
6940:
6937:
6935:
6934:Nonparametric
6932:
6930:
6927:
6926:
6924:
6920:
6914:
6911:
6909:
6906:
6904:
6901:
6899:
6896:
6895:
6893:
6891:
6887:
6881:
6878:
6876:
6873:
6871:
6868:
6866:
6863:
6861:
6858:
6857:
6855:
6853:
6849:
6843:
6840:
6838:
6835:
6833:
6830:
6828:
6825:
6824:
6822:
6820:
6816:
6812:
6805:
6802:
6800:
6797:
6796:
6792:
6788:
6772:
6769:
6768:
6767:
6764:
6762:
6759:
6757:
6754:
6750:
6747:
6745:
6742:
6741:
6740:
6737:
6736:
6734:
6732:
6728:
6718:
6715:
6711:
6705:
6703:
6697:
6695:
6689:
6688:
6687:
6684:
6683:Nonparametric
6681:
6679:
6673:
6669:
6666:
6665:
6664:
6658:
6654:
6653:Sample median
6651:
6650:
6649:
6646:
6645:
6643:
6641:
6637:
6629:
6626:
6624:
6621:
6619:
6616:
6615:
6614:
6611:
6609:
6606:
6604:
6598:
6596:
6593:
6591:
6588:
6586:
6583:
6581:
6578:
6576:
6574:
6570:
6568:
6565:
6564:
6562:
6560:
6556:
6550:
6548:
6544:
6542:
6540:
6535:
6533:
6528:
6524:
6523:
6520:
6517:
6515:
6511:
6501:
6498:
6496:
6493:
6491:
6488:
6487:
6485:
6483:
6479:
6473:
6470:
6466:
6463:
6462:
6461:
6458:
6454:
6451:
6450:
6449:
6446:
6444:
6441:
6440:
6438:
6436:
6432:
6424:
6421:
6419:
6416:
6415:
6414:
6411:
6409:
6406:
6404:
6401:
6399:
6396:
6394:
6391:
6389:
6386:
6385:
6383:
6381:
6377:
6371:
6368:
6364:
6361:
6357:
6354:
6352:
6349:
6348:
6347:
6344:
6343:
6342:
6339:
6335:
6332:
6330:
6327:
6325:
6322:
6320:
6317:
6316:
6315:
6312:
6311:
6309:
6307:
6303:
6300:
6298:
6294:
6288:
6285:
6283:
6280:
6276:
6273:
6272:
6271:
6268:
6266:
6263:
6259:
6258:loss function
6256:
6255:
6254:
6251:
6247:
6244:
6242:
6239:
6237:
6234:
6233:
6232:
6229:
6227:
6224:
6222:
6219:
6215:
6212:
6210:
6207:
6205:
6199:
6196:
6195:
6194:
6191:
6187:
6184:
6182:
6179:
6177:
6174:
6173:
6172:
6169:
6165:
6162:
6160:
6157:
6156:
6155:
6152:
6148:
6145:
6144:
6143:
6140:
6136:
6133:
6132:
6131:
6128:
6126:
6123:
6121:
6118:
6116:
6113:
6112:
6110:
6108:
6104:
6100:
6096:
6091:
6087:
6073:
6070:
6068:
6065:
6063:
6060:
6058:
6055:
6054:
6052:
6050:
6046:
6040:
6037:
6035:
6032:
6030:
6027:
6026:
6024:
6020:
6014:
6011:
6009:
6006:
6004:
6001:
5999:
5996:
5994:
5991:
5989:
5986:
5984:
5981:
5980:
5978:
5976:
5972:
5966:
5963:
5961:
5960:Questionnaire
5958:
5956:
5953:
5949:
5946:
5944:
5941:
5940:
5939:
5936:
5935:
5933:
5931:
5927:
5921:
5918:
5916:
5913:
5911:
5908:
5906:
5903:
5901:
5898:
5896:
5893:
5891:
5888:
5886:
5883:
5882:
5880:
5878:
5874:
5870:
5866:
5861:
5857:
5843:
5840:
5838:
5835:
5833:
5830:
5828:
5825:
5823:
5820:
5818:
5815:
5813:
5810:
5808:
5805:
5803:
5800:
5798:
5795:
5793:
5790:
5788:
5787:Control chart
5785:
5783:
5780:
5778:
5775:
5773:
5770:
5769:
5767:
5765:
5761:
5755:
5752:
5748:
5745:
5743:
5740:
5739:
5738:
5735:
5733:
5730:
5728:
5725:
5724:
5722:
5720:
5716:
5710:
5707:
5705:
5702:
5700:
5697:
5696:
5694:
5690:
5684:
5681:
5680:
5678:
5676:
5672:
5660:
5657:
5655:
5652:
5650:
5647:
5646:
5645:
5642:
5640:
5637:
5636:
5634:
5632:
5628:
5622:
5619:
5617:
5614:
5612:
5609:
5607:
5604:
5602:
5599:
5597:
5594:
5592:
5589:
5588:
5586:
5584:
5580:
5574:
5571:
5569:
5566:
5562:
5559:
5557:
5554:
5552:
5549:
5547:
5544:
5542:
5539:
5537:
5534:
5532:
5529:
5527:
5524:
5522:
5519:
5517:
5514:
5513:
5512:
5509:
5508:
5506:
5504:
5500:
5497:
5495:
5491:
5487:
5483:
5478:
5474:
5468:
5465:
5463:
5460:
5459:
5456:
5452:
5445:
5440:
5438:
5433:
5431:
5426:
5425:
5422:
5418:
5408:
5404:
5400:
5396:
5392:
5388:
5384:
5380:
5376:
5372:
5368:
5361:
5353:
5349:
5344:
5339:
5334:
5329:
5325:
5321:
5317:
5310:
5308:
5301:
5295:
5287:
5283:
5277:
5262:
5258:
5257:"The Rk Page"
5251:
5243:
5239:
5235:
5231:
5227:
5223:
5216:
5207:
5202:
5198:
5194:
5190:
5183:
5175:
5171:
5166:
5161:
5156:
5151:
5147:
5143:
5139:
5132:
5124:
5120:
5115:
5110:
5105:
5100:
5096:
5092:
5088:
5081:
5066:
5062:
5055:
5047:
5041:
5037:
5033:
5029:
5022:
5014:
5010:
5006:
4999:
4991:
4987:
4980:
4972:
4968:
4964:
4960:
4956:
4952:
4945:
4938:
4930:
4926:
4921:
4916:
4911:
4906:
4902:
4898:
4894:
4887:
4879:
4875:
4870:
4865:
4860:
4855:
4851:
4847:
4843:
4836:
4834:
4832:
4830:
4821:
4817:
4813:
4809:
4805:
4801:
4794:
4785:
4780:
4773:
4771:
4769:
4767:
4758:
4754:
4749:
4744:
4739:
4734:
4730:
4726:
4722:
4715:
4707:
4703:
4698:
4693:
4688:
4683:
4679:
4675:
4671:
4667:
4663:
4656:
4647:
4639:
4635:
4631:
4627:
4620:
4613:
4607:
4600:
4594:
4586:
4582:
4578:
4574:
4570:
4566:
4562:
4555:
4553:
4546:
4543:
4542:0-691-08004-6
4539:
4535:
4529:
4527:
4518:
4514:
4510:
4506:
4502:
4498:
4491:
4489:
4484:
4473:
4470:
4468:
4465:
4463:
4460:
4457:
4454:
4452:
4449:
4447:
4446:Cohen's kappa
4444:
4443:
4437:
4434:
4428:
4427:
4426:
4422:
4416:
4409:
4406:
4402:
4395:
4394:
4393:
4390:
4386:
4383:
4360:
4357:
4354:
4351:
4348:
4339:
4336:
4333:
4330:
4327:
4318:
4315:
4312:
4309:
4306:
4297:
4294:
4291:
4288:
4285:
4277:
4274:
4271:
4268:
4265:
4262:
4259:
4256:
4253:
4250:
4247:
4241:
4229:
4228:
4227:
4224:
4217:
4216:
4215:
4212:
4208:
4204:
4184:
4181:
4178:
4175:
4172:
4169:
4166:
4163:
4160:
4155:
4152:
4149:
4143:
4131:
4130:
4129:
4109:
4106:
4103:
4100:
4097:
4094:
4091:
4088:
4085:
4082:
4079:
4074:
4071:
4068:
4065:
4062:
4056:
4044:
4043:
4042:
4038:
4036:
4031:
4029:
4025:
4021:
4017:
4015:
4003:
3999:
3997:
3993:
3974:
3971:
3965:
3961:
3956:
3946:
3940:
3934:
3929:
3923:
3917:
3912:
3906:
3894:
3888:
3882:
3877:
3871:
3865:
3860:
3854:
3842:
3837:
3832:
3826:
3820:
3815:
3810:
3804:
3798:
3793:
3787:
3784:
3781:
3772:
3760:
3759:
3758:
3750:
3746:
3743:
3739:
3736:
3732:
3729:
3728:
3724:
3720:
3718:
3715:
3712:
3709:
3708:
3704:
3700:
3697:
3695:
3692:
3689:
3688:
3684:
3679:
3676:
3666:
3665:
3637:
3631:
3622:
3616:
3611:
3607:
3593:
3587:
3578:
3572:
3567:
3563:
3549:
3543:
3534:
3528:
3525:
3522:
3516:
3504:
3503:
3485:
3482:
3478:
3472:
3468:
3462:
3458:
3454:
3451:
3444:
3427:
3424:
3420:
3414:
3410:
3406:
3403:
3396:
3379:
3376:
3372:
3366:
3362:
3358:
3353:
3349:
3341:
3324:
3321:
3317:
3311:
3307:
3303:
3298:
3294:
3286:
3285:
3284:
3280:
3256:
3249:
3246:
3241:
3238:
3233:
3226:
3223:
3218:
3212:
3209:
3205:
3202:
3192:
3189:
3184:
3179:
3174:
3168:
3165:
3161:
3155:
3151:
3146:
3140:
3136:
3127:
3120:
3117:
3112:
3109:
3104:
3097:
3094:
3089:
3083:
3080:
3076:
3073:
3063:
3060:
3055:
3050:
3045:
3039:
3036:
3032:
3026:
3022:
3017:
3011:
3007:
2997:
2994:
2990:
2984:
2981:
2977:
2973:
2968:
2965:
2961:
2955:
2952:
2948:
2942:
2938:
2932:
2928:
2922:
2918:
2911:
2899:
2898:
2897:
2883:
2863:
2860:
2857:
2835:
2831:
2816:
2805:
2796:
2787:
2778:
2769:
2760:
2756:
2730:
2729:Jaccard index
2704:
2701:
2688:
2685:
2646:
2639:
2617:
2616:
2593:
2587:
2577:
2574:
2549:
2546:
2521:
2518:
2496:
2493:
2492:
2469:
2466:
2444:
2441:
2416:
2413:
2388:
2382:
2378:
2357:
2354:
2351:
2324:
2318:
2315:
2290:
2284:
2275:
2272:
2265:
2264:
2263:True negative
2260:
2253:
2252:
2248:
2246:
2243:
2242:
2216:
2206:
2203:
2178:
2172:
2168:
2164:
2161:
2154:
2153:
2149:
2142:
2141:
2140:True positive
2137:
2135:
2132:
2129:
2124:
2095:
2092:
2082:
2079:
2077:
2074:
2072:
2069:
2063:
2060:
2059:
2054:
2049:
2046:
2043:
2041:
2040:
2035:
2034:
2029:
2025:
2019:
1995:
1992:
1986:
1982:
1977:
1968:
1965:
1962:
1956:
1950:
1947:
1944:
1938:
1932:
1929:
1926:
1920:
1914:
1911:
1908:
1900:
1897:
1894:
1891:
1888:
1885:
1882:
1876:
1864:
1863:
1862:
1859:
1852:
1849:
1846:
1843:
1842:
1838:
1836:
1833:
1830:
1829:
1825:
1822:
1812:
1811:
1808:
1805:
1797:
1794:
1791:
1788:
1787:
1783:
1781:
1778:
1775:
1774:
1770:
1767:
1757:
1756:
1753:
1748:prediction =
1747:
1746:
1745:
1738:
1737:
1736:
1728:
1725:
1711:
1703:
1699:
1695:
1691:
1687:
1683:
1679:
1675:
1671:
1667:
1638:
1635:
1632:
1623:
1620:
1617:
1611:
1608:
1603:
1600:
1597:
1594:
1591:
1587:
1570:
1568:
1552:
1538:
1522:
1520:
1515:
1506:
1492:
1476:
1474:
1469:
1452:
1439:
1426:
1413:
1411:
1406:
1395:
1394:
1393:
1390:
1357:
1341:
1325:
1307:
1289:
1273:
1257:
1239:
1227:
1226:
1225:
1222:
1220:
1216:
1212:
1208:
1204:
1200:
1196:
1192:
1157:
1128:
1099:
1070:
1042:
1029:
1016:
1000:
988:
987:
986:
984:
979:
977:
972:
967:
965:
943:
938:
934:
927:
905:
904:
903:
902:
898:
892:
890:
886:
876:
853:
842:
839:
835:
831:
828:
817:
814:
810:
806:
803:
795:
792:
788:
782:
779:
775:
767:
764:
760:
754:
751:
747:
743:
738:
734:
730:
724:
721:
714:
713:
712:
696:
693:
689:
666:
663:
659:
636:
632:
611:
602:
600:
596:
577:
569:
566:
562:
556:
553:
549:
543:
540:
536:
530:
527:
523:
515:
511:
505:
501:
497:
492:
488:
482:
478:
471:
468:
461:
460:
459:
457:
453:
449:
442:
435:
428:
421:
400:
393:
377:
374:
370:
362:
346:
343:
339:
331:
328:
327:
310:
307:
303:
295:
279:
275:
267:
251:
247:
239:
236:
233:
232:
215:
212:
208:
200:
184:
180:
172:
156:
152:
144:
141:
138:
137:
133:
130:
127:
124:
121:
119:
118:
115:
114:
110:
105:
101:
99:
89:
87:
83:
79:
75:
70:
68:
64:
60:
56:
51:
49:
45:
41:
34:
30:
26:
22:
7891:
7732:
7720:
7701:
7694:
7606:Econometrics
7556: /
7539:Chemometrics
7516:Epidemiology
7509: /
7482:Applications
7324:ARIMA model
7271:Q-statistic
7220:Stationarity
7116:Multivariate
7059: /
7055: /
7053:Multivariate
7051: /
6991: /
6987: /
6761:Bayes factor
6660:Signed rank
6572:
6546:
6538:
6526:
6221:Completeness
6057:Cohort study
5955:Opinion poll
5890:Missing data
5877:Study design
5832:Scatter plot
5754:Scatter plot
5747:Spearman's Ï
5709:Grouped data
5416:
5374:
5370:
5360:
5323:
5320:BMC Genomics
5319:
5294:
5285:
5276:
5264:. Retrieved
5260:
5250:
5225:
5221:
5215:
5196:
5192:
5182:
5145:
5141:
5131:
5094:
5091:BMC Genomics
5090:
5080:
5069:. Retrieved
5064:
5054:
5030:. Springer.
5027:
5021:
5012:
5008:
4998:
4989:
4979:
4954:
4950:
4937:
4900:
4896:
4886:
4849:
4845:
4803:
4799:
4793:
4728:
4724:
4714:
4669:
4665:
4655:
4646:
4632:(4): 821â8.
4629:
4625:
4619:
4606:
4598:
4593:
4568:
4564:
4533:
4500:
4496:
4435:
4432:
4423:
4420:
4407:
4403:
4399:
4391:
4387:
4384:
4381:
4225:
4221:
4213:
4209:
4205:
4202:
4127:
4040:
4034:
4032:
4028:BMC Genomics
4023:
4022:, 2017) and
4011:
4009:
4000:
3992:Informedness
3989:
3756:
3748:
3741:
3734:
3722:
3716:
3702:
3693:
3673:Actual class
3281:
3277:
2822:
2814:
2804:
2795:
2786:
2777:
2768:
2759:
2744:TP + FN + FP
2261:
2249:
2245:Negative (N)
2244:
2150:
2138:
2134:Positive (P)
2133:
2127:
2081:Informedness
2075:
2070:
2052:
2037:
2031:
2027:
2023:
2021:
1860:
1857:
1850:
1834:
1819:Actual class
1806:
1802:
1795:
1779:
1764:Actual class
1751:
1742:
1734:
1726:
1698:Informedness
1690:Informedness
1663:
1391:
1387:
1223:
1214:
1206:
1198:
1190:
1188:
980:
968:
963:
961:
893:
885:Karl Pearson
882:
873:
603:
598:
597:and Cohen's
592:
455:
451:
447:
440:
433:
426:
419:
417:
234:
139:
128:
122:
112:
108:
106:
102:
95:
77:
74:Karl Pearson
71:
58:
52:
36:
32:
28:
24:
18:
7734:WikiProject
7649:Cartography
7611:Jurimetrics
7563:Reliability
7294:Time domain
7273:(LjungâBox)
7195:Time-series
7073:Categorical
7057:Time-series
7049:Categorical
6984:(Bernoulli)
6819:Correlation
6799:Correlation
6595:JarqueâBera
6567:Chi-squared
6329:M-estimator
6282:Asymptotics
6226:Sufficiency
5993:Interaction
5905:Replication
5885:Effect size
5842:Violin plot
5822:Radar chart
5802:Forest plot
5792:Correlogram
5742:Kendall's Ï
5266:28 December
5261:The Rk Page
5199:: 168â192.
5015:(1): 37â63.
4897:BioData Min
2654:2 PPV Ă TPR
2589:Diagnostic
2323:specificity
2171:sensitivity
8152:Categories
8107:Similarity
8049:Perplexity
7960:Rand index
7945:Dunn index
7930:Silhouette
7922:Clustering
7786:Regression
7601:Demography
7319:ARMA model
7124:Regression
6701:(Friedman)
6662:(Wilcoxon)
6600:Normality
6590:Lilliefors
6537:Student's
6413:Resampling
6287:Robustness
6275:divergence
6265:Efficiency
6203:(monotone)
6198:Likelihood
6115:Population
5948:Stratified
5900:Population
5719:Dependence
5675:Count data
5606:Percentile
5583:Dispersion
5516:Arithmetic
5451:Statistics
5148:(13): 13.
5071:2019-07-17
4852:(35): 35.
4784:2010.16061
4479:References
4456:Cramér's V
3996:Markedness
2591:odds ratio
2576:Markedness
2356:Prevalence
1739:actual = ,
1694:Markedness
1682:Markedness
899:for a 2Ă2
92:Definition
21:statistics
7877:Precision
7829:RMSE/RMSD
6982:Logistic
6749:posterior
6675:Rank sum
6423:Jackknife
6418:Bootstrap
6236:Bootstrap
6171:Parameter
6120:Statistic
5915:Statistic
5827:Run chart
5812:Pie chart
5807:Histogram
5797:Fan chart
5772:Bar chart
5654:L-moments
5541:Geometric
5407:219762950
5399:0167-8655
5377:: 71â80.
4731:(1): 13.
4272:×
4263:−
4254:×
3972:≈
3935:−
3918:−
3883:−
3866:−
3838:×
3827:−
3816:×
3805:−
3794:×
3668:Predicted
3641:→
3632:⋅
3626:→
3617:−
3597:→
3588:⋅
3582:→
3573:−
3553:→
3544:⋅
3538:→
3529:−
3469:∑
3459:∑
3411:∑
3363:∑
3308:∑
3219:∑
3210:≠
3185:∑
3152:∑
3137:∑
3090:∑
3081:≠
3056:∑
3023:∑
3008:∑
2974:−
2939:∑
2929:∑
2919:∑
2861:×
2696:PPV Ă TPR
2660:PPV + TPR
2626:TPR + TNR
2571:= 1 â FOR
2543:= 1 â PPV
2438:= 1 â NPV
2410:= 1 â FDR
2387:precision
2347:= 1 â FPR
2312:= 1 â TNR
2238:= 1 â TPR
2200:= 1 â FNR
2116:TPR - FPR
2107:TPR Ă FPR
1993:≈
1957:×
1939:×
1921:×
1898:×
1892:−
1886:×
1814:Predicted
1759:Predicted
1712:δ
1692:or Îp').
1684:(Îp) and
1636:−
1621:−
1601:×
1595:−
1358:×
1342:×
1326:×
1308:−
1290:×
1274:×
1258:×
1043:×
1030:−
1017:×
935:χ
889:Udny Yule
840:∙
832:−
818:∙
807:−
793:∙
783:∙
765:∙
755:∙
744:−
722:ϕ
694:∙
667:∙
567:∙
554:∙
544:∙
531:∙
498:−
469:ϕ
375:∙
344:∙
311:∙
216:∙
111:and
82:Udny Yule
69:in 1975.
8093:Coverage
7872:Accuracy
7696:Category
7389:Survival
7266:Johansen
6989:Binomial
6944:Isotonic
6531:(normal)
6176:location
5983:Blocking
5938:Sampling
5817:QâQ plot
5782:Box plot
5764:Graphics
5659:Skewness
5649:Kurtosis
5621:Variance
5551:Heronian
5546:Harmonic
5352:31898477
5242:15556477
5174:33541410
5123:31898477
4929:36800973
4903:(1): 4.
4878:29234465
4820:17104364
4757:33541410
4706:28574989
4666:PLOS ONE
4462:F1 score
4440:See also
4411:â
4140:F1 score
4053:accuracy
3250:′
3242:′
3227:′
3206:′
3193:′
3121:′
3113:′
3098:′
3077:′
3064:′
2495:Accuracy
2282:fall-out
976:accuracy
46:for two
7985:Ranking
7975:SimHash
7862:F-score
7722:Commons
7669:Kriging
7554:Process
7511:studies
7370:Wavelet
7203:General
6370:Plug-in
6164:L space
5943:Cluster
5644:Moments
5462:Outline
5379:Bibcode
5343:6941312
5165:7863449
5114:6941312
4971:2027090
4920:9938573
4869:5721660
4748:7863449
4697:5456046
4674:Bibcode
4585:2340126
4517:1180967
2747:
2735:
2719:√
2710:√
2694:√
2681:
2669:
2663:
2651:
2635:
2623:
2611:
2599:
2567:
2555:
2539:
2527:
2514:
2505:TP + TN
2502:
2487:
2475:
2462:
2450:
2434:
2422:
2406:
2394:
2374:
2362:
2343:
2331:
2319:(TNR),
2308:
2296:
2276:(FPR),
2234:
2222:
2207:(FNR),
2196:
2184:
2173:(SEN),
2165:(TPR),
2119:
2105:√
2101:
2066:= P + N
1731:Example
1672:of the
42:) is a
7882:Recall
7591:Census
7181:Normal
7129:Manova
6949:Robust
6699:2-way
6691:1-way
6529:-test
6200:
5777:Biplot
5568:Median
5561:Lehmer
5503:Center
5405:
5397:
5350:
5340:
5240:
5172:
5162:
5121:
5111:
5042:
4969:
4927:
4917:
4876:
4866:
4818:
4755:
4745:
4704:
4694:
4583:
4540:
4515:
2705:(MCC)
2594:(DOR)
2578:(MK),
2550:(NPV)
2522:(FDR)
2497:(ACC)
2470:(LRâ)
2445:(LR+)
2417:(FOR)
2383:(PPV),
2266:(TN),
2254:(FP),
2167:recall
2155:(FN),
2143:(TP),
1704:, the
962:where
681:, and
418:where
134:total
23:, the
7887:Kappa
7804:sMAPE
7215:Trend
6744:prior
6686:anova
6575:-test
6549:-test
6541:-test
6448:Power
6393:Pivot
6186:shape
6181:scale
5631:Shape
5611:Range
5556:Heinz
5531:Cubic
5467:Index
5403:S2CID
4967:S2CID
4947:(PDF)
4816:S2CID
4779:arXiv
4581:JSTOR
4035:Tip 8
3975:0.478
3670:class
2689:(FM)
2645:score
2511:P + N
2371:P + N
2177:power
2110:- FPR
2096:(PT)
1996:0.478
1816:class
1761:class
711:, as
329:total
8054:BLEU
8026:SSIM
8021:PSNR
7998:NDCG
7819:MSPE
7814:MASE
7809:MAPE
7448:Test
6648:Sign
6500:Wald
5573:Mode
5511:Mean
5395:ISSN
5348:PMID
5268:2016
5238:PMID
5170:PMID
5119:PMID
5040:ISBN
4925:PMID
4874:PMID
4753:PMID
4702:PMID
4538:ISBN
4513:PMID
3994:and
3966:4480
3730:Sum
3710:Dog
3690:Cat
3685:Sum
2672:2 TP
1987:1120
1789:Dog
1776:Cat
1771:Dog
1768:Cat
1696:and
1678:dual
1213:and
454:and
27:(or
8075:FID
8041:NLP
8031:IoU
7993:MRR
7970:SMC
7902:ROC
7897:AUC
7892:MCC
7844:MAD
7839:MDA
7824:RMS
7799:MAE
7794:MSE
6628:BIC
6623:AIC
5387:doi
5375:136
5338:PMC
5328:doi
5230:doi
5201:doi
5160:PMC
5150:doi
5109:PMC
5099:doi
5032:doi
4959:doi
4915:PMC
4905:doi
4864:PMC
4854:doi
4808:doi
4743:PMC
4733:doi
4692:PMC
4682:doi
4634:doi
4573:doi
4505:doi
4501:405
4238:MCC
3769:MCC
3680:Dog
3677:Cat
3513:MCC
2908:MCC
2608:LRâ
2602:LR+
2484:TNR
2478:FNR
2459:FPR
2453:TPR
2146:hit
2036:or
1873:MCC
1847:FP
1839:FN
1563:MCC
1236:MCC
997:MCC
919:MCC
458:is
425:,
237:= 0
142:= 1
131:= 0
125:= 1
53:In
35:or
19:In
8154::
8003:AP
7867:P4
5401:.
5393:.
5385:.
5373:.
5369:.
5346:.
5336:.
5324:21
5322:.
5318:.
5306:^
5284:.
5259:.
5236:.
5226:28
5224:.
5197:17
5195:.
5191:.
5168:.
5158:.
5146:14
5144:.
5140:.
5117:.
5107:.
5095:21
5093:.
5089:.
5063:.
5038:.
5011:.
5007:.
4988:.
4965:.
4955:27
4953:.
4949:.
4923:.
4913:.
4901:16
4899:.
4895:.
4872:.
4862:.
4850:10
4848:.
4844:.
4828:^
4814:.
4804:17
4802:.
4765:^
4751:.
4741:.
4729:14
4727:.
4723:.
4700:.
4690:.
4680:.
4670:12
4668:.
4664:.
4630:51
4628:.
4579:.
4569:75
4567:.
4563:.
4551:^
4525:^
4511:.
4499:.
4487:^
4037::
3998:.
3962:32
3907:12
3855:12
3799:12
3749:12
2896:.
2738:TP
2733:=
2717:-
2708:=
2692:=
2667:=
2649:=
2621:=
2597:=
2564:PN
2558:TN
2553:=
2536:PP
2530:FP
2525:=
2500:=
2473:=
2448:=
2431:PN
2425:FN
2420:=
2403:PP
2397:TP
2392:=
2360:=
2334:TN
2329:=
2299:FP
2294:=
2225:FN
2220:=
2187:TP
2182:=
2169:,
2099:=
2083:,
1983:16
1851:TN
1844:N
1835:TP
1831:P
1826:N
1823:P
1792:1
1784:2
1215:FN
1207:FP
1205:,
1199:TN
1197:,
1191:TP
739:11
651:,
637:11
624:,
516:01
506:10
493:00
483:11
444:00
439:,
437:01
432:,
430:10
423:11
280:00
252:01
185:10
157:11
96:A
50:.
7834:R
7769:e
7762:t
7755:v
6573:G
6547:F
6539:t
6527:Z
6246:V
6241:U
5443:e
5436:t
5429:v
5409:.
5389::
5381::
5354:.
5330::
5288:.
5270:.
5244:.
5232::
5209:.
5203::
5176:.
5152::
5125:.
5101::
5074:.
5048:.
5034::
5013:2
4992:.
4973:.
4961::
4931:.
4907::
4880:.
4856::
4822:.
4810::
4787:.
4781::
4759:.
4735::
4708:.
4684::
4676::
4640:.
4636::
4587:.
4575::
4519:.
4507::
4364:)
4361:N
4358:F
4355:+
4352:N
4349:T
4346:(
4343:)
4340:P
4337:F
4334:+
4331:N
4328:T
4325:(
4322:)
4319:N
4316:F
4313:+
4310:P
4307:T
4304:(
4301:)
4298:P
4295:F
4292:+
4289:P
4286:T
4283:(
4278:N
4275:F
4269:P
4266:F
4260:N
4257:T
4251:P
4248:T
4242:=
4185:N
4182:F
4179:+
4176:P
4173:F
4170:+
4167:P
4164:T
4161:2
4156:P
4153:T
4150:2
4144:=
4110:N
4107:F
4104:+
4101:P
4098:F
4095:+
4092:N
4089:T
4086:+
4083:P
4080:T
4075:N
4072:T
4069:+
4066:P
4063:T
4057:=
4026:(
4018:(
4016:"
3957:=
3947:2
3941:8
3930:2
3924:4
3913:2
3895:2
3889:7
3878:2
3872:5
3861:2
3843:8
3833:7
3821:4
3811:5
3791:)
3788:3
3785:+
3782:6
3779:(
3773:=
3742:5
3735:7
3723:4
3717:3
3713:1
3703:8
3698:2
3694:6
3638:t
3623:t
3612:2
3608:s
3594:p
3579:p
3568:2
3564:s
3550:p
3535:t
3526:s
3523:c
3517:=
3486:j
3483:i
3479:C
3473:j
3463:i
3455:=
3452:s
3428:k
3425:k
3421:C
3415:k
3407:=
3404:c
3380:i
3377:k
3373:C
3367:i
3359:=
3354:k
3350:p
3325:k
3322:i
3318:C
3312:i
3304:=
3299:k
3295:t
3257:)
3247:k
3239:l
3234:C
3224:l
3213:k
3203:k
3198:|
3190:k
3180:(
3175:)
3169:k
3166:l
3162:C
3156:l
3147:(
3141:k
3128:)
3118:l
3110:k
3105:C
3095:l
3084:k
3074:k
3069:|
3061:k
3051:(
3046:)
3040:l
3037:k
3033:C
3027:l
3018:(
3012:k
2998:k
2995:m
2991:C
2985:l
2982:k
2978:C
2969:m
2966:l
2962:C
2956:k
2953:k
2949:C
2943:m
2933:l
2923:k
2912:=
2884:C
2864:K
2858:K
2836:K
2832:R
2741:/
2675:/
2657:/
2643:1
2641:F
2632:2
2629:/
2605:/
2561:/
2533:/
2508:/
2481:/
2456:/
2428:/
2400:/
2368:/
2365:P
2340:N
2337:/
2305:N
2302:/
2231:P
2228:/
2193:P
2190:/
2113:/
2028:N
2024:P
1978:=
1972:)
1969:2
1966:+
1963:3
1960:(
1954:)
1951:1
1948:+
1945:3
1942:(
1936:)
1933:2
1930:+
1927:6
1924:(
1918:)
1915:1
1912:+
1909:6
1906:(
1901:2
1895:1
1889:3
1883:6
1877:=
1796:3
1780:6
1688:(
1642:)
1639:P
1633:1
1630:(
1627:)
1624:S
1618:1
1615:(
1612:S
1609:P
1604:P
1598:S
1592:N
1588:/
1582:P
1579:T
1571:=
1553:N
1547:P
1544:F
1539:+
1534:P
1531:T
1523:=
1516:P
1507:N
1501:N
1498:F
1493:+
1488:P
1485:T
1477:=
1470:S
1461:P
1458:F
1453:+
1448:N
1445:F
1440:+
1435:P
1432:T
1427:+
1422:N
1419:T
1414:=
1407:N
1369:R
1366:O
1363:F
1353:R
1350:P
1347:F
1337:R
1334:N
1331:F
1321:R
1318:D
1315:F
1301:V
1298:P
1295:N
1285:R
1282:N
1279:T
1269:R
1266:P
1263:T
1253:V
1250:P
1247:P
1240:=
1171:)
1166:N
1163:F
1158:+
1153:N
1150:T
1145:(
1142:)
1137:P
1134:F
1129:+
1124:N
1121:T
1116:(
1113:)
1108:N
1105:F
1100:+
1095:P
1092:T
1087:(
1084:)
1079:P
1076:F
1071:+
1066:P
1063:T
1058:(
1051:N
1048:F
1038:P
1035:F
1025:N
1022:T
1012:P
1009:T
1001:=
964:n
944:n
939:2
928:=
924:|
914:|
854:.
848:)
843:1
836:n
829:n
826:(
823:)
815:1
811:n
804:n
801:(
796:1
789:n
780:1
776:n
768:1
761:n
752:1
748:n
735:n
731:n
725:=
697:1
690:n
664:1
660:n
633:n
612:n
599:d
578:.
570:1
563:n
557:0
550:n
541:0
537:n
528:1
524:n
512:n
502:n
489:n
479:n
472:=
456:y
452:x
448:n
441:n
434:n
427:n
420:n
401:n
378:0
371:n
347:1
340:n
308:0
304:n
276:n
248:n
235:x
213:1
209:n
181:n
153:n
140:x
129:y
123:y
113:y
109:x
39:Ï
37:r
33:Ï
Text is available under the Creative Commons Attribution-ShareAlike License. Additional terms may apply.