Evaluation of phishing email classification features: Reliability ratio measure

Abstract

Heuristic-based anti-phishing systems are widely implemented to detect phishing attacks. Selecting most reliable classification features however, is a challenging task. Information Gain IG, Gain Ratio GR, Term Frequency-Inverse Document Frequency TF-IDF, Chi-Square are examples of measures that have proven their excellence in text classification field. These measures have also been used to evaluate phishing classification features. Phishing emails however, are difficult to be detected based only on their subject and content texts since they are usually constructed to look like legitimate ones. Text classification measures may produce high error rates if they are naively employed to detect phishing instances. Some attempts therefore have been done to adapt them to evaluate phishing classification features. Average Gain AG for example, which is an IG-dependent measure, was used to adapt IG measure to be used in phishing classification field. In this study, Reliability Ratio RR measure is proposed to evaluate the reliability of phishing email classification features. Experimental results have proven the effectiveness of the proposed RR measure compared with other evaluated measures such as IG and AG.