User:WikiPublisher2019/sandbox

Theory
Cost-sensitive learning is a certain type of classification that takes costs of misclasification (or any other type of cost) into consideration. Most classification methods minimize the error rate without considering the cost of misclassification. In contrast to that the goal of cost-sensitive learning is to minimize costs rather than error rates. The common classification methods assume that all misclassifications lead to the same costs and ignores different penalties. Cost-sensitive learning however takes different costs for different misclassification into consideration.

There are two ways of designing a cost-sensitive classifier. The first is to design classifiers that are cost-sensitive themselves (direct method). The other option is converting a cost-insensitive classifier into a cost-sensitive classifier (cost-sensitive meta-learning method). In both ways the costs are not necessarily monetary. Costs can also indicate a severity of sickness or a waste of time (opportunity costs).

Application
Cost-sensitive learning algorithms are often used for real-world applications, as the costs of different type of misclassification are not always equal. As an example for cost-sensitive learning the diagnosis of certain sicknesses shows how false classification leads to different costs. The diagnosis of sickness for a healthy person (false positive error) leads to less higher costs than not diagnosing a person who is actually sick (false negative error). For cost-sensitive learning algorithms we often use cost matrices, to indicate what costs are generated with certain types of misclassification.

Imbalanced learning data
Moreover, cost-sensitive algorithms are used for imbalanced learning data, which means that in a classification problem different classes are unequally represented. It leads to having a so-called minority class and a majority class. The problem of inbalanced data is widely known in machine learning. . Most classifiers show an increased false negative rate when having a majority. Cost-sensitive learning helps finding the model with the lowest misclassification costs and therefore deals with the problem of imbalanced learning data.

$$ Cost=C(0,1)*FN+C(1,0)*FP $$

Adapting loss functions to incorporate misclassification cost
It is possible to adapt certain loss functions to incorporate the costs which arise due to misclassification. For example the Perceptron loss function $$ l_{p} (w;x,y)=max(0,-yw^{T} x) $$ , can be easily changed to incorporate costs which arise due to wrongly misclassifying a data point x. Note that y corresponds to the true class of the training data point x and w to corresponding model parameters. Incorporating the misclassification costs leads to the following updated Perceptron loss function $$ l_{P,C} (w;x,y)=c_{y}(0,-yw^{T} x) $$ , where $$c_{y}$$ corresponds to a cost associated with misclassifying a training data point with label y. Remark that $$c_{y}$$ > 0 must hold in order to have a desired strictly non-negative cost.