Scorecard Validation

Gini & ROC Curve
Kolmogorov-Smirnov Curve
Risk Segments
Strategy Curve
Approval vs “Bad” Rate
“Good” and “Bad” distribution
Cumulative Event Rates
Chi-squared Scorecard Stability Test
Score Points Distribution

Gini & ROC Curve

The most widely used way to evaluate quality of a scorecard is Gini coefficient 
and ROC curve

ROC curve that located higher and more to the left is indicates better scorecard quality.

The evaluation of the quality of classification by the Gini coefficient can be checked with the help of the following tables:

Application Scoring


Collection Scoring


Behavioral Scoring


Fraud Scoring

The Gini approach is not relevant for fraud scoring because the number of fraudsters in a typical dataset is too small, and scorecard quality should be analyzed with other methods.

ROC Curve values usually calculated not only for the dataset that was used to create a scorecard (training set), but also for a separate out-of-sample validation dataset. ROC Curve values for training and validation datasets should be close to each other. When several scorecards are compared, preference is given to the one with the highest Gini value.

Unacceptable ROC curve performance.
Scorecard need to be improved.


Acceptable ROC curve performance.


Perfect ROC curve performance.



Kolmogorov-Smirnov Curve

Shows how strong is the difference between the distributions of the “Bad” and “Good” borrowers

Kolmogorov-Smirnov curve shows the difference between the distribution of “Goods” and “Bads”. The maximum difference between “Goods” and “Bads” distribution known as a Kolmogorov-Smirnov value, that is often used together with Gini value to asses scorecard quality.


Kolmogorov-Smirnov values usually calculated not only for the dataset that was used to create a scorecard (training set), but also for a separate out-of-sample validation dataset. Kolmogorov-Smirnov values for training and validation datasets should be close to each other.

When several scorecards compared, preference is given to the one with the highest Kolmogorov-Smirnov value.

Unacceptable Kolmogorov-Smirnov curves



Risk Segments

Allows evaluating the logicality and magnitude of the risks’ distribution and expected odds for each risk segment

Additionaly, Risk Segments Graph helps to select cut-off point based on “Good : Bad” odds.

Unacceptable Risk Segments Graph


Acceptable Risk Segments Graph



Strategy Curve

Allows evaluating the degree of the discrepancy between actual odds and odds predicted by scorecard and determining those score ranges where the scorecard makes majority of mistake

Additionally, Strategy Curve helps making decisions about the use of the developed Scorecard and its complete or partial restructuring, depending on the degree of the found discrepancy.

Unacceptable Strategy Curve


Acceptable Risk Segments Graph



Approval vs “Bad” Rate

Shows the dependence between the approved borrowers and the corresponding share of the “Bad” borrowers for each score

Approval vs “Bad” Rate chart allows setting the initial cut-off point value that ensures the minimum level of the share of the “Bad” borrowers under a permissible level of the approved borrowers.

Unacceptable Approval vs “Bad” Rate chart


Acceptable Approval vs “Bad” Rate chart



“Good” and “Bad” distribution

Allows to visually assess the distribution of “Good” and “Bad”, resulting from the use Scorecard

The typical “hill-like” shape of the peaks and easily seen difference between “Good” and “Bad” distributions indicate a proper Scorecard performance and its ability to differentiate between “Goods” and “Bads”.

Unacceptable distribution graphs

Scorecard does not help to differentiate “Goods” from “Bads”


Acceptable distribution graphs



Cumulative Event Rates

Shows the dependence between changes in rates of “Good” and “Bad” and changes in the score

 An increase in the share of the “Good” outcomes, accompanied by a decrease in the number of “Bad” accounts, confirms that the Scorecard’s performance is logical.

A monotonous decrease in the share of the “Bad” borrowers in the upper score range speaks about the correctness of the Scorecard’s performance and its ability to differentiate “Bad” borrowers into the lower part of the working range.

Acceptable Cumulative Event Rates graph


Unacceptable Cumulative Event Rates graph



Chi-squared Scorecard Stability Test

Scorecard Stability – is scorecard ability to perform with expected quality even with a drift in customer base.

Chi-squared Test calculates difference between actual and predicted distributions for all borrower characteristics. The lower the difference, the more stable the developed scorecard is.


Chi-squared Test is based on Hosmer-Lemeshow factor that is calculated for each scorecard characteristics as a sum of chi-square values for each attribute (sub-characteristic).


HL factor is then compared with a reference value of chi-squared distribution for the corresponding number of the degrees of freedom. The number of the degrees of freedom is equal to the number of attributes minus 1.



Score Points Distribution

Allows visual evaluation of the scorecard working range. The columns in the diagram correspond to the number of borrowers who got assigned a certain score.


Score distribution must have a “hill-like” visual form that maximally resembles normal distribution.


If the distribution has a “hill-like” visual form that is concentrated in a very narrow score range, that will cause problems with risk control. The bigger the number of borrowers within a small range is, the scarcer is the possibility of preventing changes in the credit portfolio, the more change-sensitive is the risk control system. In this case, we need to re-calibrate the scorecard by setting broader limits of the score range.


If the distribution is divided in to separate segments, that means that there are borrowers, whose credit behavior differs in principle from that of the main population. 
In this case, we need to find the categories of borrowers, whose behavior is “very good” or “very bad”, andchange the way, in which corresponding characteristics are categorized.



Need Scoring trainings?   Scorecard Development Services

FREE Trial Scorecard Development Software   FREE Trial Credit Scoring System

Credit Scoring Software is the most easy-to-use and the fastest to integrate scoring system.