`

Our blog

Know what we are thinking and doing

December 20, 2019 / by /

Shapley Value Regression

Introduction

In marketing research, the problem of multicollinearity can be raised as a result of using clients' rating responses. People tend to answer to the question of some section/topic in relatively same way. The stronger the correlations between the explanatory variables, the larger the population variances of the distributions of their coefficients, and thus greater the risk of obtaining unreliable coefficients. Although the estimated coefficients will be unbiased, their high variance will decrease the $t$ value, which leads to the failure in rejecting the null hypothesis $H_0:\beta = 0$, and following reduction in the probability of correctly detecting a non-zero coefficient. However, high correlation does not necessarily mean having poor estimation: a large number of observations and sample variances of regressors with a low variance of the error term can produce good estimates. In the case of multicollinearity, data do not contain enough information to distinguish the individual effect of explanatory variables on the dependent variable. There are a lot of ways for solving the multicollinearity, here are some of them:

  • Try to reduce disturbance term by including an omitted important variable in the model.
  • Increase the number of observations to make standard errors smaller.
  • Combine the collinear variables together into a single predictor.
  • Drop one of the problematic variables from the regression.

It is often hard to obtain more data, so the first two methods are not applicable if the design stage of a survey is finished. The last method of solving the multicollinearity problem involves the risk that some of the missing variables may indeed belong to the model which can cause omitted variable bias. In this article, the Shapley value will be used to evaluate the importance of each explanatory variable in the model. The use of Shapley Values comes from the game theory and its purpose is to evaluate the worth of each player's input over all possible combinations of players. As stated by Lipovetsky (Lipovetsky,2006), a regression model can be considered from the perspective of a coalition among players (predictors) to maximize the total value (quality of fitting). This approach yields a model called Shapley Value Regression. The case described in the article adresses customer satisfaction with the restaurant business. The main purpose will be to detect key drivers in the restaurant business and to avoid the problem of the high correlation between variables using this approach.

Data description

># load the required libraries used in the article
if (!require("pacman")) install.packages("pacman")
pacman::p_load(dplyr, readxl, knitr, kableExtra, networkD3, ggcorrplot, 
               stringr, radiant.data, textshape, formattable, RColorBrewer, ggraph, igraph)

In order to solve the problem of finding the key drivers of the restaurant industry, a survey was conducted asking customers to complete a questionnaire covering various aspects of the restaurant.

Below is the description of variables from the synthetic data of clients' responses to a questionnaire that measures how people feel about the main business drivers for restaurants. People express their level of satisfaction after using the products of a restaurant. There is a hierarchy in the data (see the graph below). The variable Satisfaction will be described with two variables Target 1 and Target 2. These two variables are represented with red dots on the dendrogram below. These variables, in turn, will be explained by the Drivers of Targets (blue and light blue dots on the dendrogram) and, finally, each driver should be explained by items of drivers. Each Driver of Target has 13 explanatory variables - items of drivers grouped by a specific color.

All variables are observed and collected via survey and measured in rating scales (1 to 10).

>data_desc <- read_excel("Shapley_Data_Rest.xlsx", sheet = "Labels")
options(knitr.kable.NA = '')
kable(data_desc) %>%
 kable_styling(bootstrap_options = "striped", full_width = F, font_size = 14) %>%
  pack_rows("Drivers of Targets", 4, 15) %>%
  pack_rows("Items of Drivers", 16, 28) %>%
  column_spec(2, bold = T, italic = T, width = "5cm")

Variable Name Group Statements (Variable Description)
Satisfaction Overall satisfaction of customer
Target 1 A restaurant that enables you to step up in life
Target 2 A restaurant that allows you to live the life you choose
Drivers of Targets
Commitment Target 1 Showing commitment to people
Update Continuously updating restaurant
Dedication Sincerely acting in your interest
Discovering better ways Discovering better ways to win favor with you
Fulfill expectation Focus on fulfilling guest's expectations
Comfort/relax Delivering the resources to enable you to feel comfortable and relax
Guest complaint Target 2 Taking into consideration guest complaints
Changing needs Continuously staying relevant by understanding changing customer needs
Consumer health Focus on consumer health
Menu by norms Modifying of menu items driven by regulatory norms
Various segments Work for to various customer segment
New mind-set Understanding the new consumer mind-set
Items of Drivers
Service1 Service Gives polite and prompt answers/advice to my questions
Service2 Use state-of-the-art technology (waiters enter order digital system, POS, online payment etc.)
Service3 Bring an order quickly and properly
Food1 Food Serving food
Food2 Taste of food
Food3 The variety of menu
Food4 The quality of food
Delivery1 Delivery Preciseness
Delivery2 Timing
Delivery3 Quality of food
Cleanliness1 Cleanliness Cleanliness in the kitchen
Cleanliness2 Cleanliness in the hall
Cleanliness3 Cleanliness in the bathroom

There are a total of 28 questions. Thirteen are about restaurant service, food, delivery, and cleanliness. For example Service2: clients are asked to evaluate their satisfaction with the usage of state-of-the-art technology (waiters enter order digital system, POS, online payment).

We want to know how the variables relate to the satisfaction of clients at each level. In each step, the most important variable should be selected.

>df <- read_excel("Shapley_Data_Rest.xlsx", sheet = "Data")
dim(df)
## [1] 500  28

There are 500 respondents who have answered 28 questions in the synthetic dataset. It can be seen that the ratings are highly correlated:

>paste("The correlation between Target 1 and Target 2 is", round(cor(df[,2:3])[2],3))
## [1] "The correlation between Target 1 and Target 2 is 0.859"

There is high correlation between the effort of the restaurant to help the clients step up in life and its effort to provide the life their customers choose. The presence of a high correlation between the independent variables can produce erratic coefficients.

>corfun <- function(x){corr <- round(cor(x), 2)
ggcorrplot(corr, lab = TRUE, method = "circle", lab_size = 3)}
>corfun(df[,4:9])

The plot above shows the correlation between the drivers of target 1. It can be seen that the Pearson correlation coefficients for all pairs are more than 0.73. The highest correlation is between the variables Dedication (the restaurant is sincerely acting in your interest) and Comfort/relax (restaurant provides the resources to enable you to feel comfortable and relaxed).

>corfun(df[,10:15])

The plot above shows the correlation between the drivers of target 2. Similarly, there are high correlations between all pairs of variables. The highest correlation is between the variables Menu by norms (restaurant modifies menu items driven by regulatory norms) and Various segments (restaurant works for various customer segments).

>corfun(df[,16:28])

Finally, for the last level, the correlation between the items of drivers is considered. As can be expected, there is high correlation in each group of items. The highest correlation is between the cleanliness in the hall and bathroom, followed by the strong correlation between the taste of food and quality of food, and between the taste of food and the variety of the menu.

We are going to study Shapley value to detect how it can be used to avoid multicollinearity and detecting the key driver for the restaurant industry.

Shapley value regression

Shapley Value Regression is based on the thesis and post-doctoral work of an American mathematician and a Nobel Prize-winning economist Lloyd Shapley (1953). The Shapley value is a central solution concept in cooperative game theory. In order to assess the player's contribution in a game, each individual player has its own assigned value. The Shapley value associated with each player in each game has a unique payoff - his 'value' (expected marginal contribution to a random coalition). The application of this value in regression analysis is quite intuitive: thisS approach evaluates the contribution of each regressor variables to the model. When fitting the multiple linear regression model, the obtained $R^2$ does not show the effect of each variable in explaining the depending variable (in the cooperative game). To distinguish the contribution made by the individual member of the game, the Shapley value decomposition should be used. The share of the regressor variable $x_i$ for a given set of $k$ predictor variables is given by the following formula:

$$ S({x_i}) = \frac{1}{k} * \sum_{r=1}^{k} * \frac{ \sum_{c=1}^{l} (R^2_{i,r}-R^2_{j,r-1})}{l} $$

where

  • $k$ is the number of regressor variables in the multiple linear regression model
  • $R^2_{(i,r)}$ obtained from the model where the regressors are an r-membered subset of all possible regressors with $i^{th}$ variable
  • $R^2_{(j,r-1)}$ obtained from the model where the regressors are an r-membered subset of all possible regressors without $i^{th}$ variable.

Suppose we have 3 variables, and we want to obtain Shapley value for the variable $x_1$. We will have the eight possibilities to fit a linear regression with these 3 variables: 0, x1, x2, x3, x1 x2, x1 x3, x2 x3, x1 x2 x3. In this case $i=1$, because the computation is done for $x_1$, $r=3$ and, thus, $r-1=2$, $k=3$ is the number of cases of possible models, $l$ is the number of models in each case:

The weights of the regressions are based on the number of possible models. We will have

  • 2 regressions where $x_1$ is used with one other explanatory variable: (x1, x2); (x1, x3)
  • 1 regression where $x_1$ is used alone (x1)
  • 1 regression where $x_1$ is used with two other variables (x1,x2,x3)

Thus we will have the following weighted Shapley value for the variable $x_1$:

$$ SV_{x_1} = \dfrac{1}{3}(R^2_{x_1}-R^2_{\beta_0})+\dfrac{1}{6}(R^2_{x_1;x_2}-R^2_{x_2}) + \dfrac{1}{6}(R^2_{x_1;x_3}-R^2_{x_3}) + \dfrac{1}{3}(R^2_{x_1;x_2;x_3}-R^2_{x_2;x_3}) $$

In order to evaluate the key drivers of restaurant industries, we will use the regression described above.

Implementing in R

Level 1

For the first level, we need to evaluate how the variables Target 1 (a restaurant that enables you to step up in life) and Target 2 (a restaurant that allows you to live the life you choose) explain the overall satisfaction of the customer.

>colnames(df)[2] <- "Target1"
colnames(df)[3] <- "Target2"
reglev1 <- lm(Satisfaction ~ Target1 + Target2, data = df)
summary(reglev1)

## 
## Call:
## lm(formula = Satisfaction ~ Target1 + Target2, data = df)
## 
## Residuals:
##     Min      1Q  Median      3Q     Max 
## -5.2795 -0.7771  0.0047  0.8526  7.0956 
## 
## Coefficients:
##             Estimate Std. Error t value Pr(>|t|)    
## (Intercept)  2.18626    0.23658   9.241  < 2e-16 ***
## Target1      0.50239    0.06295   7.981 1.01e-14 ***
## Target2      0.21578    0.06381   3.381 0.000778 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 1.488 on 497 degrees of freedom
## Multiple R-squared:  0.4831, Adjusted R-squared:  0.481 
## F-statistic: 232.2 on 2 and 497 DF,  p-value: < 2.2e-16

The implemented regression shows that both variables are statistically significant. And based on estimated coefficients the variable Target 1 is more important. However, we cannot see the relative importance of each variable by just looking at the summary of linear regression models. So, the Shapley Values will be used, for both coefficients separately, to identify the most important driver of clients' overall satisfaction at the first level.

Creating a universal function for Shapley value calculation:

>shap <- function(formula, var){
  ifelse(length(str_split(paste(formula)[3], pattern = "\\s\\+", simplify = T))!=1, 
      summary(lm(data=df, formula))$r.squared - 
      summary(lm(data=df, as.formula(gsub(paste0(paste(formula)[2], paste(formula)[1],
          paste(formula)[3]), pattern = paste0("\\+", var),replacement = ""))))$r.squared, 
      summary(lm(data=df, formula ))$r.squared -
      summary(lm(data=df, as.formula(paste0(paste(formula)[2],paste(formula)[1],1))))$r.squared
  )}

We need to calculate the Shapley value for each independent variable using the functions above:

>(ShapValT1 <- 1/2*shap(formula = Satisfaction ~ +Target1+Target2, var = "Target1") +
    1/2 * shap(formula = Satisfaction ~ +Target1, var = "Target1"))
## [1] 0.2687259
>(ShapValT2 <- 1/2*shap(formula = Satisfaction ~ +Target1+Target2, var = "Target2") +
    1/2 * shap(formula = Satisfaction ~ +Target2, var = "Target2"))
## [1] 0.2084265

Finally, calculated values are re-based so that they add up to 1:

>shaplev1 <- data.frame(ShapValT1 = ShapValT1/(ShapValT1+ShapValT2), 
                       ShapValT2 = ShapValT2/(ShapValT1+ShapValT2))
kable(t(shaplev1)) %>%
 kable_styling(bootstrap_options = "striped", full_width = F)

ShapValT1 0.5631867
ShapValT2 0.4368133

It can be seen that Target 1 is the most important at 0.56. In other words, the overall satisfaction of the customer is mostly described by the ability of the restaurant to help clients step up in life. We can decompose the regression and find out what the r-squared is made of and have a similar result by using the function calc.relimp from the package relaimpo. The package provides the relative importance metric lmg introduced by Lindemann, Merenda, and Gold.

>library(relaimpo)
calc.relimp(reglev1, type = c("lmg"), rela = TRUE, rank = TRUE)

## Response variable: Satisfaction 
## Total response variance: 4.26796 
## Analysis based on 500 observations 
## 
## 2 Regressors: 
## Target1 Target2 
## Proportion of variance explained by model: 48.31%
## Metrics are normalized to sum to 100% (rela=TRUE). 
## 
## Relative importance metrics: 
## 
##               lmg
## Target1 0.5562547
## Target2 0.4437453
## 
## Average coefficients for different model sizes: 
## 
##                1X       2Xs
## Target1 0.6853051 0.5023914
## Target2 0.6533959 0.2157771

It can be seen from the table Relative importance metrics: that, although, the lmg value slightly differs from the above calculated one, the conclusion is the same.

Level 2

To indicate the most important explanatory variable/s for Target 1 and Target 2 the drivers of these variables will now be studied. Now, the variable Target 1 and Target 2 are dependent on their drivers of the target correspondingly.

>reglev2t1 <- lm(Target1 ~ td1_1 + td1_2 + td1_3 + td1_4 + td1_5 + td1_6, data = df)
reglev2t2 <- lm(Target2 ~ td2_1 + td2_2 + td2_3 + td2_4 + td2_5 + td2_6, data = df)

To see the relative importance of each variable, we need to calculate 6 Shapley values using the approach above for each target variable. As the procedure of calculation is similar and in order to facilitate the computational process, we will use the result obtained using the function from `relaimpo`. The result is in the table below:

>shaplev2t1 <- calc.relimp(reglev2t1, type = c("lmg"), rela = TRUE, rank = TRUE)$lmg
shaplev2t2 <- calc.relimp(reglev2t2, type = c("lmg"), rela = TRUE, rank = TRUE)$lmg
shaplev2 <- data.frame(cbind(c("Commitment", "Update", "Dedication", "Discovering better ways", 
  "Fulfill expectation", "Comfort/relax"), round(as.numeric(shaplev2t1),2), 
  c("Guest complaint", "Changing needs", "Consumer health", "Menu by norms", 
  "Various segments", "New mind-set"), round(as.numeric(shaplev2t2),2)))
colnames(shaplev2) <- c("Var T1", "Target1", "Var T2", "Target2")

>shaplev2 %>%
  rownames_to_column('shap') %>%
    mutate(Target1 = color_tile("white","lightpink")(Target1),
           Target2 = color_tile("white","khaki1")(Target2)) %>%
  column_to_rownames('shap') %>%    
kable(escape = F, booktabs = T) %>%
kable_styling( full_width = F, font_size = 13)

In the case of Target 1, the focus of the restaurant on discovering better ways to win favor with its clients is the most important key driver for the target variable. For Target 2 (a restaurant that allows you to live the life you choose) the most two important variables are the focus of the restaurant on consumer health (actually, the score for Consumer health is 0.1899209, the score for Menu by norms is 0.1895162).

Level 3

And finally, we will attempt to reveal the most important variable for each driver of targets. The result from package relaimpo is in the table below:

>shaplev3 <- sapply(colnames(df[-c(1:3, 16:28)]),
  function(x){form <- as.formula(paste0(x,"~Service1 + Service2 + Service3 + Food1 + Food2 + 
   Food3 + Food4 + Delivery1 + Delivery2 + Delivery3 + Cleanliness1 + Cleanliness2 + Cleanliness3"))
    reglev3  <- lm(form, data = df)
    calc.relimp(reglev3 , type = c("lmg"), rela = TRUE, rank = TRUE)$lmg})
cn <- colnames(shaplev3)
rownames(shaplev3) <- c("Politeness", "Technology", "Proper order",  "Food serving", "Food taste", 
  "Food variety","Food quality", "Delivery accuracy", "Delivery timing", "Delivery food quality", 
  "Clean kitchen", "Clean hall", "Clean bathroom")
data.frame(round(shaplev3,2)) %>%
  rownames_to_column('shap') %>% mutate(
    td1_1 =color_tile("white","lightgreen")(td1_1), td1_2 =color_tile("white","lightgreen")(td1_2),
    td1_3 =color_tile("white","lightgreen")(td1_3), td1_4 =color_tile("white","lightgreen")(td1_4),
    td1_5 =color_tile("white","lightgreen")(td1_5), td1_6 =color_tile("white","lightgreen")(td2_6),
    td2_1 =color_tile("white","lightblue")(td2_1), td2_2 =color_tile("white","lightblue")(td2_2),
    td2_3 =color_tile("white","lightblue")(td2_3), td2_4 =color_tile("white","lightblue")(td2_4),
    td2_5 =color_tile("white","lightblue")(td2_5), td2_6 =color_tile("white","lightblue")(td2_6)) %>%
  column_to_rownames('shap') %>%
  rename("Commitment" = cn[1], "Update" = cn[2], "Dedication" = cn[3], "Discovering better ways" = cn[4],
    "Fulfill expectation" = cn[5], "Comfort/relax" = cn[6], "Guest complaint" = cn[7], "Changing needs" = cn[8],
    "Consumer health" = cn[9], "Menu by norms" = cn[10], "Various segments" = cn[11], "New mind-set" = cn[12]) %>%
kable(escape = F, booktabs = T) %>%
kable_styling( full_width = F, font_size = 13) %>%
  row_spec(0, bold = T, font_size = 9)

It seems that politeness of staff is the most important item for all drivers (for both targets). In other words, the satisfaction of clients with the staff attitude toward customers has the biggest impact and makes up to approximately 20% of their argument in your favor. For almost all drivers the quality of food is the second key item in terms of importance. The cleanliness in the kitchen is the third dominant item for drivers like continuously updating restaurants, acting in the interest of the customer, fulfilling their expectations and paying attention to consumer health. For the following drivers: discovering better ways to win favor with the customer, working for various customer segments the quality of delivery food is the third important variable. And finally, the accuracy of delivery is important in the positive assessment of such drivers as delivering the resources to enable clients to feel comfortable and relax, taking into consideration guest complaints, understanding the changing needs and a new mindset of customer and paying attention to consumer health.

Conclusion

Shapley Value Regression is widely used in ranked customer responses, because of multicollinearity. In order to understand the key drivers of successful restaurant business the Shapley Value, which shows the relative importance of predictors, can be used. The idea of Shapley value regression is in the changes in $R^2$ when the chosen predictor is removed from the model. So for each regressor, all possible subsets of the remaining predictors are used to evaluate the regression and the explanatory variable with the highest additional contribution will be the most important prediction.

Shapley value assigns relative ranking for each predictor by showing the dominance of explanatory variables: from the Shapley values regression, we can conclude that the variable about the ability of restaurant assist its customers in stepping up in life is the most important in describing their overall satisfaction. In the second stage analysis, the dominant variable became the effort of the restaurant to discover better ways to win favor with clients (for Target 1) and paying attention to consumer's health is most important for Target 2. Finally, the third level shows that for this task the politeness of staff mostly impacts gaining a positive attitude of customers toward the restaurant.

For other types of models (for example generalized linear models: logistic, Poisson regression, etc.) calculation of $R^2$ is not appropriate and the approach of using pseudo-$R^2$ or other approached to calculate the relative weights (see here) will be applied. Also, for the future work adjusted $R^2$ (as number of regressors in different models is different) instead of $R^2$ can be considered.

Reference List

Mishra, S.K. (2016) "Shapley value regression and the resolution of multicollinearity", MPRA, (72116). Available at: https://mpra.ub.uni-muenchen.de/72116/.

Lipovetsky, S. (2006) "Entropy Criterion In Logistic Regression And Shapley Value Of Predictors", Journal of modern applied statistical methods: JMASM

Hart S. (1989) Shapley Value. In: Eatwell J., Milgate M., Newman P. (eds) Game Theory. The New Palgrave. Palgrave Macmillan, London