BIC of -inf and not applicable Rho-square (0) and adjusted Adj.Rho-square (0) when running a best worst mixed logit
Posted: 27 May 2021, 13:24
We are fitting a best worst mixed logit model in our data collected through an unlabelled DCE on different treatments about a certain medical condition.
The issue we have is that when we run the model, we do not get results for Rho-square (0) and adjusted Adj.Rho-square (0) for both of which the output is “Not applicable”. In addition, BIC returns -inf. Could you please help us understand why this is the case?
FYI: When we run a best worst multinomial logit model on the same data, we get values for all those three outputs.
Below I am providing a few more evidence below from the output file and the code used which may be helpful in answering our questions.
Please let us know if you need more information.
Thank you,
##### For best-worst mixed logit model we used the following code:
### Compute probabilities for 'best' choice using MNL model
mnl_settings = list(
alternatives = c(treat_A=1,treat_B=2, no_treat=3),
avail = list(treat_A=1,treat_B=1, no_treat=1),
choiceVar = best.choice,
V = V,
componentName = "best"
)
P[['choice_best']] = apollo_mnl(mnl_settings, functionality)
### Compute probabilities for 'worst' choice using MNL model
mnl_settings$avail = list(treat_A=(best.choice!=1), treat_B=(best.choice!=2), no_treat=(best.choice!=3))
mnl_settings$choiceVar = worst.choice
mnl_settings$V = lapply(V,"*",-mu_worst)
mnl_settings$componentName = "worst"
P[['choice_worst']] = apollo_mnl(mnl_settings, functionality)
### Combined model
P = apollo_combineModels(P, apollo_inputs, functionality)$model
### Take product across observation for same individual
P = apollo_panelProd(P, apollo_inputs, functionality)
### Average across inter-individual draws
P = apollo_avgInterDraws(P, apollo_inputs, functionality)
### Prepare and return outputs of function
P = apollo_prepareProb(P, apollo_inputs, functionality)
return(P)
##############################
##Output file of the best worst mixed logit model
Model run using Apollo for R, version 0.2.4 on Windows by ezormpas
www.ApolloChoiceModelling.com
Model name : 184_final_Normal_Best_worst_mixed_3000_dr
Model description : 184_final_Normal_Best_worst_mixed_3000_dr
Model run at : 2021-05-05 09:30:32
Estimation method : bfgs
Model diagnosis : successful convergence
Number of individuals : 184
Number of rows in database : 2208
Number of modelled outcomes : 0
Number of cores used : 8
Number of inter-individual draws : 3000 (mlhs)
LL(start) : -3956.205
LL(0) : -3956.205
LL(final) : -1560.958
Rho-square (0) : Not applicable
Adj.Rho-square (0) : Not applicable
AIC : 3163.92
BIC : -Inf
Estimated parameters : 21
Time taken (hh:mm:ss) : 09:06:0.14
pre-estimation : 00:04:24.11
estimation : 03:36:40.59
post-estimation : 05:24:55.44
Iterations : 71
Min abs eigenvalue of Hessian : 3.275636
###################################################
The issue we have is that when we run the model, we do not get results for Rho-square (0) and adjusted Adj.Rho-square (0) for both of which the output is “Not applicable”. In addition, BIC returns -inf. Could you please help us understand why this is the case?
FYI: When we run a best worst multinomial logit model on the same data, we get values for all those three outputs.
Below I am providing a few more evidence below from the output file and the code used which may be helpful in answering our questions.
Please let us know if you need more information.
Thank you,
##### For best-worst mixed logit model we used the following code:
### Compute probabilities for 'best' choice using MNL model
mnl_settings = list(
alternatives = c(treat_A=1,treat_B=2, no_treat=3),
avail = list(treat_A=1,treat_B=1, no_treat=1),
choiceVar = best.choice,
V = V,
componentName = "best"
)
P[['choice_best']] = apollo_mnl(mnl_settings, functionality)
### Compute probabilities for 'worst' choice using MNL model
mnl_settings$avail = list(treat_A=(best.choice!=1), treat_B=(best.choice!=2), no_treat=(best.choice!=3))
mnl_settings$choiceVar = worst.choice
mnl_settings$V = lapply(V,"*",-mu_worst)
mnl_settings$componentName = "worst"
P[['choice_worst']] = apollo_mnl(mnl_settings, functionality)
### Combined model
P = apollo_combineModels(P, apollo_inputs, functionality)$model
### Take product across observation for same individual
P = apollo_panelProd(P, apollo_inputs, functionality)
### Average across inter-individual draws
P = apollo_avgInterDraws(P, apollo_inputs, functionality)
### Prepare and return outputs of function
P = apollo_prepareProb(P, apollo_inputs, functionality)
return(P)
##############################
##Output file of the best worst mixed logit model
Model run using Apollo for R, version 0.2.4 on Windows by ezormpas
www.ApolloChoiceModelling.com
Model name : 184_final_Normal_Best_worst_mixed_3000_dr
Model description : 184_final_Normal_Best_worst_mixed_3000_dr
Model run at : 2021-05-05 09:30:32
Estimation method : bfgs
Model diagnosis : successful convergence
Number of individuals : 184
Number of rows in database : 2208
Number of modelled outcomes : 0
Number of cores used : 8
Number of inter-individual draws : 3000 (mlhs)
LL(start) : -3956.205
LL(0) : -3956.205
LL(final) : -1560.958
Rho-square (0) : Not applicable
Adj.Rho-square (0) : Not applicable
AIC : 3163.92
BIC : -Inf
Estimated parameters : 21
Time taken (hh:mm:ss) : 09:06:0.14
pre-estimation : 00:04:24.11
estimation : 03:36:40.59
post-estimation : 05:24:55.44
Iterations : 71
Min abs eigenvalue of Hessian : 3.275636
###################################################