ApolloChoiceModelling forum

We are fitting a best worst mixed logit model in our data collected through an unlabelled DCE on different treatments about a certain medical condition.

The issue we have is that when we run the model, we do not get results for Rho-square (0) and adjusted Adj.Rho-square (0) for both of which the output is “Not applicable”. In addition, BIC returns -inf. Could you please help us understand why this is the case?

FYI: When we run a best worst multinomial logit model on the same data, we get values for all those three outputs.

Below I am providing a few more evidence below from the output file and the code used which may be helpful in answering our questions.

Please let us know if you need more information.

Thank you,


##### For best-worst mixed logit model we used the following code:

### Compute probabilities for 'best' choice using MNL model
mnl_settings = list(
alternatives = c(treat_A=1,treat_B=2, no_treat=3),
avail = list(treat_A=1,treat_B=1, no_treat=1),
choiceVar = best.choice,
V = V,
componentName = "best"
)
P[['choice_best']] = apollo_mnl(mnl_settings, functionality)

### Compute probabilities for 'worst' choice using MNL model
mnl_settings$avail = list(treat_A=(best.choice!=1), treat_B=(best.choice!=2), no_treat=(best.choice!=3))
mnl_settings$choiceVar = worst.choice
mnl_settings$V = lapply(V,"*",-mu_worst)
mnl_settings$componentName = "worst"

P[['choice_worst']] = apollo_mnl(mnl_settings, functionality)

### Combined model
P = apollo_combineModels(P, apollo_inputs, functionality)$model

### Take product across observation for same individual
P = apollo_panelProd(P, apollo_inputs, functionality)

### Average across inter-individual draws
P = apollo_avgInterDraws(P, apollo_inputs, functionality)

### Prepare and return outputs of function
P = apollo_prepareProb(P, apollo_inputs, functionality)
return(P)

##############################

##Output file of the best worst mixed logit model

Model run using Apollo for R, version 0.2.4 on Windows by ezormpas
www.ApolloChoiceModelling.com

Model name : 184_final_Normal_Best_worst_mixed_3000_dr
Model description : 184_final_Normal_Best_worst_mixed_3000_dr
Model run at : 2021-05-05 09:30:32
Estimation method : bfgs
Model diagnosis : successful convergence
Number of individuals : 184
Number of rows in database : 2208
Number of modelled outcomes : 0

Number of cores used : 8
Number of inter-individual draws : 3000 (mlhs)

LL(start) : -3956.205
LL(0) : -3956.205
LL(final) : -1560.958
Rho-square (0) : Not applicable
Adj.Rho-square (0) : Not applicable
AIC : 3163.92
BIC : -Inf


Estimated parameters : 21
Time taken (hh:mm:ss) : 09:06:0.14
pre-estimation : 00:04:24.11
estimation : 03:36:40.59
post-estimation : 05:24:55.44
Iterations : 71
Min abs eigenvalue of Hessian : 3.275636
###################################################

Hi, could you please share the files with me and I'll investigate

Hi,

Thanks. We have sent the output file by email.

Hi, to allow us to diagnose problems, we need the data file and code file too. We delete these after testing of course

Hi,

Thanks for offering to help us. We have eventually managed to solve the issue internally. We figured out we simply had to remove "$model" from the code below. After that the BIC and Rho-square values were normally provided by the model.

### Combined model
P = apollo_combineModels(P, apollo_inputs, functionality)$model

I assume we got a little confused as the code was not producing an error and all the remaining output except those values was normally being presented before the removal.

Thanks