Important: Read this before posting to this forum

  1. This forum is for questions related to the use of Apollo. We will answer some general choice modelling questions too, where appropriate, and time permitting. We cannot answer questions about how to estimate choice models with other software packages.
  2. There is a very detailed manual for Apollo available at http://www.ApolloChoiceModelling.com/manual.html. This contains detailed descriptions of the various Apollo functions, and numerous examples are available at http://www.ApolloChoiceModelling.com/examples.html. In addition, help files are available for all functions, using e.g. ?apollo_mnl
  3. Before asking a question on the forum, users are kindly requested to follow these steps:
    1. Check that the same issue has not already been addressed in the forum - there is a search tool.
    2. Ensure that the correct syntax has been used. For any function, detailed instructions are available directly in Apollo, e.g. by using ?apollo_mnl for apollo_mnl
    3. Check the frequently asked questions section on the Apollo website, which discusses some common issues/failures. Please see http://www.apollochoicemodelling.com/faq.html
    4. Make sure that R is using the latest official release of Apollo.
  4. If the above steps do not resolve the issue, then users should follow these steps when posting a question:
    1. provide full details on the issue, including the entire code and output, including any error messages
    2. posts will not immediately appear on the forum, but will be checked by a moderator first. This may take a day or two at busy times. There is no need to submit the post multiple times.

BIC of -inf and not applicable Rho-square (0) and adjusted Adj.Rho-square (0) when running a best worst mixed logit

Ask questions about the results reported after estimation. If the output includes errors, please include your model code if possible.
Post Reply
MKarimi
Posts: 3
Joined: 27 May 2021, 11:37

BIC of -inf and not applicable Rho-square (0) and adjusted Adj.Rho-square (0) when running a best worst mixed logit

Post by MKarimi »

We are fitting a best worst mixed logit model in our data collected through an unlabelled DCE on different treatments about a certain medical condition.

The issue we have is that when we run the model, we do not get results for Rho-square (0) and adjusted Adj.Rho-square (0) for both of which the output is “Not applicable”. In addition, BIC returns -inf. Could you please help us understand why this is the case?

FYI: When we run a best worst multinomial logit model on the same data, we get values for all those three outputs.

Below I am providing a few more evidence below from the output file and the code used which may be helpful in answering our questions.

Please let us know if you need more information.

Thank you,


##### For best-worst mixed logit model we used the following code:

### Compute probabilities for 'best' choice using MNL model
mnl_settings = list(
alternatives = c(treat_A=1,treat_B=2, no_treat=3),
avail = list(treat_A=1,treat_B=1, no_treat=1),
choiceVar = best.choice,
V = V,
componentName = "best"
)
P[['choice_best']] = apollo_mnl(mnl_settings, functionality)

### Compute probabilities for 'worst' choice using MNL model
mnl_settings$avail = list(treat_A=(best.choice!=1), treat_B=(best.choice!=2), no_treat=(best.choice!=3))
mnl_settings$choiceVar = worst.choice
mnl_settings$V = lapply(V,"*",-mu_worst)
mnl_settings$componentName = "worst"

P[['choice_worst']] = apollo_mnl(mnl_settings, functionality)

### Combined model
P = apollo_combineModels(P, apollo_inputs, functionality)$model

### Take product across observation for same individual
P = apollo_panelProd(P, apollo_inputs, functionality)

### Average across inter-individual draws
P = apollo_avgInterDraws(P, apollo_inputs, functionality)

### Prepare and return outputs of function
P = apollo_prepareProb(P, apollo_inputs, functionality)
return(P)

##############################

##Output file of the best worst mixed logit model

Model run using Apollo for R, version 0.2.4 on Windows by ezormpas
www.ApolloChoiceModelling.com

Model name : 184_final_Normal_Best_worst_mixed_3000_dr
Model description : 184_final_Normal_Best_worst_mixed_3000_dr
Model run at : 2021-05-05 09:30:32
Estimation method : bfgs
Model diagnosis : successful convergence
Number of individuals : 184
Number of rows in database : 2208
Number of modelled outcomes : 0

Number of cores used : 8
Number of inter-individual draws : 3000 (mlhs)

LL(start) : -3956.205
LL(0) : -3956.205
LL(final) : -1560.958
Rho-square (0) : Not applicable
Adj.Rho-square (0) : Not applicable
AIC : 3163.92
BIC : -Inf


Estimated parameters : 21
Time taken (hh:mm:ss) : 09:06:0.14
pre-estimation : 00:04:24.11
estimation : 03:36:40.59
post-estimation : 05:24:55.44
Iterations : 71
Min abs eigenvalue of Hessian : 3.275636
###################################################
stephanehess
Site Admin
Posts: 974
Joined: 24 Apr 2020, 16:29

Re: BIC of -inf and not applicable Rho-square (0) and adjusted Adj.Rho-square (0) when running a best worst mixed logit

Post by stephanehess »

Hi, could you please share the files with me and I'll investigate
--------------------------------
Stephane Hess
www.stephanehess.me.uk
MKarimi
Posts: 3
Joined: 27 May 2021, 11:37

Re: BIC of -inf and not applicable Rho-square (0) and adjusted Adj.Rho-square (0) when running a best worst mixed logit

Post by MKarimi »

Hi,

Thanks. We have sent the output file by email.
stephanehess
Site Admin
Posts: 974
Joined: 24 Apr 2020, 16:29

Re: BIC of -inf and not applicable Rho-square (0) and adjusted Adj.Rho-square (0) when running a best worst mixed logit

Post by stephanehess »

Hi, to allow us to diagnose problems, we need the data file and code file too. We delete these after testing of course
--------------------------------
Stephane Hess
www.stephanehess.me.uk
MKarimi
Posts: 3
Joined: 27 May 2021, 11:37

Re: BIC of -inf and not applicable Rho-square (0) and adjusted Adj.Rho-square (0) when running a best worst mixed logit

Post by MKarimi »

Hi,

Thanks for offering to help us. We have eventually managed to solve the issue internally. We figured out we simply had to remove "$model" from the code below. After that the BIC and Rho-square values were normally provided by the model.

### Combined model
P = apollo_combineModels(P, apollo_inputs, functionality)$model

I assume we got a little confused as the code was not producing an error and all the remaining output except those values was normally being presented before the removal.

Thanks
Post Reply