Important: Read this before posting to this forum

  1. This forum is for questions related to the use of Apollo. We will answer some general choice modelling questions too, where appropriate, and time permitting. We cannot answer questions about how to estimate choice models with other software packages.
  2. There is a very detailed manual for Apollo available at http://www.ApolloChoiceModelling.com/manual.html. This contains detailed descriptions of the various Apollo functions, and numerous examples are available at http://www.ApolloChoiceModelling.com/examples.html. In addition, help files are available for all functions, using e.g. ?apollo_mnl
  3. Before asking a question on the forum, users are kindly requested to follow these steps:
    1. Check that the same issue has not already been addressed in the forum - there is a search tool.
    2. Ensure that the correct syntax has been used. For any function, detailed instructions are available directly in Apollo, e.g. by using ?apollo_mnl for apollo_mnl
    3. Check the frequently asked questions section on the Apollo website, which discusses some common issues/failures. Please see http://www.apollochoicemodelling.com/faq.html
    4. Make sure that R is using the latest official release of Apollo.
  4. If the above steps do not resolve the issue, then users should follow these steps when posting a question:
    1. provide full details on the issue, including the entire code and output, including any error messages
    2. posts will not immediately appear on the forum, but will be checked by a moderator first. This may take a day or two at busy times. There is no need to submit the post multiple times.

Error with class convergence and lambda values for latent class nested logit model

Ask questions about errors you encouunter. Please make sure to include full details about your model specifications, and ideally your model file.
Post Reply
robbie_research
Posts: 2
Joined: 25 Aug 2022, 23:00

Error with class convergence and lambda values for latent class nested logit model

Post by robbie_research »

Hi Stephane and the Apollo Choice Modelling Community,

I am trying to run a latent class nested logit model (with three classes and five alternatives) and am running into some puzzling issues around the latent class estimation.

Firstly, the latent class likelihoods are not converging, despite the overall model converging and Apollo providing estimates with standard errors. I'm not sure how much of an issue this is and I am struggling to fix it. The error message is: "-Inf Likelihood equal to zero for at least one individual in this component."

Secondly, and perhaps relatedly, the lambda latent class estimates do not correspond to the nesting parameters that Apollo shows after the main results. The lambda estimate for class 2 will read something like 0.6 whereas the nesting parameter (in the nesting structure results) will read ~0.98. I suspect that solving one issue will solve the other also, but I'm not sure.

Any thoughts or insights into these issues would be immensely appreciated. I have pasted the full code followed by the results below.

Thanks in advance for your help :)

____________________________________________________________________________________________
Code

# ################################################################# #

### Clear memory
rm(list = ls())

### Load Apollo library
library(apollo)
### Initialise code
apollo_initialise()

### Set core controls
apollo_control = list(
modelName ="LC_NL_3",
modelDescr ="LC-NL, 3 classes, no socios",
indivID ="id",
outputDirectory = "tables",
nCores = 4
)

# ################################################################# #
#### LOAD DATA AND APPLY ANY TRANSFORMATIONS ####
# ################################################################# #

database = read.csv("data/nz_wide_choice.csv",header=TRUE)

### basic data cleaning
database = subset(database,database$w1_dur==1)

### for use in prediction later
database$avail_free_A=1
database$avail_paid_A=1
database$avail_free_B=1
database$avail_paid_B=1
database$avail_no_vacc=1

# ################################################################# #
#### DEFINE MODEL PARAMETERS ####
# ################################################################# #

### Vector of parameters, including any that are kept fixed in estimation
apollo_beta=c(c1_asc_free=0.2,
c2_asc_free=0,
c3_asc_free=0,
c1_asc_paid=0,
c2_asc_paid=0,
c3_asc_paid=0,
asc_no_vacc=0,
c1_risk_infection=0,
c2_risk_infection=0,
c3_risk_infection=0,
c1_risk_illness=0,
c2_risk_illness=0,
c3_risk_illness=0,
c1_protection_unknown=0,
c2_protection_unknown=0,
c3_protection_unknown=0,
c1_protection=0,
c2_protection=0,
c3_protection=0,
c1_mild=0,
c2_mild=0,
c3_mild=0,
c1_severe=0,
c2_severe=0,
c3_severe=0,
c1_wait=0,
c2_wait=0,
c3_wait=0,
c1_fee=0,
c2_fee=0,
c3_fee=0,
c1_pop_coverage=0,
c2_pop_coverage=0,
c3_pop_coverage=0,
c1_exempt=0,
c2_exempt=0,
c3_exempt=0,
c1_lambda_vacc=1,
c2_lambda_vacc=0.5,
c3_lambda_vacc=0.5,
c1_delta=0,
c2_delta=-1.4,
c3_delta=0
)

apollo_scaling=c(c1_severe=10,
c2_severe=10,
c3_severe=10)

### Vector with names (in quotes) of parameters to be kept fixed at their starting value in apollo_beta, use apollo_beta_fixed = c() if none
apollo_fixed = c("asc_no_vacc","c1_delta","c1_lambda_vacc")

################################################################### #
#### DEFINE LATENT CLASS COMPONENTS ####
# ################################################################# #

apollo_lcPars=function(apollo_beta, apollo_inputs){

lcpars = list()

#lcpars[["asc_position"]]=list(c1_asc_position,c2_asc_position,c3_asc_position)
lcpars[["asc_free"]]=list(c1_asc_free,c2_asc_free,c3_asc_free)
lcpars[["asc_paid"]]=list(c1_asc_paid,c2_asc_paid,c3_asc_paid)
lcpars[["risk_infection"]]=list(c1_risk_infection,c2_risk_infection,c3_risk_infection)
lcpars[["risk_illness"]]=list(c1_risk_illness,c2_risk_illness,c3_risk_illness)
lcpars[["protection_unknown"]]=list(c1_protection_unknown,c2_protection_unknown,c3_protection_unknown)
lcpars[["protection"]]=list(c1_protection,c2_protection,c3_protection)
lcpars[["mild"]]=list(c1_mild,c2_mild,c3_mild)
lcpars[["severe"]]=list(c1_severe,c2_severe,c3_severe)
lcpars[["wait"]]=list(c1_wait,c2_wait,c3_wait)
lcpars[["fee"]]=list(c1_fee,c2_fee,c3_fee)
lcpars[["pop_coverage"]]=list(c1_pop_coverage,c2_pop_coverage,c3_pop_coverage)
lcpars[["exempt"]]=list(c1_exempt,c2_exempt,c3_exempt)
lcpars[["lambda_vacc"]]=list(c1_lambda_vacc,c2_lambda_vacc,c3_lambda_vacc)

### Utilities for class allocation model: class defined without covariates
V=list()
V[["class_a"]] = c1_delta
V[["class_b"]] = c2_delta
V[["class_c"]] = c3_delta

### Settings for class allocation models
mnl_settings = list(
alternatives = c(class_a=1, class_b=2, class_c=3),
avail = 1,
choiceVar = NA,
V = V
)

lcpars[["pi_values"]] = apollo_mnl(mnl_settings, functionality="raw")

lcpars[["pi_values"]] = apollo_firstRow(lcpars[["pi_values"]], apollo_inputs)

return(lcpars)
}


# ################################################################# #
#### GROUP AND VALIDATE INPUTS ####
# ################################################################# #

apollo_inputs = apollo_validateInputs()

# ################################################################# #
#### DEFINE MODEL AND LIKELIHOOD FUNCTION ####
# ################################################################# #

apollo_probabilities=function(apollo_beta, apollo_inputs, functionality="estimate"){

### Attach inputs and detach after function exit
apollo_attach(apollo_beta, apollo_inputs)
on.exit(apollo_detach(apollo_beta, apollo_inputs))


### Create list of probabilities P
P = list()

### Loop over classes
for(s in 1:length(pi_values)){

## create parameters for use inside utilities
asc_1 = asc_free[[s]]
asc_2 = asc_paid[[s]]
asc_3 = asc_free[[s]]
asc_4 = asc_paid[[s]]
asc_5 = asc_no_vacc
b_risk_infection = risk_infection[[s]]
b_risk_illness = risk_illness[[s]]
b_protection_unknown = protection_unknown[[s]]
b_protection = protection[[s]]
b_mild = mild[[s]]
b_severe = severe[[s]]
b_wait = wait[[s]]
b_fee = fee[[s]]
b_exempt = exempt[[s]]
b_pop_coverage = pop_coverage[[s]]

### Compute class-specific utilities
V = list()
V[['free_A']] = ( asc_1 + b_risk_infection * infection_riska + b_risk_illness * illness_riska + b_protection_unknown * ( protection_dura == 0 ) + b_protection * protection_dura + b_mild * mild_riska + b_severe * severe_riska + b_wait * waita + b_pop_coverage * share + b_exempt * ( right_ab == 2) )
V[['paid_A']] = ( asc_2 + b_risk_infection * infection_riska + b_risk_illness * illness_riska + b_protection_unknown * ( protection_dura == 0 ) + b_protection * protection_dura + b_mild * mild_riska + b_severe * severe_riska + b_fee * costa + b_pop_coverage * share + b_exempt * ( right_ab == 2) )
V[['free_B']] = ( asc_3 + b_risk_infection * infection_riskb + b_risk_illness * illness_riskb + b_protection_unknown * ( protection_durb == 0 ) + b_protection * protection_durb + b_mild * mild_riskb + b_severe * severe_riskb + b_wait * waitb + b_pop_coverage * share + b_exempt * ( right_ab == 2) )
V[['paid_B']] = ( asc_4 + b_risk_infection * infection_riskb + b_risk_illness * illness_riskb + b_protection_unknown * ( protection_durb == 0 ) + b_protection * protection_durb + b_mild * mild_riskb + b_severe * severe_riskb + b_fee * costb + b_pop_coverage * share + b_exempt * ( right_ab == 2) )
V[['no_vacc']] = ( asc_5 + b_risk_infection * infection_risk_novac + b_risk_illness * illness_risk_novac )

### Specify nests for NL model
nlNests = list(root=1, vacc=lambda_vacc[[s]])

### Specify tree structure for NL model
nlStructure= list()
nlStructure[["root"]] = c("vacc","no_vacc")
nlStructure[["vacc"]] = c("free_A","paid_A","free_B","paid_B")

### Define settings for NL model component
nl_settings = list(
alternatives = c(free_A=1, paid_A=2, free_B=3, paid_B=4, no_vacc = 5),
avail = list(free_A=avail_free_A,paid_A=avail_paid_A,free_B=avail_free_B,paid_B=avail_paid_B,no_vacc=avail_no_vacc),
choiceVar = choice,
nlNests = nlNests,
nlStructure = nlStructure,
V = V
)

### Compute within-class choice probabilities using MNL model
P[[paste0("Class_",s)]] = apollo_nl(nl_settings, functionality)

### Take product across observation for same individual
P[[paste0("Class_",s)]] = apollo_panelProd(P[[paste0("Class_",s)]], apollo_inputs ,functionality)
}

### Compute latent class model probabilities
lc_settings = list(inClassProb = P, classProb=pi_values)
P[["model"]] = apollo_lc(lc_settings, apollo_inputs, functionality)

### Prepare and return outputs of function
P = apollo_prepareProb(P, apollo_inputs, functionality)
return(P)
}

# ################################################################# #
#### MODEL ESTIMATION ####
# ################################################################# #

model=apollo_estimate(apollo_beta, apollo_fixed, apollo_probabilities,apollo_inputs)

# ----------------------------------------------------------------- #
#---- FORMATTED OUTPUT TO SCREEN ----
# ----------------------------------------------------------------- #

apollo_modelOutput(model,modelOutput_settings = list(printT1=TRUE,printClassical=FALSE,printPVal=2))

# ----------------------------------------------------------------- #
#---- FORMATTED OUTPUT (TO FILE, using model name) ----
# ----------------------------------------------------------------- #

apollo_saveOutput(model,saveOutput_settings = list(printT1=TRUE,printClassical=FALSE,printPVal=2))




______________________________________________________________________________________________________________________
Results


Model name : LC_NL_3
Model description : LC-NL, 3 classes, no socios
Model run at : 2022-08-23 14:34:13
Estimation method : bfgs
Model diagnosis : successful convergence
Number of individuals : 710
Number of rows in database : 9018
Number of modelled outcomes : 9018

Number of cores used : 4
Model without mixing

LL(start) : -14186.24
LL(0, whole model) : -14513.91
LL(C, whole model) : -12937.23
LL(final, whole model) : -10349.29
Rho-square (0) : 0.2869
Adj.Rho-square (0) : 0.2842
Rho-square (C) : 0.2
Adj.Rho-square (C) : 0.1969
AIC : 20778.59
BIC : 21062.87

LL(0,Class_1) : -14513.91
LL(final,Class_1) : -Inf Likelihood equal to zero for at least
one individual in this component.
LL(0,Class_2) : -14513.91
LL(final,Class_2) : -Inf Likelihood equal to zero for at least
one individual in this component.
LL(0,Class_3) : -14513.91
LL(final,Class_3) : -Inf Likelihood equal to zero for at least
one individual in this component.

Estimated parameters : 40
Time taken (hh:mm:ss) : 00:09:58.04
pre-estimation : 00:00:22.02
estimation : 00:06:18.75
post-estimation : 00:03:17.28
Iterations : 123
Min abs eigenvalue of Hessian : 0.036791

Unconstrained optimisation.

Estimates:
Estimate Rob.s.e. Rob.t.rat.(0) p(2-sided) Rob.t.rat.(1) p(2-sided)
c1_asc_free 1.056163 0.304715 3.46606 5.2814e-04 0.1843 0.853767
c2_asc_free 1.416643 0.381747 3.71095 2.0648e-04 1.0914 0.275091
c3_asc_free -0.720113 0.229110 -3.14309 0.001672 -7.5078 6.017e-14
c1_asc_paid -0.106558 0.340000 -0.31341 0.753973 -3.2546 0.001136
c2_asc_paid 1.591814 0.379695 4.19235 2.761e-05 1.5587 0.119078
c3_asc_paid -1.139175 0.348406 -3.26968 0.001077 -6.1399 8.257e-10
asc_no_vacc 0.000000 NA NA NA NA NA
c1_risk_infection -0.145533 0.022140 -6.57345 4.916e-11 -51.7415 0.000000
c2_risk_infection -0.075226 0.025583 -2.94045 0.003277 -42.0287 0.000000
c3_risk_infection -0.015737 0.014281 -1.10197 0.270477 -71.1248 0.000000
c1_risk_illness -0.106541 0.012297 -8.66434 0.000000 -89.9883 0.000000
c2_risk_illness -0.057245 0.016897 -3.38782 7.0451e-04 -62.5687 0.000000
c3_risk_illness -0.014568 0.009326 -1.56205 0.118277 -108.7875 0.000000
c1_protection_unknown -0.080585 0.085260 -0.94516 0.344575 -12.6740 0.000000
c2_protection_unknown -0.039841 0.081520 -0.48872 0.625037 -12.7557 0.000000
c3_protection_unknown -0.053126 0.056341 -0.94294 0.345712 -18.6921 0.000000
c1_protection 0.020262 0.001756 11.53925 0.000000 -557.9710 0.000000
c2_protection 0.015024 0.003967 3.78725 1.5233e-04 -248.2846 0.000000
c3_protection 0.001856 0.001255 1.47880 0.139195 -795.4071 0.000000
c1_mild -0.059523 0.009207 -6.46462 1.016e-10 -115.0722 0.000000
c2_mild -0.029460 0.011091 -2.65626 0.007901 -92.8209 0.000000
c3_mild -0.007482 0.006000 -1.24715 0.212344 -167.9246 0.000000
c1_severe -32.185523 5.724805 -5.62212 1.886e-08 -5.7968 6.759e-09
c2_severe -13.238613 5.072748 -2.60975 0.009061 -2.8069 0.005002
c3_severe -8.826133 4.329786 -2.03847 0.041503 -2.2694 0.023242
c1_wait -0.024799 0.005105 -4.85809 1.185e-06 -200.7559 0.000000
c2_wait -0.031486 0.010386 -3.03162 0.002432 -99.3170 0.000000
c3_wait -0.006484 0.003244 -1.99884 0.045626 -310.2819 0.000000
c1_fee -0.008335 0.001793 -4.65002 3.319e-06 -562.5150 0.000000
c2_fee -0.001945 5.7061e-04 -3.40820 6.5393e-04 -1755.9332 0.000000
c3_fee -7.1913e-04 4.0436e-04 -1.77841 0.075336 -2474.8003 0.000000
c1_pop_coverage 0.015168 0.009448 1.60539 0.108408 -104.2359 0.000000
c2_pop_coverage 0.018249 0.011583 1.57551 0.115138 -84.7603 0.000000
c3_pop_coverage 1.5964e-04 0.002241 0.07123 0.943212 -446.1478 0.000000
c1_exempt -0.928788 0.558776 -1.66218 0.096476 -3.4518 5.5684e-04
c2_exempt -0.823396 0.711477 -1.15730 0.247148 -2.5628 0.010382
c3_exempt 0.160198 0.160443 0.99847 0.318050 -5.2343 1.656e-07
c1_lambda_vacc 1.000000 NA NA NA NA NA
c2_lambda_vacc 0.670958 0.186255 3.60237 3.1533e-04 -1.7666 0.077292
c3_lambda_vacc 0.213181 0.099533 2.14180 0.032209 -7.9051 2.665e-15
c1_delta 0.000000 NA NA NA NA NA
c2_delta -0.549145 0.150002 -3.66091 2.5132e-04 -10.3275 0.000000
c3_delta -1.471583 0.134614 -10.93186 0.000000 -18.3605 0.000000

Nesting structure for NL model component Class_1:
Nest: root (1)
|-Nest: vacc (1)
| |-Alternative: free_A
| |-Alternative: paid_A
| |-Alternative: free_B
| '-Alternative: paid_B
'----Alternative: no_vacc

Nesting structure for NL model component Class_2:
Nest: root (1)
|-Nest: vacc (0.9836)
| |-Alternative: free_A
| |-Alternative: paid_A
| |-Alternative: free_B
| '-Alternative: paid_B
'----Alternative: no_vacc

Nesting structure for NL model component Class_3:
Nest: root (1)
|-Nest: vacc (0.9931)
| |-Alternative: free_A
| |-Alternative: paid_A
| |-Alternative: free_B
| '-Alternative: paid_B
'----Alternative: no_vacc


Summary of class allocation for LC model component :
Mean prob.
Class_1 0.5761
Class_2 0.2119
Class_3 0.2120
stephanehess
Site Admin
Posts: 974
Joined: 24 Apr 2020, 16:29

Re: Error with class convergence and lambda values for latent class nested logit model

Post by stephanehess »

Robbie

the first point is not a problem. With a latent class model, it's quite possible that for some individuals, the likelihood is zero in some of the classes. As long as it's non-zero for at least one class per individual, your model is still going to be fine as it's a weighted average across classes.

The second point however seems like a bug. Could you please share your files with me outside the forum so I can look into this

Thanks

Stephane
--------------------------------
Stephane Hess
www.stephanehess.me.uk
stephanehess
Site Admin
Posts: 974
Joined: 24 Apr 2020, 16:29

Re: Error with class convergence and lambda values for latent class nested logit model

Post by stephanehess »

Hi Robbie

thanks for sending me the data. I ran the model on my machine using the newest version of Apollo, and it gives the correct results. Can you please update Apollo. You must have an older version as the class allocation probabilities you get in the output are also not correct.

Also, please look at the new LC examples and the use of the function apollo_classAlloc which will simplify your code in apollo_lcPars

Stephane
--------------------------------
Stephane Hess
www.stephanehess.me.uk
robbie_research
Posts: 2
Joined: 25 Aug 2022, 23:00

Re: Error with class convergence and lambda values for latent class nested logit model

Post by robbie_research »

Thank you Stephane - your help is much appreciated.

I can confirm that using the latest version of Apollo (0.2.8 vs 0.2.7) has resolved all of the issues described!

As an FYI for anyone else, when running the command install.packages("Apollo"), my machine was unable to update to version 0.2.8 from binaries (and just kept using version 0.2.7 for some reason). I ended up using Rtools4 to install from source rather than binaries, and that was successful. Hence, if anyone else is struggling to get the latest version installed, building from source may be a good option! In saying that, the Apollo website recommends installing using binaries (http://www.apollochoicemodelling.com/code.html) so maybe building from source is a good backup option if needed!

Thanks again Stephane :)

Kind regards,

Robbie
stephanehess
Site Admin
Posts: 974
Joined: 24 Apr 2020, 16:29

Re: Error with class convergence and lambda values for latent class nested logit model

Post by stephanehess »

Robbie

sometimes, it helps to delete the old version before installing

Stephane
--------------------------------
Stephane Hess
www.stephanehess.me.uk
Post Reply