ApolloChoiceModelling forum

Hi Stephane and the Apollo Choice Modelling Community,

I am trying to run a latent class nested logit model (with three classes and five alternatives) and am running into some puzzling issues around the latent class estimation.

Firstly, the latent class likelihoods are not converging, despite the overall model converging and Apollo providing estimates with standard errors. I'm not sure how much of an issue this is and I am struggling to fix it. The error message is: "-Inf Likelihood equal to zero for at least one individual in this component."

Secondly, and perhaps relatedly, the lambda latent class estimates do not correspond to the nesting parameters that Apollo shows after the main results. The lambda estimate for class 2 will read something like 0.6 whereas the nesting parameter (in the nesting structure results) will read ~0.98. I suspect that solving one issue will solve the other also, but I'm not sure.

Any thoughts or insights into these issues would be immensely appreciated. I have pasted the full code followed by the results below.

Thanks in advance for your help

____________________________________________________________________________________________
Code

# ################################################################# #

### Clear memory
rm(list = ls())

### Load Apollo library
library(apollo)
### Initialise code
apollo_initialise()

### Set core controls
apollo_control = list(
modelName ="LC_NL_3",
modelDescr ="LC-NL, 3 classes, no socios",
indivID ="id",
outputDirectory = "tables",
nCores = 4
)

# ################################################################# #
#### LOAD DATA AND APPLY ANY TRANSFORMATIONS ####
# ################################################################# #

database = read.csv("data/nz_wide_choice.csv",header=TRUE)

### basic data cleaning
database = subset(database,database$w1_dur==1)

### for use in prediction later
database$avail_free_A=1
database$avail_paid_A=1
database$avail_free_B=1
database$avail_paid_B=1
database$avail_no_vacc=1

# ################################################################# #
#### DEFINE MODEL PARAMETERS ####
# ################################################################# #

### Vector of parameters, including any that are kept fixed in estimation
apollo_beta=c(c1_asc_free=0.2,
c2_asc_free=0,
c3_asc_free=0,
c1_asc_paid=0,
c2_asc_paid=0,
c3_asc_paid=0,
asc_no_vacc=0,
c1_risk_infection=0,
c2_risk_infection=0,
c3_risk_infection=0,
c1_risk_illness=0,
c2_risk_illness=0,
c3_risk_illness=0,
c1_protection_unknown=0,
c2_protection_unknown=0,
c3_protection_unknown=0,
c1_protection=0,
c2_protection=0,
c3_protection=0,
c1_mild=0,
c2_mild=0,
c3_mild=0,
c1_severe=0,
c2_severe=0,
c3_severe=0,
c1_wait=0,
c2_wait=0,
c3_wait=0,
c1_fee=0,
c2_fee=0,
c3_fee=0,
c1_pop_coverage=0,
c2_pop_coverage=0,
c3_pop_coverage=0,
c1_exempt=0,
c2_exempt=0,
c3_exempt=0,
c1_lambda_vacc=1,
c2_lambda_vacc=0.5,
c3_lambda_vacc=0.5,
c1_delta=0,
c2_delta=-1.4,
c3_delta=0
)

apollo_scaling=c(c1_severe=10,
c2_severe=10,
c3_severe=10)

### Vector with names (in quotes) of parameters to be kept fixed at their starting value in apollo_beta, use apollo_beta_fixed = c() if none
apollo_fixed = c("asc_no_vacc","c1_delta","c1_lambda_vacc")

################################################################### #
#### DEFINE LATENT CLASS COMPONENTS ####
# ################################################################# #

apollo_lcPars=function(apollo_beta, apollo_inputs){

lcpars = list()

#lcpars[["asc_position"]]=list(c1_asc_position,c2_asc_position,c3_asc_position)
lcpars[["asc_free"]]=list(c1_asc_free,c2_asc_free,c3_asc_free)
lcpars[["asc_paid"]]=list(c1_asc_paid,c2_asc_paid,c3_asc_paid)
lcpars[["risk_infection"]]=list(c1_risk_infection,c2_risk_infection,c3_risk_infection)
lcpars[["risk_illness"]]=list(c1_risk_illness,c2_risk_illness,c3_risk_illness)
lcpars[["protection_unknown"]]=list(c1_protection_unknown,c2_protection_unknown,c3_protection_unknown)
lcpars[["protection"]]=list(c1_protection,c2_protection,c3_protection)
lcpars[["mild"]]=list(c1_mild,c2_mild,c3_mild)
lcpars[["severe"]]=list(c1_severe,c2_severe,c3_severe)
lcpars[["wait"]]=list(c1_wait,c2_wait,c3_wait)
lcpars[["fee"]]=list(c1_fee,c2_fee,c3_fee)
lcpars[["pop_coverage"]]=list(c1_pop_coverage,c2_pop_coverage,c3_pop_coverage)
lcpars[["exempt"]]=list(c1_exempt,c2_exempt,c3_exempt)
lcpars[["lambda_vacc"]]=list(c1_lambda_vacc,c2_lambda_vacc,c3_lambda_vacc)

### Utilities for class allocation model: class defined without covariates
V=list()
V[["class_a"]] = c1_delta
V[["class_b"]] = c2_delta
V[["class_c"]] = c3_delta

### Settings for class allocation models
mnl_settings = list(
alternatives = c(class_a=1, class_b=2, class_c=3),
avail = 1,
choiceVar = NA,
V = V
)

lcpars[["pi_values"]] = apollo_mnl(mnl_settings, functionality="raw")

lcpars[["pi_values"]] = apollo_firstRow(lcpars[["pi_values"]], apollo_inputs)

return(lcpars)
}


# ################################################################# #
#### GROUP AND VALIDATE INPUTS ####
# ################################################################# #

apollo_inputs = apollo_validateInputs()

# ################################################################# #
#### DEFINE MODEL AND LIKELIHOOD FUNCTION ####
# ################################################################# #

apollo_probabilities=function(apollo_beta, apollo_inputs, functionality="estimate"){

### Attach inputs and detach after function exit
apollo_attach(apollo_beta, apollo_inputs)
on.exit(apollo_detach(apollo_beta, apollo_inputs))


### Create list of probabilities P
P = list()

### Loop over classes
for(s in 1:length(pi_values)){

## create parameters for use inside utilities
asc_1 = asc_free[[s]]
asc_2 = asc_paid[[s]]
asc_3 = asc_free[[s]]
asc_4 = asc_paid[[s]]
asc_5 = asc_no_vacc
b_risk_infection = risk_infection[[s]]
b_risk_illness = risk_illness[[s]]
b_protection_unknown = protection_unknown[[s]]
b_protection = protection[[s]]
b_mild = mild[[s]]
b_severe = severe[[s]]
b_wait = wait[[s]]
b_fee = fee[[s]]
b_exempt = exempt[[s]]
b_pop_coverage = pop_coverage[[s]]

### Compute class-specific utilities
V = list()
V[['free_A']] = ( asc_1 + b_risk_infection * infection_riska + b_risk_illness * illness_riska + b_protection_unknown * ( protection_dura == 0 ) + b_protection * protection_dura + b_mild * mild_riska + b_severe * severe_riska + b_wait * waita + b_pop_coverage * share + b_exempt * ( right_ab == 2) )
V[['paid_A']] = ( asc_2 + b_risk_infection * infection_riska + b_risk_illness * illness_riska + b_protection_unknown * ( protection_dura == 0 ) + b_protection * protection_dura + b_mild * mild_riska + b_severe * severe_riska + b_fee * costa + b_pop_coverage * share + b_exempt * ( right_ab == 2) )
V[['free_B']] = ( asc_3 + b_risk_infection * infection_riskb + b_risk_illness * illness_riskb + b_protection_unknown * ( protection_durb == 0 ) + b_protection * protection_durb + b_mild * mild_riskb + b_severe * severe_riskb + b_wait * waitb + b_pop_coverage * share + b_exempt * ( right_ab == 2) )
V[['paid_B']] = ( asc_4 + b_risk_infection * infection_riskb + b_risk_illness * illness_riskb + b_protection_unknown * ( protection_durb == 0 ) + b_protection * protection_durb + b_mild * mild_riskb + b_severe * severe_riskb + b_fee * costb + b_pop_coverage * share + b_exempt * ( right_ab == 2) )
V[['no_vacc']] = ( asc_5 + b_risk_infection * infection_risk_novac + b_risk_illness * illness_risk_novac )

### Specify nests for NL model
nlNests = list(root=1, vacc=lambda_vacc[[s]])

### Specify tree structure for NL model
nlStructure= list()
nlStructure[["root"]] = c("vacc","no_vacc")
nlStructure[["vacc"]] = c("free_A","paid_A","free_B","paid_B")

### Define settings for NL model component
nl_settings = list(
alternatives = c(free_A=1, paid_A=2, free_B=3, paid_B=4, no_vacc = 5),
avail = list(free_A=avail_free_A,paid_A=avail_paid_A,free_B=avail_free_B,paid_B=avail_paid_B,no_vacc=avail_no_vacc),
choiceVar = choice,
nlNests = nlNests,
nlStructure = nlStructure,
V = V
)

### Compute within-class choice probabilities using MNL model
P[[paste0("Class_",s)]] = apollo_nl(nl_settings, functionality)

### Take product across observation for same individual
P[[paste0("Class_",s)]] = apollo_panelProd(P[[paste0("Class_",s)]], apollo_inputs ,functionality)
}

### Compute latent class model probabilities
lc_settings = list(inClassProb = P, classProb=pi_values)
P[["model"]] = apollo_lc(lc_settings, apollo_inputs, functionality)

### Prepare and return outputs of function
P = apollo_prepareProb(P, apollo_inputs, functionality)
return(P)
}

# ################################################################# #
#### MODEL ESTIMATION ####
# ################################################################# #

model=apollo_estimate(apollo_beta, apollo_fixed, apollo_probabilities,apollo_inputs)

# ----------------------------------------------------------------- #
#---- FORMATTED OUTPUT TO SCREEN ----
# ----------------------------------------------------------------- #

apollo_modelOutput(model,modelOutput_settings = list(printT1=TRUE,printClassical=FALSE,printPVal=2))

# ----------------------------------------------------------------- #
#---- FORMATTED OUTPUT (TO FILE, using model name) ----
# ----------------------------------------------------------------- #

apollo_saveOutput(model,saveOutput_settings = list(printT1=TRUE,printClassical=FALSE,printPVal=2))




______________________________________________________________________________________________________________________
Results


Model name : LC_NL_3
Model description : LC-NL, 3 classes, no socios
Model run at : 2022-08-23 14:34:13
Estimation method : bfgs
Model diagnosis : successful convergence
Number of individuals : 710
Number of rows in database : 9018
Number of modelled outcomes : 9018

Number of cores used : 4
Model without mixing

LL(start) : -14186.24
LL(0, whole model) : -14513.91
LL(C, whole model) : -12937.23
LL(final, whole model) : -10349.29
Rho-square (0) : 0.2869
Adj.Rho-square (0) : 0.2842
Rho-square (C) : 0.2
Adj.Rho-square (C) : 0.1969
AIC : 20778.59
BIC : 21062.87

LL(0,Class_1) : -14513.91
LL(final,Class_1) : -Inf Likelihood equal to zero for at least
one individual in this component.
LL(0,Class_2) : -14513.91
LL(final,Class_2) : -Inf Likelihood equal to zero for at least
one individual in this component.
LL(0,Class_3) : -14513.91
LL(final,Class_3) : -Inf Likelihood equal to zero for at least
one individual in this component.

Estimated parameters : 40
Time taken (hh:mm:ss) : 00:09:58.04
pre-estimation : 00:00:22.02
estimation : 00:06:18.75
post-estimation : 00:03:17.28
Iterations : 123
Min abs eigenvalue of Hessian : 0.036791

Unconstrained optimisation.

Estimates:
Estimate Rob.s.e. Rob.t.rat.(0) p(2-sided) Rob.t.rat.(1) p(2-sided)
c1_asc_free 1.056163 0.304715 3.46606 5.2814e-04 0.1843 0.853767
c2_asc_free 1.416643 0.381747 3.71095 2.0648e-04 1.0914 0.275091
c3_asc_free -0.720113 0.229110 -3.14309 0.001672 -7.5078 6.017e-14
c1_asc_paid -0.106558 0.340000 -0.31341 0.753973 -3.2546 0.001136
c2_asc_paid 1.591814 0.379695 4.19235 2.761e-05 1.5587 0.119078
c3_asc_paid -1.139175 0.348406 -3.26968 0.001077 -6.1399 8.257e-10
asc_no_vacc 0.000000 NA NA NA NA NA
c1_risk_infection -0.145533 0.022140 -6.57345 4.916e-11 -51.7415 0.000000
c2_risk_infection -0.075226 0.025583 -2.94045 0.003277 -42.0287 0.000000
c3_risk_infection -0.015737 0.014281 -1.10197 0.270477 -71.1248 0.000000
c1_risk_illness -0.106541 0.012297 -8.66434 0.000000 -89.9883 0.000000
c2_risk_illness -0.057245 0.016897 -3.38782 7.0451e-04 -62.5687 0.000000
c3_risk_illness -0.014568 0.009326 -1.56205 0.118277 -108.7875 0.000000
c1_protection_unknown -0.080585 0.085260 -0.94516 0.344575 -12.6740 0.000000
c2_protection_unknown -0.039841 0.081520 -0.48872 0.625037 -12.7557 0.000000
c3_protection_unknown -0.053126 0.056341 -0.94294 0.345712 -18.6921 0.000000
c1_protection 0.020262 0.001756 11.53925 0.000000 -557.9710 0.000000
c2_protection 0.015024 0.003967 3.78725 1.5233e-04 -248.2846 0.000000
c3_protection 0.001856 0.001255 1.47880 0.139195 -795.4071 0.000000
c1_mild -0.059523 0.009207 -6.46462 1.016e-10 -115.0722 0.000000
c2_mild -0.029460 0.011091 -2.65626 0.007901 -92.8209 0.000000
c3_mild -0.007482 0.006000 -1.24715 0.212344 -167.9246 0.000000
c1_severe -32.185523 5.724805 -5.62212 1.886e-08 -5.7968 6.759e-09
c2_severe -13.238613 5.072748 -2.60975 0.009061 -2.8069 0.005002
c3_severe -8.826133 4.329786 -2.03847 0.041503 -2.2694 0.023242
c1_wait -0.024799 0.005105 -4.85809 1.185e-06 -200.7559 0.000000
c2_wait -0.031486 0.010386 -3.03162 0.002432 -99.3170 0.000000
c3_wait -0.006484 0.003244 -1.99884 0.045626 -310.2819 0.000000
c1_fee -0.008335 0.001793 -4.65002 3.319e-06 -562.5150 0.000000
c2_fee -0.001945 5.7061e-04 -3.40820 6.5393e-04 -1755.9332 0.000000
c3_fee -7.1913e-04 4.0436e-04 -1.77841 0.075336 -2474.8003 0.000000
c1_pop_coverage 0.015168 0.009448 1.60539 0.108408 -104.2359 0.000000
c2_pop_coverage 0.018249 0.011583 1.57551 0.115138 -84.7603 0.000000
c3_pop_coverage 1.5964e-04 0.002241 0.07123 0.943212 -446.1478 0.000000
c1_exempt -0.928788 0.558776 -1.66218 0.096476 -3.4518 5.5684e-04
c2_exempt -0.823396 0.711477 -1.15730 0.247148 -2.5628 0.010382
c3_exempt 0.160198 0.160443 0.99847 0.318050 -5.2343 1.656e-07
c1_lambda_vacc 1.000000 NA NA NA NA NA
c2_lambda_vacc 0.670958 0.186255 3.60237 3.1533e-04 -1.7666 0.077292
c3_lambda_vacc 0.213181 0.099533 2.14180 0.032209 -7.9051 2.665e-15
c1_delta 0.000000 NA NA NA NA NA
c2_delta -0.549145 0.150002 -3.66091 2.5132e-04 -10.3275 0.000000
c3_delta -1.471583 0.134614 -10.93186 0.000000 -18.3605 0.000000

Nesting structure for NL model component Class_1:
Nest: root (1)
|-Nest: vacc (1)
| |-Alternative: free_A
| |-Alternative: paid_A
| |-Alternative: free_B
| '-Alternative: paid_B
'----Alternative: no_vacc

Nesting structure for NL model component Class_2:
Nest: root (1)
|-Nest: vacc (0.9836)
| |-Alternative: free_A
| |-Alternative: paid_A
| |-Alternative: free_B
| '-Alternative: paid_B
'----Alternative: no_vacc

Nesting structure for NL model component Class_3:
Nest: root (1)
|-Nest: vacc (0.9931)
| |-Alternative: free_A
| |-Alternative: paid_A
| |-Alternative: free_B
| '-Alternative: paid_B
'----Alternative: no_vacc


Summary of class allocation for LC model component :
Mean prob.
Class_1 0.5761
Class_2 0.2119
Class_3 0.2120

Robbie

the first point is not a problem. With a latent class model, it's quite possible that for some individuals, the likelihood is zero in some of the classes. As long as it's non-zero for at least one class per individual, your model is still going to be fine as it's a weighted average across classes.

The second point however seems like a bug. Could you please share your files with me outside the forum so I can look into this

Thanks

Stephane

Hi Robbie

thanks for sending me the data. I ran the model on my machine using the newest version of Apollo, and it gives the correct results. Can you please update Apollo. You must have an older version as the class allocation probabilities you get in the output are also not correct.

Also, please look at the new LC examples and the use of the function apollo_classAlloc which will simplify your code in apollo_lcPars

Stephane

Thank you Stephane - your help is much appreciated.

I can confirm that using the latest version of Apollo (0.2.8 vs 0.2.7) has resolved all of the issues described!

As an FYI for anyone else, when running the command install.packages("Apollo"), my machine was unable to update to version 0.2.8 from binaries (and just kept using version 0.2.7 for some reason). I ended up using Rtools4 to install from source rather than binaries, and that was successful. Hence, if anyone else is struggling to get the latest version installed, building from source may be a good option! In saying that, the Apollo website recommends installing using binaries (http://www.apollochoicemodelling.com/code.html) so maybe building from source is a good backup option if needed!

Thanks again Stephane

Kind regards,

Robbie

Robbie

sometimes, it helps to delete the old version before installing

Stephane