Rho squares and BIC NA for Latent class dual response model

[Iris123]

Hi,

I am fitting a latent class model on DCE data with dual response using the apollo_combineModels function (one model with forced choice and one model with additional opt-out option). When I run the model, the output returns "not applicable" for the Rho-squares and the "NA" for the BIC. The output does return values for the LL and AIC. When I run both models separately, the Rho-squares and BIC are estimated.

Below is the code I used and some of the output. Can you help to figure out why the Rho-squares and BIC are not applicable?

Thank you in advance!

Iris

############################### 3 classes ######################################
rm(list = ls())
gc()

#load package
library(apollo)

#detach variables
apollo_initialise()

#Set core controls
apollo_control = list(
modelName = "LCA_DCE_dualresponse_3classes_syn",
modelDescr = "LCA model with dual response 3 classes synthetic data",
indivID = "ID",
nCores = 4,
outputDirectory = "Resultaten"
)

database <- readRDS("Data/database.rds") %>% arrange(ID)

#Define parameters and starting values
apollo_beta <- c(
asc_1 = 0,
asc_2 = 0,
asc_3 = 0,
beta_benefits_a = 0,
beta_benefits_b = 0,
beta_benefits_c = 0,
beta_equal_a = 0,
beta_equal_b = 0,
beta_equal_c = 0,
beta_high_a = 0,
beta_high_b = 0,
beta_high_c = 0,
beta_low_a = 0,
beta_low_b = 0,
beta_low_c = 0,
beta_cost_a = 0,
beta_cost_b = 0,
beta_cost_c = 0,
beta_type_a = 0,
beta_type_b = 0,
beta_type_c = 0,
delta_a = 0,
delta_b = 0,
delta_c = 0,
mu_forced = 1,
mu_optout = 1
)

#Define which parameters to keep fixed at starting values
#Set beta_equal as reference for the categorical income attribute (consisting of beta_equal, beta_low, beta_high)
apollo_fixed = c("beta_equal_a", "beta_equal_b", "beta_equal_c",
"delta_a", "asc_1", "mu_forced")

#Define function apollo_lcPars (used by the LCA model)
apollo_lcPars = function(apollo_beta, apollo_inputs) {
lcpars = list()
lcpars[["beta_benefits"]] = list(beta_benefits_a, beta_benefits_b, beta_benefits_c)
lcpars[["beta_cost"]] = list(beta_cost_a, beta_cost_b, beta_cost_c)
lcpars[["beta_type"]] = list(beta_type_a, beta_type_b, beta_type_c)
lcpars[["beta_equal"]] = list(beta_equal_a, beta_equal_b, beta_equal_c)
lcpars[["beta_low"]] = list(beta_low_a, beta_low_b, beta_low_c)
lcpars[["beta_high"]] = list(beta_high_a, beta_high_b,beta_high_c)

#Utilities of class allocation model
V = list()
V[["class_a"]] = delta_a
V[["class_b"]] = delta_b
V[["class_c"]] = delta_c

#Settings for class allocation models
classAlloc_settings = list(
classes = c(class_a = 1, class_b = 2, class_c = 3),
utilities = V
)
lcpars[["pi_values"]] = apollo_classAlloc(classAlloc_settings)
return(lcpars)

}

apollo_inputs = apollo_validateInputs()

apollo_probabilities = function(apollo_beta, apollo_inputs, functionality = "estimate") {

### Attach inputs and detach after function exit
apollo_attach(apollo_beta, apollo_inputs)
on.exit(apollo_detach(apollo_beta, apollo_inputs))

### Create list of probabilities P
P = list()
P_temp = list()

### Loop over classes
for(s in 1:length(pi_values)){

### Compute class-specific utilities
V=list()
V[["alt1"]] = asc_1 + beta_benefits[[s]]*benefits_1 + beta_cost[[s]]*cost_1 + beta_type[[s]]*type_1 + beta_equal[[s]]*(income_1 == 0) + beta_high[[s]]*(income_1 == 1) + beta_low[[s]]*(income_1 == 2)
V[["alt2"]] = asc_2 + beta_benefits[[s]]*benefits_2 + beta_cost[[s]]*cost_2 + beta_type[[s]]*type_2 + beta_equal[[s]]*(income_2 == 0) + beta_high[[s]]*(income_2 == 1) + beta_low[[s]]*(income_2 == 2)
V[["alt3"]] = asc_3

### Define settings for MNL model component that are generic across classes (forced)
mnl_settings_forced = list(
alternatives = c(alt1=1, alt2=2),
avail = list(alt1=1, alt2=1),
choiceVar = choice,
utilities = list(
alt1 = mu_forced*V[["alt1"]],
alt2 = mu_forced*V[["alt2"]]),
rows = (forced == 1)
)

P_temp[[paste0("forced_class_",s)]] = apollo_mnl(mnl_settings_forced, functionality)

### Define settings for MNL model component that are generic across classes (optout)
mnl_settings_optout = list(
alternatives = c(alt1=1, alt2=2, alt3=3),
avail = list(alt1=1, alt2=1, alt3=1),
choiceVar = choice,
utilities = list(
alt1 = mu_optout*V[["alt1"]],
alt2 = mu_optout*V[["alt2"]],
alt3 = mu_optout*V[["alt3"]]),
rows = (optout == 1)
)

P_temp[[paste0("optout_class_",s)]] = apollo_mnl(mnl_settings_optout, functionality)

### Combined model
P[[paste0("Class_",s)]] = apollo_combineModels(
P_temp, apollo_inputs, functionality,
components=c(paste0("forced_class_",s),paste0("optout_class_",s)),asList=FALSE)


### Take product across observation for same individual
P[[paste0("Class_",s)]] = apollo_panelProd(P[[paste0("Class_",s)]], apollo_inputs ,functionality)
}

### Compute latent class model probabilities
lc_settings = list(inClassProb = P, classProb=pi_values)
P[["model"]] = apollo_lc(lc_settings, apollo_inputs, functionality)

### Prepare and return outputs of function
P = apollo_prepareProb(P, apollo_inputs, functionality)
return(P)
}

### Search starting values
apollo_beta=apollo_searchStart(apollo_beta, apollo_fixed,apollo_probabilities,
apollo_inputs,searchStart_settings = list(nCandidates=10))
### model estimation
model = apollo_estimate(apollo_beta, apollo_fixed, apollo_probabilities, apollo_inputs)

apollo_modelOutput(model)
apollo_saveOutput(model)

########################### OUTPUT ############################################


Model name : LCA_DCE_dualresponse_3classes
Model description : LCA model with dual response 3 classes
Model run at : 2024-08-08 10:43:03.882494
Estimation method : bgw
Model diagnosis : Relative function convergence
Optimisation diagnosis : Maximum found
hessian properties : Negative definite
maximum eigenvalue : -31.25857
reciprocal of condition number : 8.64217e-06
Number of individuals : 484
Number of rows in database : 11616
Number of modelled outcomes : 0

Number of cores used : 4
Model without mixing

LL(start) : -8564.26
LL (whole model) at equal shares, LL(0) : -10406.54
LL (whole model) at observed shares, LL(C) : -10278.81
LL(final, whole model) : -8564.25
Rho-squared vs equal shares : Not applicable
Adj.Rho-squared vs equal shares : Not applicable
Rho-squared vs observed shares : Not applicable
Adj.Rho-squared vs observed shares : Not applicable
AIC : 17168.5
BIC : NA

LL(0,Class_1) : -10406.54
LL(final,Class_1) : -11325.7
LL(0,Class_2) : -10406.54
LL(final,Class_2) : -10167.25
LL(0,Class_3) : -10406.54
LL(final,Class_3) : -10721.58

[stephanehess]

Hi

I can look into this if you can share your code and data with me outside the forum

Thanks

Stephane

[stephanehess]

Hi

in your case, Apollo fails to do these calculations automatically. We will add some further flexibility in future releases. For now, adding this after estimation (and before apollo_modelOutput) will resolve it for you (countAlt is the number of alternatives):

Code: Select all

model$rho2_0 <- 1-(model$maximum/model$LL0[1])
model$adjRho2_0 <- 1-((model$maximum-model$nFreeParams)/model$LL0[1])
model$rho2_C <- 1-(model$maximum/model$LLC[1])
countAlt=3
model$adjRho2_C <- 1-((model$maximum-model$nFreeParams+sum(countAlt-1))/model$LLC[1])
model$AIC <- -2*model$maximum + 2*model$nFreeParams
model$BIC <- -2*model$maximum + model$nFreeParams*log(model$nObs)

Stephane

[Iris123]

Perfect, thank you Stephane!

Just to be sure: in our dual-response model, the forced model has 2 alternatives and the opt-out model has 3 alternatives. Does this matter for countAlt?

Iris

[stephanehess]

ah, tricky one. I think for rho2C, I would then just use 3, as they would be generic constants