ApolloChoiceModelling forum

Hi everyone,

I want to estimate a mixed logit model that is very close to the (old) examplaes from the Apollo Homepage (Apollo Example 14: Mixed logit model on Swiss route choice data, uncorrelated Lognormals in utility space). The model (with my parameter) converges, however, while computing the covariance matrix I get the following Errors/Warings:

- ERROR: Singular Hessian, cannot calculate s.e.
- WARNING: Some eigenvalues of the Hessian are positive, indicating convergence to a saddle point!

I used my parameter and the utility function that I wanted to with the mixed logit model and did an basic MNL estimation (based on Apollo Example 2: "Simple MNL model on mode choice SP data"), where the model converges and I get all the results just fine. So I guess, there is something wrong in my model definition for the mixed logit. Do you have any idea what that might be?

Below you find first the results of the mixed logit and afterwards the code for it.

If you have any questions, I glady provide more information.

Thank you very much in advance!
Sutsch

1) Results of the mixed logit ############################################################
####################################################################################

Computing covariance matrix using numerical methods (maxLik). This may take a while, no progress bar displayed.
ERROR: Singular Hessian, cannot calculate s.e.
Hessian written to P3_MLM_MULL_hessian.csv
WARNING: Some eigenvalues of the Hessian are positive, indicating convergence to a saddle point!
Computing score matrix...
Calculating LL(0)...
Calculating LL of each model component...


Model run using Apollo for R, version 0.2.1 on Windows by XXX
www.ApolloChoiceModelling.com

Model name : P3_MLM
Model description : MLM on XXX
Model run at : 2022-01-14 00:01:22
Estimation method : bfgs
Model diagnosis : successful convergence
Number of individuals : 778
Number of observations : 9336

Number of cores used : 4
Number of inter-individual draws : 30 (mlhs)

LL(start) : -6471.222
LL(0) : -6471.222
LL(final) : -4837.951
Rho-square (0) : 0.2524
Adj.Rho-square (0) : 0.2484
AIC : 9727.9
BIC : 9913.58


Estimated parameters : 26
Time taken (hh:mm:ss) : 00:08:10.09
pre-estimation : 00:01:6.21
estimation : 00:01:18.03
post-estimation : 00:05:45.86
Iterations : 43
Min abs eigenvalue of Hessian : 243.249
Some eigenvalues of Hessian are positive, indicating potential problems!

Estimates:
Estimate s.e. t.rat.(0) Rob.s.e. Rob.t.rat.(0)
asc_alt1 0.12162 NA NA NA NA
mu_p1 -0.37516 NA NA NA NA
mu_p2 -0.11042 NA NA NA NA
mu_a1 0.16079 NA NA NA NA
mu_a2 0.12996 NA NA NA NA
mu_a3 0.13071 NA NA NA NA
mu_s1 0.14692 NA NA NA NA
mu_s2 0.07806 NA NA NA NA
mu_s3 0.12928 NA NA NA NA
mu_d1 0.31295 NA NA NA NA
mu_d2 0.24027 NA NA NA NA
mu_d3 0.44767 NA NA NA NA
mu_d4 0.54913 NA NA NA NA
b_c -0.11947 NA NA NA NA
sigma_p1 0.74017 NA NA NA NA
sigma_p2 0.79890 NA NA NA NA
sigma_a1 -0.22254 NA NA NA NA
sigma_a2 0.01733 NA NA NA NA
sigma_a3 0.13300 NA NA NA NA
sigma_s1 0.09250 NA NA NA NA
sigma_s2 -0.09569 NA NA NA NA
sigma_s3 -0.17211 NA NA NA NA
sigma_d1 -0.07971 NA NA NA NA
sigma_d2 0.06219 NA NA NA NA
sigma_d3 0.11595 NA NA NA NA
sigma_d4 -0.39956 NA NA NA NA




2) Code of the mixed logit (Except for database transformations)#############################
####################################################################################

apollo_control = list(
modelName ="P3_MLM_XXX",
modelDescr ="MLM on XXX",
indivID ="ID",
mixing = TRUE,
nCores = 4
)

database <- read.csv2("XXX.csv",header=TRUE, encoding="latin1")
colnames(database) <- c("ID", colnames(database)[-1])

pollo_beta = c(asc_alt1 = 0,
mu_p1 = 0,
mu_p2 = 0,
mu_a1 = 0,
mu_a2 = 0,
mu_a3 = 0,
mu_s1 = 0,
mu_s2 = 0,
mu_s3 = 0,
mu_d1 = 0,
mu_d2 = 0,
mu_d3 = 0,
mu_d4 = 0,
b_c = 0,

sigma_p1 = 0,
sigma_p2 = 0,
sigma_a1 = 0,
sigma_a2 = 0,
sigma_a3 = 0,
sigma_s1 = 0,
sigma_s2 = 0,
sigma_s3 = 0,
sigma_d1 = 0,
sigma_d2 = 0,
sigma_d3 = 0,
sigma_d4 = 0)

apollo_fixed = c()

apollo_draws = list(
interDrawsType = "mlhs", #more than 5 dimensions, paper: mlhs > halton (Hess, et al. 2006)
interNDraws = 30, #increase later
interUnifDraws = c(),
interNormDraws = c("draws_p1",
"draws_p2",
"draws_a1",
"draws_a2",
"draws_a3",
"draws_s1",
"draws_s2",
"draws_s3",
"draws_d1",
"draws_d2",
"draws_d3",
"draws_d4"),
intraDrawsType = "mlhs",
intraNDraws = 0,
intraUnifDraws = c(),
intraNormDraws = c()

pollo_randCoeff = function(apollo_beta, apollo_inputs){
randcoeff = list()

#Base Model

randcoeff[["b_p1"]] = (mu_p1 + sigma_p1 * draws_p1)
randcoeff[["b_p2"]] = (mu_p2 + sigma_p2 * draws_p2)
randcoeff[["b_a1"]] = (mu_a1 + sigma_a1 * draws_a1)
randcoeff[["b_a2"]] = (mu_a2 + sigma_a2 * draws_a2)
randcoeff[["b_a3"]] = (mu_a3 + sigma_a3 * draws_a3)
randcoeff[["b_s1"]] = (mu_s1 + sigma_s1 * draws_s1)
randcoeff[["b_s2"]] = (mu_s2 + sigma_s2 * draws_s2)
randcoeff[["b_s3"]] = (mu_s3 + sigma_s3 * draws_s3)
randcoeff[["b_d1"]] = (mu_d1 + sigma_d1 * draws_d1)
randcoeff[["b_d2"]] = (mu_d2 + sigma_d2 * draws_d2)
randcoeff[["b_d3"]] = (mu_d3 + sigma_d3 * draws_d3)
randcoeff[["b_d4"]] = (mu_d4 + sigma_d4 * draws_d4)

return(randcoeff)
}
apollo_inputs = apollo_validateInputs()

apollo_probabilities=function(apollo_beta, apollo_inputs, functionality="estimate"){

apollo_attach(apollo_beta, apollo_inputs)
on.exit(apollo_detach(apollo_beta, apollo_inputs))

P = list()
V = list()

V[['alt1']] = asc_alt1 +
b_p1 * (att_P_alt1==2) +
b_p2 * (att_P_alt1==3) +
b_a1 * (att_A1_alt1==1) +
b_a2 * (att_A2_alt1==1) +
b_a3 * (att_A3_alt1==1) +
b_s1 * (att_S1_alt1==1) +
b_s2 * (att_S2_alt1==1) +
b_s3 * (att_S3_alt1==1) +
b_d1 * (att_T_alt1==1) +
b_d2 * (att_T_alt1==2) +
b_d3 * (att_T_alt1==3) +
b_d4 * (att_T_alt1==4) +
b_c * att_C_alt1

V[['alt2']] =
b_p1 * (att_P_alt2==2) +
b_p2 * (att_P_alt2==3) +
b_a1 * (att_A1_alt2==1) +
b_a2 * (att_A2_alt2==1) +
b_a3 * (att_A3_alt2==1) +
b_s1 * (att_S1_alt2==1) +
b_s2 * (att_S2_alt2==1) +
b_s3 * (att_S3_alt2==1) +
b_d1 * (att_T_alt2==1) +
b_d2 * (att_T_alt2==2) +
b_d3 * (att_T_alt2==3) +
b_d4 * (att_T_alt2==4) +
b_c * att_C_alt2

mnl_settings = list(
alternatives = c(alt1=1, alt2=2),
avail = list(alt1=1, alt2=1),
choiceVar = choice,
V = V
)

P[['model']] = apollo_mnl(mnl_settings, functionality)

P = apollo_panelProd(P, apollo_inputs, functionality)

P = apollo_avgInterDraws(P, apollo_inputs, functionality)

P = apollo_prepareProb(P, apollo_inputs, functionality)
return(P)
}

model = apollo_estimate(apollo_beta, apollo_fixed,
apollo_probabilities, apollo_inputs,
estimate_settings=list(hessianRoutine="maxLik"))

Hi

as a first step, can you update to a recent version of Apollo - your version is ancient.

Second, can you show us your MNL results, please?

Stephane

Hi Stephane,

thank you for your quick reply! below you'll find the results of the MNL (and in addition the code afterwards).

I ll do the update tonight (What happened to: "never change a running system?").

Thank you in advance for your help!

Greetings
Sutsch



Results MNL###########################

Model run using Apollo for R, version 0.2.1 on Windows by XXX
www.ApolloChoiceModelling.com

Model name : 220106_P3_MNL_XXX
Model description : MNL MOdel for preferences on XXXX
Model run at : 2022-01-14 17:19:39
Estimation method : bfgs
Model diagnosis : successful convergence
Number of individuals : 778
Number of observations : 9336

Number of cores used : 1
Model without mixing

LL(start) : -6471.222
LL(0) : -6471.222
LL(final) : -4925.563
Rho-square (0) : 0.2389
Adj.Rho-square (0) : 0.2367
AIC : 9879.13
BIC : 9979.11


Estimated parameters : 14
Time taken (hh:mm:ss) : 00:00:10.39
pre-estimation : 00:00:3.69
estimation : 00:00:3.14
post-estimation : 00:00:3.56
Iterations : 21
Min abs eigenvalue of Hessian : 151.4354

Estimates:
Estimate s.e. t.rat.(0) p(1-sided) Rob.s.e. Rob.t.rat.(0) p(1-sided)
asc_alt1 0.10580 0.024776 4.270 9.767e-06 0.026953 3.925 4.333e-05
beta_p1 -0.31086 0.038147 -8.149 2.220e-16 0.048754 -6.376 9.082e-11
beta_p2 -0.10046 0.037383 -2.687 0.003603 0.048719 -2.062 0.019606
beta_a1 0.13648 0.027176 5.022 2.556e-07 0.028246 4.832 6.762e-07
beta_a2 0.10120 0.026845 3.770 8.166e-05 0.025946 3.901 4.798e-05
beta_a3 0.11850 0.027009 4.387 5.739e-06 0.025527 4.642 1.726e-06
beta_s1 0.13345 0.027154 4.915 4.449e-07 0.027430 4.865 5.721e-07
beta_s2 0.06509 0.026933 2.417 0.007827 0.026110 2.493 0.006332
beta_s3 0.11460 0.027250 4.206 1.302e-05 0.027461 4.173 1.501e-05
beta_d1 0.28566 0.050985 5.603 1.054e-08 0.052984 5.392 3.493e-08
beta_d2 0.21586 0.051150 4.220 1.220e-05 0.054326 3.973 3.541e-05
beta_d3 0.39520 0.051594 7.660 9.326e-15 0.057137 6.917 2.312e-12
beta_d4 0.47822 0.051768 9.238 0.000000 0.059136 8.087 3.331e-16
beta_c -0.10529 0.002340 -45.001 0.000000 0.004318 -24.386 0.000000


Code of the MNL#######################################

apollo_initialise()

### Set core controls
apollo_control = list(
modelName ="220106_P3_MNL_XXX",
modelDescr ="MNL MOdel for preferences on XXXX",
indivID ="ID"
)

database <- read.csv2("XXX",header=TRUE, encoding="latin1")

colnames(database) <- c("ID", colnames(database)[-1])

pollo_beta = c(

asc_alt1 = 0,
beta_p1 = 0,
beta_p2 = 0,
beta_a1 = 0,
beta_a2 = 0,
beta_a3 = 0,
beta_s1 = 0,
beta_s2 = 0,
beta_s3 = 0,
beta_d1 = 0,
beta_d2 = 0,
beta_d3 = 0,
beta_d4 = 0,
beta_c = 0

apollo_fixed = c()
apollo_inputs = apollo_validateInputs()

apollo_probabilities=function(apollo_beta, apollo_inputs, functionality="estimate"){

apollo_attach(apollo_beta, apollo_inputs)
on.exit(apollo_detach(apollo_beta, apollo_inputs))

P = list()

V = list()

V[['alt1']] =
asc_alt1 +
beta_p1 * (att_P_alt1==2) +
beta_p2 * (att_P_alt1==3) +
beta_a1 * (att_A1_alt1==1) +
beta_a2 * (att_A2_alt1==1) +
beta_a3 * (att_A3_alt1==1) +
beta_s1 * (att_S1_alt1==1) +
beta_s2 * (att_S2_alt1==1) +
beta_s3 * (att_S3_alt1==1) +
beta_d1 * (att_T_alt1==1) +
beta_d2 * (att_T_alt1==2) +
beta_d3 * (att_T_alt1==3) +
beta_d4 * (att_T_alt1==4) +
beta_c * att_C_alt1

V[['alt2']] =
beta_p1 * (att_P_alt2==2) +
beta_p2 * (att_P_alt2==3) +
beta_a1 * (att_A1_alt2==1) +
beta_a2 * (att_A2_alt2==1) +
beta_a3 * (att_A3_alt2==1) +
beta_s1 * (att_S1_alt2==1) +
beta_s2 * (att_S2_alt2==1) +
beta_s3 * (att_S3_alt2==1) +
beta_d1 * (att_T_alt2==1) +
beta_d2 * (att_T_alt2==2) +
beta_d3 * (att_T_alt2==3) +
beta_d4 * (att_T_alt2==4) +
beta_c * att_C_alt2


mnl_settings = list(
alternatives = c(alt1=1, alt2=2),
avail = list(alt1=1, alt2=1),
choiceVar = choice,
V = V
)


P[['model']] = apollo_mnl(mnl_settings, functionality)

P = apollo_panelProd(P, apollo_inputs, functionality)

P = apollo_prepareProb(P, apollo_inputs, functionality)
return(P)
}

model = apollo_estimate(apollo_beta, apollo_fixed, apollo_probabilities, apollo_inputs)

Are these attributes all just 0/1? The data might not be rich enough to support all the random parameters

The other issue you have is that your number of draws is very low and your estimation and calculation of the covariance matrix will be very unstable as a result

Hi Stephane,

I updated Apollo and increased the mlhs draws to 100 and I got results that I have to investigate now (See below). So I guess my issue ist resolved.
Thank you also for your concern regarding my attributes! I have 2 sets of binary attributes (A1/A2/A3 and S1/S2/S3) that have been presented as one attribute to the respondents so that I am able to estimate the different components seperately. The other attributes are 3 and 5 level attributes.

Thank you for your help!

Best wishes
Sascha


Model run using Apollo for R, version 0.2.6 on Windows by sasch
www.ApolloChoiceModelling.com

Model name : P3_MLM_XXX
Model description : MLM on pervceived DXXX
Model run at : 2022-01-15 00:19:38
Estimation method : bfgs
Model diagnosis : successful convergence
Number of individuals : 778
Number of rows in database : 9336
Number of modelled outcomes : 9336

Number of cores used : 4
Number of inter-individual draws : 100 (mlhs)

LL(start) : -6471.222
LL(0) : -6471.222
LL(C) : -6468.905
LL(final) : -4836.483
Rho-square (0) : 0.2526
Adj.Rho-square (0) : 0.2486
Rho-square (C) : 0.2523
Adj.Rho-square (C) : 0.2483
AIC : 9724.97
BIC : 9910.65

Estimated parameters : 26
Time taken (hh:mm:ss) : 00:15:37.94
pre-estimation : 00:01:59.99
estimation : 00:01:55.99
post-estimation : 00:11:41.97
Iterations : 41
Min abs eigenvalue of Hessian : 2467.523
Some eigenvalues of Hessian are positive, indicating potential problems!

Unconstrained optimisation.

Estimates:
Estimate s.e. t.rat.(0) Rob.s.e. Rob.t.rat.(0)
asc_alt1 0.12590 0.028525 4.4137 0.032403 3.8854
mu_p1 -0.36498 0.053227 -6.8570 0.065725 -5.5531
mu_p2 -0.11400 0.052688 -2.1636 0.062602 -1.8210
mu_a1 0.16180 0.031593 5.1212 0.031495 5.1371
mu_a2 0.12846 0.030591 4.1992 0.029774 4.3145
mu_a3 0.13250 0.031270 4.2374 0.031306 4.2326
mu_s1 0.14960 0.030828 4.8528 0.030341 4.9306
mu_s2 0.08389 0.007756 10.8168 0.024726 3.3930
mu_s3 0.12728 0.031616 4.0258 0.031832 3.9985
mu_d1 0.32414 0.059764 5.4237 0.063772 5.0829
mu_d2 0.23576 0.063965 3.6858 0.076917 3.0651
mu_d3 0.45958 0.059356 7.7428 0.068038 6.7547
mu_d4 0.55209 0.062641 8.8136 0.071993 7.6687
b_c -0.12237 0.003430 -35.6794 0.005672 -21.5733
sigma_p1 -0.80186 0.068741 -11.6649 0.091694 -8.7450
sigma_p2 0.82995 0.064910 12.7861 0.084455 9.8272
sigma_a1 -0.23930 0.078163 -3.0616 0.088413 -2.7067
sigma_a2 -0.07411 0.107013 -0.6925 0.168009 -0.4411
sigma_a3 0.03863 NaN NaN 0.156666 0.2466
sigma_s1 0.19502 0.085713 2.2753 0.090351 2.1585
sigma_s2 -0.06321 0.073074 -0.8651 0.054464 -1.1606
sigma_s3 0.20490 0.101464 2.0195 0.128872 1.5900
sigma_d1 -0.24001 0.203304 -1.1805 0.412669 -0.5816
sigma_d2 -0.34966 0.159963 -2.1859 0.214226 -1.6322
sigma_d3 0.02547 NaN NaN 0.099269 0.2565
sigma_d4 -0.50677 0.104180 -4.8643 0.115656 -4.3817