I have been analysing a DCE with three alternatives (1 opt out) which includes 5 attributes, 1 categorical with 4 levels (dummy coded) and 4 continuous variables. The utility function is defined with 1 ASC (as the ASCs were very similar and the use of 1 did not impact model fit statistics).
I have estimated an MNL, partial MXL (only the ASC is random), a fully random MXL and a random MXL with correlations. The increased complexity led to significant LR tests and reductions in the AIC and BIC for each successive model. Therefore the random MXL with correlations has the best fit.
My goal is to report the mean preferences for respondents whilst also presenting the heterogeneity that is present. I aim to explore observable heterogeneity within a separate paper. The model output is attached for information.
My question is: the standard deviations of the correlated model are very large relative to the mean(e.g., for one dummy variable the mean=0.391 and St.Dev = 1.512). Plotting density plots of the conditional mean estimates suggests non-normality due to multiple peaks in the distribution for attributes or wide tails for others. To me this is suggesting that reporting this model does not adequately capture population preferences as the mean is just averaging across the peaks.
I am relatively new to DCEs as I am a PhD student. I'm looking for your opinion, is there still value in this model for presenting population level preferences or is this a case where those preferences are best expressed through a latent class approach?
The worst culprit's density plots are below. In particular, cancer detection crosses 0, I have explored the log-normal distribution for this attribute yet text responses which report why respondents chose to opt out suggest that some were scared of being diagnosed with cancer so it is theoretically plausible that it may be negative for some respondents.
- v_mam_rev2.png (95.64 KiB) Viewed 720 times
- v_can_det.png (88.85 KiB) Viewed 720 times
Code: Select all
Model run at : 2025-11-01 14:00:21.281343
Estimation method : bgw
Model diagnosis : Relative function convergence
Optimisation diagnosis : Maximum found
hessian properties : Negative definite
maximum eigenvalue : -1.274276
reciprocal of condition number : 4.58539e-06
Number of individuals : 962
Number of rows in database : 7696
Number of modelled outcomes : 7696
Number of cores used : 24
Number of inter-individual draws : 5000 (sobol)
LL(start) : -6994.5
LL at equal shares, LL(0) : -8454.92
LL at observed shares, LL(C) : -7141.45
LL(final) : -5335.69
Rho-squared vs equal shares : 0.3689
Adj.Rho-squared vs equal shares : 0.3637
Rho-squared vs observed shares : 0.2529
Adj.Rho-squared vs observed shares : 0.247
AIC : 10759.38
BIC : 11065.11
Estimated parameters : 44
Time taken (hh:mm:ss) : 03:18:55.7
pre-estimation : 00:18:31.19
estimation : 00:22:33.24
post-estimation : 02:37:51.27
Iterations : 37
Unconstrained optimisation.
Estimates:
Estimate s.e. t.rat.(0) p(2-sided) Rob.s.e. Rob.t.rat.(0) p(2-sided)
mu_b_asc 2.787339 0.852937 3.2679 0.001083 0.841085 3.31398 9.1978e-04
mu_b_mr2 0.391036 0.089317 4.3780 1.197e-05 0.091144 4.29031 1.784e-05
mu_b_mr3 0.110041 0.086199 1.2766 0.201742 0.084671 1.29964 0.193725
mu_b_mr4 -0.766809 0.098051 -7.8205 5.329e-15 0.103249 -7.42677 1.112e-13
mu_b_cd 0.086893 0.007028 12.3630 0.000000 0.007649 11.36075 0.000000
mu_b_ir -0.021861 0.005882 -3.7165 2.0202e-04 0.005810 -3.76283 1.6800e-04
mu_b_ov -0.239297 0.043576 -5.4915 3.985e-08 0.043022 -5.56220 2.664e-08
mu_b_wt -0.217753 0.017175 -12.6781 0.000000 0.018705 -11.64165 0.000000
sigma_b_asc -10.970206 0.865753 -12.6713 0.000000 0.900534 -12.18189 0.000000
sigma_b_asc_cd 0.094412 0.010500 8.9914 0.000000 0.010797 8.74391 0.000000
sigma_b_asc_mr2 0.036573 0.155767 0.2348 0.814367 0.143784 0.25436 0.799215
sigma_b_asc_mr3 -0.186790 0.156824 -1.1911 0.233621 0.153676 -1.21548 0.224185
sigma_b_asc_mr4 0.509273 0.170595 2.9853 0.002833 0.161946 3.14471 0.001663
sigma_b_asc_ir 0.007978 0.012013 0.6641 0.506613 0.015313 0.52099 0.602376
sigma_b_asc_ov 0.283022 0.076504 3.6994 2.1609e-04 0.082587 3.42697 6.1036e-04
sigma_b_asc_wt 0.050994 0.032901 1.5499 0.121161 0.038544 1.32299 0.185838
sigma_b_cd -0.087253 0.005948 -14.6684 0.000000 0.006461 -13.50504 0.000000
sigma_b_cd_mr2 -0.264063 0.143938 -1.8346 0.066572 0.148370 -1.77976 0.075115
sigma_b_cd_mr3 -0.249627 0.145045 -1.7210 0.085245 0.159650 -1.56359 0.117915
sigma_b_cd_mr4 -0.081542 0.166436 -0.4899 0.624184 0.177576 -0.45920 0.646094
sigma_b_cd_ir 0.007641 0.009929 0.7695 0.441611 0.010238 0.74632 0.455474
sigma_b_cd_ov 0.325801 0.073429 4.4370 9.124e-06 0.064563 5.04629 4.505e-07
sigma_b_cd_wt 0.131393 0.027700 4.7434 2.102e-06 0.026870 4.88996 1.009e-06
sigma_b_mr2 1.488525 0.138150 10.7747 0.000000 0.174792 8.51596 0.000000
sigma_b_mr2_mr3 1.366969 0.159142 8.5896 0.000000 0.221621 6.16805 6.914e-10
sigma_b_mr2_mr4 0.907699 0.177095 5.1255 2.967e-07 0.233729 3.88356 1.0294e-04
sigma_b_mr2_ir 0.004015 0.012723 0.3156 0.752327 0.018980 0.21154 0.832467
sigma_b_mr2_ov 0.288801 0.070433 4.1004 4.125e-05 0.073370 3.93625 8.277e-05
sigma_b_mr2_wt 0.155911 0.041312 3.7740 1.6068e-04 0.071050 2.19437 0.028209
sigma_b_mr3 0.632958 0.135180 4.6823 2.836e-06 0.205070 3.08654 0.002025
sigma_b_mr3_mr4 1.375926 0.138265 9.9514 0.000000 0.159269 8.63900 0.000000
sigma_b_mr3_ir 0.009464 0.020850 0.4539 0.649890 0.034620 0.27338 0.784563
sigma_b_mr3_ov -0.205567 0.084173 -2.4422 0.014598 0.089309 -2.30176 0.021349
sigma_b_mr3_wt -0.053925 0.033817 -1.5946 0.110798 0.032364 -1.66623 0.095668
sigma_b_mr4 -0.140086 0.332039 -0.4219 0.673101 0.540962 -0.25896 0.795668
sigma_b_mr4_ir -0.069534 0.013113 -5.3026 1.142e-07 0.015178 -4.58114 4.624e-06
sigma_b_mr4_ov -0.168469 0.227694 -0.7399 0.459367 0.350722 -0.48035 0.630980
sigma_b_mr4_wt -0.078201 0.136666 -0.5722 0.567182 0.265571 -0.29447 0.768402
sigma_b_ir -0.009474 0.053419 -0.1773 0.859238 0.100120 -0.09462 0.924615
sigma_b_ir_ov 0.348711 0.179286 1.9450 0.051775 0.199128 1.75119 0.079913
sigma_b_ir_wt -0.144087 0.069504 -2.0731 0.038165 0.056248 -2.56163 0.010418
sigma_b_ov -0.583270 0.158171 -3.6876 2.2639e-04 0.219800 -2.65364 0.007963
sigma_b_ov_wt 0.061985 0.080316 0.7718 0.440255 0.068170 0.90926 0.363212
sigma_b_wt -0.065380 0.096041 -0.6808 0.496027 0.121303 -0.53898 0.589900
Settings and functions used in model definition:
apollo_control
--------------
Value
modelDescr "PreferAI: MNL preference space model"
modelName "PreferAI_MXLall_rand_corr_FINAL"
indivID "id"
outputDirectory "output/"
nCores "24"
mixing "TRUE"
debug "FALSE"
workInLogs "FALSE"
seed "13"
HB "FALSE"
noValidation "FALSE"
noDiagnostics "FALSE"
calculateLLC "TRUE"
analyticHessian "FALSE"
memorySaver "FALSE"
panelData "TRUE"
analyticGrad "TRUE"
analyticGrad_manualSet "FALSE"
overridePanel "FALSE"
preventOverridePanel "FALSE"
noModification "FALSE"
Hessian routines attempted
--------------------------
numerical jacobian of LL analytical gradient
Scaling used in computing Hessian
---------------------------------
Value
mu_b_asc 2.787338911
mu_b_mr2 0.391036269
mu_b_mr3 0.110041447
mu_b_mr4 0.766809327
mu_b_cd 0.086893417
mu_b_ir 0.021861356
mu_b_ov 0.239296677
mu_b_wt 0.217752884
sigma_b_asc 10.970206335
sigma_b_asc_cd 0.094411576
sigma_b_asc_mr2 0.036573419
sigma_b_asc_mr3 0.186789931
sigma_b_asc_mr4 0.509272585
sigma_b_asc_ir 0.007978048
sigma_b_asc_ov 0.283021752
sigma_b_asc_wt 0.050993730
sigma_b_cd 0.087252567
sigma_b_cd_mr2 0.264062815
sigma_b_cd_mr3 0.249626631
sigma_b_cd_mr4 0.081541812
sigma_b_cd_ir 0.007640505
sigma_b_cd_ov 0.325801348
sigma_b_cd_wt 0.131392582
sigma_b_mr2 1.488524505
sigma_b_mr2_mr3 1.366969330
sigma_b_mr2_mr4 0.907698566
sigma_b_mr2_ir 0.004014937
sigma_b_mr2_ov 0.288800596
sigma_b_mr2_wt 0.155911019
sigma_b_mr3 0.632958148
sigma_b_mr3_mr4 1.375925797
sigma_b_mr3_ir 0.009464213
sigma_b_mr3_ov 0.205567256
sigma_b_mr3_wt 0.053925402
sigma_b_mr4 0.140086017
sigma_b_mr4_ir 0.069534334
sigma_b_mr4_ov 0.168468628
sigma_b_mr4_wt 0.078201397
sigma_b_ir 0.009473535
sigma_b_ir_ov 0.348711090
sigma_b_ir_wt 0.144087044
sigma_b_ov 0.583269566
sigma_b_ov_wt 0.061984631
sigma_b_wt 0.065380072
apollo_randCoeff
------------------
function(apollo_beta, apollo_inputs){
randcoeff = list()
randcoeff[["v_asc"]] = mu_b_asc + sigma_b_asc *draws_asc
randcoeff[["v_can_det"]] = mu_b_cd + sigma_b_asc_cd *draws_asc + sigma_b_cd *draws_cd
randcoeff[["v_mam_rev2"]] = mu_b_mr2 + sigma_b_asc_mr2*draws_asc + sigma_b_cd_mr2*draws_cd + sigma_b_mr2 *draws_mr2
randcoeff[["v_mam_rev3"]] = mu_b_mr3 + sigma_b_asc_mr3*draws_asc + sigma_b_cd_mr3*draws_cd + sigma_b_mr2_mr3*draws_mr2 + sigma_b_mr3 *draws_mr3
randcoeff[["v_mam_rev4"]] = mu_b_mr4 + sigma_b_asc_mr4*draws_asc + sigma_b_cd_mr4*draws_cd + sigma_b_mr2_mr4*draws_mr2 + sigma_b_mr3_mr4*draws_mr3 + sigma_b_mr4 *draws_mr4
randcoeff[["v_inap_rec"]] = mu_b_ir + sigma_b_asc_ir *draws_asc + sigma_b_cd_ir *draws_cd + sigma_b_mr2_ir *draws_mr2 + sigma_b_mr3_ir *draws_mr3 + sigma_b_mr4_ir *draws_mr4 + sigma_b_ir *draws_ir
randcoeff[["v_over"]] = mu_b_ov + sigma_b_asc_ov *draws_asc + sigma_b_cd_ov *draws_cd + sigma_b_mr2_ov *draws_mr2 + sigma_b_mr3_ov *draws_mr3 + sigma_b_mr4_ov *draws_mr4 + sigma_b_ir_ov*draws_ir + sigma_b_ov *draws_ov
randcoeff[["v_wait"]] = mu_b_wt + sigma_b_asc_wt *draws_asc + sigma_b_cd_wt *draws_cd + sigma_b_mr2_wt *draws_mr2 + sigma_b_mr3_wt *draws_mr3 + sigma_b_mr4_wt *draws_mr4 + sigma_b_ir_wt*draws_ir + sigma_b_ov_wt*draws_ov +sigma_b_wt*draws_wt
return(randcoeff)
}
<bytecode: 0x00000187d0e27370>
apollo_probabilities
----------------------
function(apollo_beta, apollo_inputs, functionality = "estimate") {
apollo_attach(apollo_beta, apollo_inputs)
on.exit(apollo_detach(apollo_beta, apollo_inputs))
# Create list of probabilities P
P = list()
# List of utilities using the random coefficients described above and NOT the betas
V = list()
V[["A"]] = v_asc +
v_mam_rev2 * mam_rev2_1 +
v_mam_rev3 * mam_rev3_1 +
v_mam_rev4 * mam_rev4_1 +
v_can_det * can_det_1+
v_inap_rec * inap_rec_1 +
v_over * over_1 +
v_wait * wait_1
V[["B"]] = v_asc+
v_mam_rev2 * mam_rev2_2 +
v_mam_rev3 * mam_rev3_2 +
v_mam_rev4 * mam_rev4_2 +
v_can_det * can_det_2 +
v_inap_rec * inap_rec_2 +
v_over * over_2 +
v_wait * wait_2
V[["OO"]] = 0 # Opt-out
# Define MNL settings
mnl_settings = list(
alternatives = c(A = 1, B = 2, OO = 3),
choiceVar = choice[,1],
utilities = V
)
# Compute Probabilities
P[["model"]] = apollo_mnl(mnl_settings, functionality)
# Take product across observation for same individual
P = apollo_panelProd(P, apollo_inputs, functionality)
# Average across inter-individual draws
P = apollo_avgInterDraws(P, apollo_inputs, functionality)
# Prepare and return outputs of function
P = apollo_prepareProb(P, apollo_inputs, functionality)
return(P)
}