WARNING: Estimation failed. No covariance matrix to compute. MNL

[DTLAndreoli]

Dear,

It is my first time working with the Apollo package and upon running a MNL model, I receive the following warning: WARNING: Estimation failed. No covariance matrix to compute, with the associated N/A values for the s.e. ect. I have had a look at some of the other mentions of similar issues on this forum, but cannot seem to find a solution to the issue. The code and generated outcomes are presented below. The used data consists of 5940 observations by 1485 individuals; all attributes used for V are included in a fully orthogonal experimental design and have been effect coded.

Any help is kindly appreciated.

Best, Dennis

> ### Clear memory
> rm(list = ls())
>
> ### Load Apollo library
> library(apollo)
>
> ### Initialise code
> apollo_initialise()
Apollo ignition sequence completed
>
> ### Set core controls
> apollo_control = list(
+ modelName = "MNL_RespBiasTest",
+ modelDescr = "MNL model used to test the effect of respbias indicator on model performance",
+ indivID = "RespID",
+ outputDirectory = "output"
+ )
>
> # ################################################################# #
> #### LOAD DATA AND APPLY ANY TRANSFORMATIONS ####
> # ################################################################# #
>
> ### Load in data
> database = read.csv("C:/Users/s160511/OneDrive - TU Eindhoven/Documents/PhD/03. Experiments/Experiment 1&2/Experiment 1 (SC)/Data/03. Data conversion/DCM/250317 all.csv",header=TRUE)
>
> database = subset(database,database$ExpID==1) ### Exclude data of res experiment
> ### database = subset(database,database$RespBias==0) ### Exclude Respondents with bias indicator
>
> # ################################################################# #
> #### DEFINE MODEL PARAMETERS ####
> # ################################################################# #
>
> ### Vecotr of parameters
> apollo_beta=c(
+ asc_Constant_A1 = 0,
+ asc_Constant_A2 = 0,
+ b_PL1 = 0,
+ b_PL2 = 0,
+ b_PL3 = 0,
+ b_PR1 = 0,
+ b_PR2 = 0,
+ b_PR3 = 0,
+ b_OpenF1 = 0,
+ b_BuildH1 = 0,
+ b_BuildH2 = 0,
+ b_BuildH3 = 0,
+ b_ToInt1 = 0,
+ b_ToInt2 = 0,
+ b_ToInt3 = 0,
+ b_Green1 = 0,
+ b_Green2 = 0,
+ b_Green3 = 0,
+ b_Parking1 = 0,
+ b_ToCI1 = 0,
+ b_ToCI2 = 0,
+ b_ToCI3 = 0,
+ b_CIw1 = 0,
+ b_Curb1 = 0,
+ b_Speed1 = 0,
+ b_MTV1 = 0,
+ b_X2WTV1 = 0,
+ b_PV1 = 0,
+ b_AppCInt1 = 0,
+ b_AppCInt2 = 0,
+ b_AppCInt3 = 0,
+ b_Weather1 = 0,
+ b_Weather2 = 0,
+ b_Weather3 = 0,
+ b_TOD1 = 0
+ )
>
> apollo_fixed = c("asc_Constant_A1")
>
> # ################################################################# #
> #### GROUP AND VALIDATE INPUTS ####
> # ################################################################# #
>
> apollo_inputs=apollo_validateInputs()
Several observations per individual detected based on the value of RespID. Setting panelData in
apollo_control set to TRUE.
All checks on apollo_control completed.
WARNING: Your database contains some entries that are NA. This may well be intentional, but be advised that if these
entries are used in your model, the behaviour may be unexpected.
All checks on database completed.
>
>
> # ################################################################# #
> #### DEFINE MODEL AND LIKELIHOOD FUNCTION ####
> # ################################################################# #
>
> apollo_probabilities = function(apollo_beta, apollo_inputs, functionality="estimate") {
+
+ ### Attach inputs and detach after function exit
+ apollo_attach(apollo_beta, apollo_inputs)
+ on.exit(apollo_detach(apollo_beta, apollo_inputs))
+
+ ### Initialize probabilities list
+ P = list()
+
+ ### List of utilities: these must use the same names as in mnl_settings, order is irrelevant
+ V = list()
+ V[["alt1"]] =
+ asc_Constant_A1 +
+ b_PL1 * PL1_A1 + b_PL2 * PL2_A1 + b_PL3 * PL3_A1 +
+ b_PR1 * PR1_A1 + b_PR2 * PR2_A1 + b_PR3 * PR3_A1 +
+ b_OpenF1 * OpenF1_A1 +
+ b_BuildH1 * BuildH1_A1 + b_BuildH2 * BuildH2_A1 + b_BuildH3 * BuildH3_A1 +
+ b_ToInt1 * ToInt1_A1 + b_ToInt2 * ToInt2_A1 + b_ToInt3 * ToInt3_A1 +
+ b_Green1 * Green1_A1 + b_Green2 * Green2_A1 + b_Green3 * Green3_A1 +
+ b_Parking1 * Parking1_A1 +
+ b_ToCI1 * ToCI1_A1 + b_ToCI2 * ToCI2_A1 + b_ToCI3 * ToCI3_A1 +
+ b_CIw1 * CIw1_A1 + b_Curb1 * Curb1_A1 + b_Speed1 * Speed1_A1 +
+ b_MTV1 * MTV1_A1 + b_X2WTV1 * X2WTV1_A1 + b_PV1 * PV1_A1 +
+ b_AppCInt1 * AppCInt1_A1 + b_AppCInt2 * AppCInt2_A1 + b_AppCInt3 * AppCInt3_A1 +
+ b_Weather1 * Weather1_A1 + b_Weather2 * Weather2_A1 + b_Weather3 * Weather3_A1 +
+ b_TOD1 * TOD1_A1
+ V[["alt2"]] =
+ asc_Constant_A2 +
+ b_PL1 * PL1_A2 + b_PL2 * PL2_A2 + b_PL3 * PL3_A2 +
+ b_PR1 * PR1_A2 + b_PR2 * PR2_A2 + b_PR3 * PR3_A2 +
+ b_OpenF1 * OpenF1_A2 +
+ b_BuildH1 * BuildH1_A2 + b_BuildH2 * BuildH2_A2 + b_BuildH3 * BuildH3_A2 +
+ b_ToInt1 * ToInt1_A2 + b_ToInt2 * ToInt2_A2 + b_ToInt3 * ToInt3_A2 +
+ b_Green1 * Green1_A2 + b_Green2 * Green2_A2 + b_Green3 * Green3_A2 +
+ b_Parking1 * Parking1_A2 +
+ b_ToCI1 * ToCI1_A2 + b_ToCI2 * ToCI2_A2 + b_ToCI3 * ToCI3_A2 +
+ b_CIw1 * CIw1_A2 + b_Curb1 * Curb1_A2 + b_Speed1 * Speed1_A2 +
+ b_MTV1 * MTV1_A2 + b_X2WTV1 * X2WTV1_A2 + b_PV1 * PV1_A2 +
+ b_AppCInt1 * AppCInt1_A2 + b_AppCInt2 * AppCInt2_A2 + b_AppCInt3 * AppCInt3_A2 +
+ b_Weather1 * Weather1_A2 + b_Weather2 * Weather2_A2 + b_Weather3 * Weather3_A2 +
+ b_TOD1 * TOD1_A2
+
+ ### Define settings for MNL model component
+ mnl_settings = list(
+ alternatives = c(alt1 = 1, alt2 = 2),
+ choiceVar = ChosenAC,
+ utilities = V
+ )
+
+ ### Compute probabilities using MNL model
+ P[["model"]] = apollo_mnl(mnl_settings, functionality)
+
+ ### Take product across observation for same individual
+ P = apollo_panelProd(P, apollo_inputs, functionality)
+
+ ### Call apollo_prepareProb() to finalize probability calculation
+ P = apollo_prepareProb(P, apollo_inputs, functionality)
+ return(P)
+ }
>
> # ################################################################# #
> #### MODEL ESTIMATION ####
> # ################################################################# #
>
> model = apollo_estimate(apollo_beta, apollo_fixed,
+ apollo_probabilities, apollo_inputs)
Preparing user-defined functions.

Testing likelihood function...
Setting "avail" is missing, so full availability is assumed.

Overview of choices for MNL model component :
alt1 alt2
Times available 5940.00 5940.00
Times chosen 2849.00 3091.00
Percentage chosen overall 47.96 52.04
Percentage chosen when available 47.96 52.04


Pre-processing likelihood function...

Testing influence of parameters
Starting main estimation

BGW using analytic model derivatives supplied by caller...


Iterates will be written to:
output/MNL_RespBiasTest_iterations.csv
it nf F RELDF PRELDF RELDX MODEL stppar
0 1 4.117294253e+03
1 3 4.061545505e+03 1.354e-02 1.341e-02 1.00e+00 G 8.74e-02
2 4 4.061489028e+03 1.391e-05 1.394e-05 1.00e+00 G 9.00e-14
3 5 4.061489028e+03 6.430e-11 6.432e-11 2.00e-01 S -8.99e-14

***** Singular convergence *****

Estimated parameters:
Estimate
asc_Constant_A1 0.000000
asc_Constant_A2 0.006112
b_PL1 0.008260
b_PL2 -0.017483
b_PL3 0.040018
b_PR1 -0.033650
b_PR2 -0.003788
b_PR3 -0.026857
b_OpenF1 -0.003437
b_BuildH1 -0.001172
b_BuildH2 0.070025
b_BuildH3 -0.025759
b_ToInt1 0.016944
b_ToInt2 -0.033950
b_ToInt3 0.001637
b_Green1 0.010541
b_Green2 -0.011408
b_Green3 -0.011388
b_Parking1 5.501273
b_ToCI1 -0.022943
b_ToCI2 -0.007085
b_ToCI3 0.019029
b_CIw1 -0.002838
b_Curb1 -0.003760
b_Speed1 -0.002666
b_MTV1 0.005884
b_X2WTV1 -0.003733
b_PV1 -0.001100
b_AppCInt1 -0.016936
b_AppCInt2 -0.023859
b_AppCInt3 -0.016406
b_Weather1 0.031411
b_Weather2 0.024366
b_Weather3 0.010831
b_TOD1 -9.4981e-04

Final LL: -4061.489

WARNING: Estimation failed. No covariance matrix to compute.
Calculating log-likelihood at equal shares (LL(0)) for applicable models...
Calculating log-likelihood at observed shares from estimation data (LL(c)) for applicable models...
Calculating LL of each model component...
Calculating other model fit measures

Your model was estimated using the BGW algorithm. Please acknowledge this by citing Bunch et al. (1993) -
doi.org/10.1145/151271.151279

Please acknowledge the use of Apollo by citing Hess & Palma (2019) - doi.org/10.1016/j.jocm.2019.100170
>
> # ################################################################# #
> #### MODEL OUTPUTS ####
> # ################################################################# #
>
> apollo_modelOutput(model)
Model run by s160511 using Apollo 0.3.5 on R 4.4.3 for Windows.
Please acknowledge the use of Apollo by citing Hess & Palma (2019)
DOI 10.1016/j.jocm.2019.100170
www.ApolloChoiceModelling.com

Model name : MNL_RespBiasTest
Model description : MNL model used to test the effect of respbias indicator on model performance
Model run at : 2025-03-18 13:53:21.007701
Estimation method : bgw
Model diagnosis : Singular convergence
Number of individuals : 1485
Number of rows in database : 5940
Number of modelled outcomes : 5940

Number of cores used : 1
Model without mixing

LL(start) : -4117.29
LL at equal shares, LL(0) : -4117.29
LL at observed shares, LL(C) : -4112.36
LL(final) : -4061.49
Rho-squared vs equal shares : 0.0136
Adj.Rho-squared vs equal shares : 0.0053
Rho-squared vs observed shares : 0.0124
Adj.Rho-squared vs observed shares : 0.0043
AIC : 8190.98
BIC : 8418.42

Estimated parameters : 34
Time taken (hh:mm:ss) : 00:00:0.88
pre-estimation : 00:00:0.76
estimation : 00:00:0.1
post-estimation : 00:00:0.02
Iterations : 3 (Singular convergence)

Unconstrained optimisation.

Estimates:
Estimate s.e. t.rat.(0) Rob.s.e. Rob.t.rat.(0)
asc_Constant_A1 0.000000 NA NA NA NA
asc_Constant_A2 0.006112 NA NA NA NA
b_PL1 0.008260 NA NA NA NA
b_PL2 -0.017483 NA NA NA NA
b_PL3 0.040018 NA NA NA NA
b_PR1 -0.033650 NA NA NA NA
b_PR2 -0.003788 NA NA NA NA
b_PR3 -0.026857 NA NA NA NA
b_OpenF1 -0.003437 NA NA NA NA
b_BuildH1 -0.001172 NA NA NA NA
b_BuildH2 0.070025 NA NA NA NA
b_BuildH3 -0.025759 NA NA NA NA
b_ToInt1 0.016944 NA NA NA NA
b_ToInt2 -0.033950 NA NA NA NA
b_ToInt3 0.001637 NA NA NA NA
b_Green1 0.010541 NA NA NA NA
b_Green2 -0.011408 NA NA NA NA
b_Green3 -0.011388 NA NA NA NA
b_Parking1 5.501273 NA NA NA NA
b_ToCI1 -0.022943 NA NA NA NA
b_ToCI2 -0.007085 NA NA NA NA
b_ToCI3 0.019029 NA NA NA NA
b_CIw1 -0.002838 NA NA NA NA
b_Curb1 -0.003760 NA NA NA NA
b_Speed1 -0.002666 NA NA NA NA
b_MTV1 0.005884 NA NA NA NA
b_X2WTV1 -0.003733 NA NA NA NA
b_PV1 -0.001100 NA NA NA NA
b_AppCInt1 -0.016936 NA NA NA NA
b_AppCInt2 -0.023859 NA NA NA NA
b_AppCInt3 -0.016406 NA NA NA NA
b_Weather1 0.031411 NA NA NA NA
b_Weather2 0.024366 NA NA NA NA
b_Weather3 0.010831 NA NA NA NA
b_TOD1 -9.4981e-04 NA NA NA NA

>
> apollo_saveOutput(model)

Old result file "output/MNL_RespBiasTest_output.txt"
renamed to: "output/MNL_RespBiasTest_OLD3_output.txt"
Old result file "output/MNL_RespBiasTest_estimates.csv"
renamed to: "output/MNL_RespBiasTest_OLD3_estimates.csv"
Old result file "output/MNL_RespBiasTest_model.rds"
renamed to: "output/MNL_RespBiasTest_OLD3_model.rds"
Model output saved to output/MNL_RespBiasTest_output.txt
Estimates saved to output/MNL_RespBiasTest_estimates.csv
Model object saved to output/MNL_RespBiasTest.rds
>
> # ################################################################# #
> ##### POST-PROCESSING ####
> # ################################################################# #
>
> ### Print outputs of additional diagnostics to new output file (remember to close file writing when complete)
> apollo_sink()
Writing output to file output/MNL_RespBiasTest_additional_output.txt. Please run "apollo_sink()" again
after finishing writing results.
>
> apollo_sink()
Output is no longer being written to file.
>

[stephanehess]

Hi

your model is overspecified. I don't see where you implemented effects coding, but I suggest you use dummy coding, which is much easier (and gives the same findings). Either way, for an attribute with three levels, you can only estimate two parameters, not three

Stephane

[DTLAndreoli]

Hi Stephane,

Thanks for your swift response. I implemented the effects coding outside of apollo, so it is already part of the imported data. The experiment consists of attributes with 2 or 4 levels, hence the estimation of three parameters. So, overspecification should not be the issue here.

Any other ideas on where it might be going wrong?

Best,
Dennis

[stephanehess]

Dennis

difficult to know a priori what is causing the identification issue then. I suggest you look at adding one attribute at a time to see where it goes wrong

Stephane