Important: Read this before posting to this forum

  1. This forum is for questions related to the use of Apollo. We will answer some general choice modelling questions too, where appropriate, and time permitting. We cannot answer questions about how to estimate choice models with other software packages.
  2. There is a very detailed manual for Apollo available at http://www.ApolloChoiceModelling.com/manual.html. This contains detailed descriptions of the various Apollo functions, and numerous examples are available at http://www.ApolloChoiceModelling.com/examples.html. In addition, help files are available for all functions, using e.g. ?apollo_mnl
  3. Before asking a question on the forum, users are kindly requested to follow these steps:
    1. Check that the same issue has not already been addressed in the forum - there is a search tool.
    2. Ensure that the correct syntax has been used. For any function, detailed instructions are available directly in Apollo, e.g. by using ?apollo_mnl for apollo_mnl
    3. Check the frequently asked questions section on the Apollo website, which discusses some common issues/failures. Please see http://www.apollochoicemodelling.com/faq.html
    4. Make sure that R is using the latest official release of Apollo.
  4. If the above steps do not resolve the issue, then users should follow these steps when posting a question:
    1. provide full details on the issue, including the entire code and output, including any error messages
    2. posts will not immediately appear on the forum, but will be checked by a moderator first. We check the forum at least twice a week. It may thus take a couple of days for your post to appear and before we reply. There is no need to submit the post multiple times.

Error on MNL Estimation Result - No covariance matrix to compute.

Ask questions about errors you encouunter. Please make sure to include full details about your model specifications, and ideally your model file.
Post Reply
dpurba
Posts: 2
Joined: 09 Sep 2024, 17:28

Error on MNL Estimation Result - No covariance matrix to compute.

Post by dpurba »

Hello,

I need help with my MNL model. I received error in my estimation result: WARNING: Estimation failed. No covariance matrix to compute. I review similar issue in the forum, some discussion mentioned some potential over-specified model. But, I think I missed on how to exactly solve it. Could you help me on this? Thank you.

Here is my model. There are 148 data with 1776 obs, 6 alternatives and 8 parameters to be estimated.

Code: Select all

### Clear memory
rm(list = ls())

### Load libraries
library(apollo)

### Initialise code
apollo_initialise()

### Set core controls
apollo_control = list(
  modelName       = "MNL_modeChoice_SP_base_model_6alt",
  modelDescr      = "MNL model with 6 alts",
  indivID         = "ID", 
  outputDirectory = "output"
)

### Loading data from package
database = read.csv("choice_data_6alt_test1.csv",header=TRUE)

#### Define parameters

apollo_beta=c(
              b_1  = 0,
              b_2 = 0,
              b_3 = 0,
              b_4 = 0,
              b_5 = 0,
              b_6 = 0,
              b_7 = 0,
              b_8 = 0)

apollo_fixed = c()
apollo_inputs = apollo_validateInputs()


#### Define Model
apollo_probabilities=function(apollo_beta, apollo_inputs, functionality="estimate"){

  ### Attach inputs and detach after function exit
  apollo_attach(apollo_beta, apollo_inputs)
  on.exit(apollo_detach(apollo_beta, apollo_inputs))

  ### Create list of probabilities P
  P = list()
  
V = list()
  V[["r1"]]  =  b_1 * distance_1 + b_2 * traveltime_1 + b_3 * chargetime_1 + b_4 * queuetime_1 + b_5 * evacuationtime_1 + b_6 * soc_org_1 + b_7 * soc_des_1 + b_8 * chargeact_1
  V[["r2"]]  =  b_1 * distance_2 + b_2 * traveltime_2 + b_3 * chargetime_2 + b_4 * queuetime_2 + b_5 * evacuationtime_2 + b_6 * soc_org_2 + b_7 * soc_des_2 + b_8 * chargeact_2
  V[["r3"]]  =  b_1 * distance_3 + b_2 * traveltime_3 + b_3 * chargetime_3 + b_4 * queuetime_3 + b_5 * evacuationtime_3 + b_6 * soc_org_3 + b_7 * soc_des_3 + b_8 * chargeact_3 
  V[["r4"]]  =  b_1 * distance_4 + b_2 * traveltime_4 + b_3 * chargetime_4 + b_4 * queuetime_4 + b_5 * evacuationtime_4 + b_6 * soc_org_4 + b_7 * soc_des_4 + b_8 * chargeact_4
  V[["r5"]]  =  b_1 * distance_5 + b_2 * traveltime_5 + b_3 * chargetime_5 + b_4 * queuetime_5 + b_5 * evacuationtime_5 + b_6 * soc_org_5 + b_7 * soc_des_5 + b_8 * chargeact_5
  V[["r6"]]  =  b_1 * distance_6 + b_2 * traveltime_6 + b_3 * chargetime_6 + b_4 * queuetime_6 + b_5 * evacuationtime_6 + b_6 * soc_org_6 + b_7 * soc_des_6 + b_8 * chargeact_6
  
  
  mnl_settings = list(
    alternatives  = c(r1=1, r2=2, r3=3, r4=4, r5=5, r6=6), 
    avail         = list(r1=av_r1, r2=av_r2, r3=av_r3, r4=av_r4, r5=av_r5, r6=av_r6), 
    choiceVar     = choice,
    utilities     = V
  )
  
  P[["model"]] = apollo_mnl(mnl_settings, functionality)
  P = apollo_panelProd(P, apollo_inputs, functionality)
  P = apollo_prepareProb(P, apollo_inputs, functionality)
  return(P)
}

### Model Estimation
model = apollo_estimate(apollo_beta, apollo_fixed, apollo_probabilities, apollo_inputs)
apollo_modelOutput(model)
Here is the output:

Code: Select all

> model = apollo_estimate(apollo_beta, apollo_fixed, apollo_probabilities, apollo_inputs)
Preparing user-defined functions.

Testing likelihood function...

Overview of choices for MNL model component :
                                      r1     r2     r3      r4      r5      r6
Times available                  1776.00 1776.0 1776.0 1776.00 1776.00 1776.00
Times chosen                       73.00   87.0  158.0  434.00  355.00  669.00
Percentage chosen overall           4.11    4.9    8.9   24.44   19.99   37.67
Percentage chosen when available    4.11    4.9    8.9   24.44   19.99   37.67


Pre-processing likelihood function...

Testing influence of parameters
Starting main estimation

BGW using analytic model derivatives supplied by caller...


Iterates will be written to: 
 output/MNL_modeChoice_SP_base_model_6_nocat_iterations.csv
    it    nf     F            RELDF    PRELDF    RELDX    MODEL stppar
     0     1 3.182164817e+03
     1     4 1.921620933e+03 3.961e-01 2.519e-01 1.00e+00   G   7.61e-01
     2     5 6.961425872e+02 6.377e-01 4.497e-01 9.01e-01   G   3.53e-18
     3     6 1.007312339e+02 8.553e-01 1.021e+00 4.95e-01   S   5.29e-03
     4     7 9.483360786e+01 5.855e-02 3.212e-01 4.71e-01   S   6.40e-04
     5     8 6.652723966e+01 2.985e-01 2.767e-01 4.78e-01   S   6.40e-07
     6     9 5.180501529e+01 2.213e-01 3.446e-01 3.02e-01   S   5.65e-04
     7    10 4.141212099e+01 2.006e-01 2.744e-01 3.81e-01   S   1.01e-03
     8    11 3.762252068e+01 9.151e-02 2.159e-01 7.67e-01   S   1.01e-06
     9    12 3.091563548e+01 1.783e-01 1.902e-01 2.71e-01   S   -1.01e-06
    10    13 2.185448276e+01 2.931e-01 2.022e-01 4.30e-01   S   -1.01e-06
    11    14 1.224005274e+01 4.399e-01 3.051e-01 1.99e-01   S   -1.01e-06
    12    15 6.380432917e+00 4.787e-01 4.310e-01 1.87e-01   S   -1.01e-06
    13    16 2.531440457e+00 6.032e-01 5.375e-01 9.32e-02   S   -1.01e-06
    14    17 1.444230108e+00 4.295e-01 2.820e-01 6.07e-02   S   -1.01e-09
    15    18 6.922112915e-01 5.207e-01 3.690e-01 6.26e-02   S   -1.01e-09
    16    19 3.544495492e-01 4.879e-01 3.355e-01 5.04e-02   S   -1.01e-09
    17    20 1.767414130e-01 5.014e-01 3.486e-01 4.61e-02   S   -1.01e-09
    18    21 8.890578974e-02 4.970e-01 3.441e-01 4.20e-02   S   -1.01e-09
    19    22 4.449177070e-02 4.996e-01 3.467e-01 3.76e-02   S   -1.01e-09
    20    23 2.226779377e-02 4.995e-01 3.469e-01 3.60e-02   S   -1.01e-09
    21    24 1.111597882e-02 5.008e-01 3.488e-01 3.17e-02   S   -1.01e-09
    22    25 5.535124992e-03 5.021e-01 3.507e-01 3.20e-02   S   -1.01e-09
    23    26 2.747965079e-03 5.035e-01 3.533e-01 2.73e-02   S   -1.01e-09
    24    27 1.361533689e-03 5.045e-01 3.541e-01 5.71e-01   S   0.00e+00
    25    28 6.769534435e-04 5.028e-01 3.517e-01 3.69e-01   S   0.00e+00
    26    29 3.370110074e-04 5.022e-01 3.500e-01 1.32e-02   S   -2.52e-09
    27    30 1.683148253e-04 5.006e-01 3.477e-01 1.19e-02   S   -2.52e-09
    28    31 8.407776422e-05 5.005e-01 3.473e-01 2.50e-02   S   0.00e+00
    29    32 4.203207203e-05 5.001e-01 3.468e-01 1.25e-02   S   -2.31e-09
    30    33 2.101274254e-05 5.001e-01 3.467e-01 1.29e-02   S   -2.31e-09
    31    34 1.050619473e-05 5.000e-01 3.466e-01 3.89e-01   S   0.00e+00
    32    35 5.252994990e-06 5.000e-01 3.466e-01 6.51e-01   S   0.00e+00
    33    36 2.626502044e-06 5.000e-01 3.466e-01 3.64e-01   S   0.00e+00
    34    37 1.313250974e-06 5.000e-01 3.466e-01 1.13e-01   S   0.00e+00
    35    38 6.566269638e-07 5.000e-01 3.466e-01 1.01e-02   S   -2.34e-09
    36    39 3.283139736e-07 5.000e-01 3.466e-01 1.01e-02   S   -2.34e-12
    37    40 1.641571840e-07 5.000e-01 3.466e-01 3.36e-01   S   0.00e+00
    38    41 8.207862371e-08 5.000e-01 3.466e-01 1.94e-01   S   0.00e+00
    39    42 4.103938634e-08 5.000e-01 3.466e-01 5.40e-01   S   0.00e+00
    40    43 2.051967729e-08 5.000e-01 3.466e-01 1.42e-02   S   -2.35e-12
    41    44 1.025986163e-08 5.000e-01 3.466e-01 1.38e-02   S   -1.23e-14
    42    45 5.129923375e-09 5.000e-01 3.466e-01 1.35e-02   S   -1.23e-14
    43    46 2.564956247e-09 5.000e-01 3.466e-01 2.64e-02   S   0.00e+00
    44    47 1.282482565e-09 5.000e-01 3.466e-01 2.19e-02   S   0.00e+00
    45    48 6.412392839e-10 5.000e-01 3.466e-01 2.39e-01   S   0.00e+00
    46    49 3.206237498e-10 5.000e-01 3.466e-01 1.21e-02   S   -2.35e-12
    47    50 1.602857846e-10 5.001e-01 3.466e-01 1.69e-01   S   0.00e+00
    48    51 8.015810238e-11 4.999e-01 3.465e-01 1.16e-02   S   -8.42e-15
    49    52 4.005862309e-11 5.003e-01 3.466e-01 1.69e-01   S   0.00e+00
    50    53 2.003863742e-11 4.998e-01 3.462e-01 1.10e-02   S   0.00e+00
    51    54 1.000421967e-11 5.008e-01 3.466e-01 1.08e-02   S   -2.34e-15
    52    55 5.059064279e-12 4.943e-01 3.455e-01 1.57e-02   S   0.00e+00
    53    56 2.446487457e-12 5.164e-01 3.540e-01 1.08e-02   S   -1.34e-14
    54    57 1.275646255e-12 4.786e-01 3.318e-01 1.87e-02   S   -4.23e-16
    55    58 5.906386491e-13 5.370e-01 3.657e-01 1.26e-01   S   0.00e+00
    56    59 3.019806627e-13 4.887e-01 3.173e-01 3.70e-02   S   0.00e+00
    57    60 1.449951270e-13 5.199e-01 3.414e-01 4.12e-01   S   0.00e+00
    58    61 8.837375276e-14 3.905e-01 3.301e-01 2.15e-02   S   -1.50e-15
    59    62 7.993605777e-15 9.095e-01 5.966e-01 1.80e-02   S   3.11e-16
    60    63 1.998401444e-15 7.500e-01 9.404e-02 3.43e-04   S   8.66e-13

***** False convergence *****

Estimated parameters:
       Estimate
b_1    -0.02704
b_2    -0.05647
b_3    -0.04068
b_4    -0.03387
b_5     0.06684
b_6     1.26799
b_7     0.01509
b_8     0.45306

Final LL: 0

WARNING: Estimation failed. No covariance matrix to compute. 
Calculating log-likelihood at equal shares (LL(0)) for applicable models...
Calculating log-likelihood at observed shares from estimation data (LL(c)) for applicable
  models...
Calculating LL of each model component...
Calculating other model fit measures

Your model was estimated using the BGW algorithm. Please acknowledge this by citing Bunch
  et al. (1993) - DOI 10.1145/151271.151279
> apollo_modelOutput(model)
Model run by denissapurba_air using Apollo 0.3.3 on R 4.4.1 for Darwin.
Please acknowledge the use of Apollo by citing Hess & Palma (2019)
  DOI 10.1016/j.jocm.2019.100170
  www.ApolloChoiceModelling.com

Model name                                  : MNL_modeChoice_SP_base_model_6_nocat
Model description                           : MNL model with 6 alts no dummy catgeory
Model run at                                : 2024-09-09 11:23:39.714686
Estimation method                           : bgw
Model diagnosis                             : False convergence
Number of individuals                       : 148
Number of rows in database                  : 1776
Number of modelled outcomes                 : 1776

Number of cores used                        :  1 
Model without mixing

LL(start)                                   : -3182.16
LL at equal shares, LL(0)                   : -3182.16
LL at observed shares, LL(C)                : -2713.94
LL(final)                                   : 0
Rho-squared vs equal shares                  :  1 
Adj.Rho-squared vs equal shares              :  0.9975 
Rho-squared vs observed shares               :  1 
Adj.Rho-squared vs observed shares           :  0.9989 
AIC                                         :  16 
BIC                                         :  59.86 

Estimated parameters                        : 8
Time taken (hh:mm:ss)                       :  00:00:0.58 
     pre-estimation                         :  00:00:0.28 
     estimation                             :  00:00:0.3 
     post-estimation                        :  00:00:0 
Iterations                                  :  61 (False convergence) 

Unconstrained optimisation.

Estimates:
       Estimate        s.e.   t.rat.(0)    Rob.s.e. Rob.t.rat.(0)
b_1    -0.02704          NA          NA          NA            NA
b_2    -0.05647          NA          NA          NA            NA
b_3    -0.04068          NA          NA          NA            NA
b_4    -0.03387          NA          NA          NA            NA
b_5     0.06684          NA          NA          NA            NA
b_6     1.26799          NA          NA          NA            NA
b_7     0.01509          NA          NA          NA            NA
b_8     0.45306          NA          NA          NA            NA

stephanehess
Site Admin
Posts: 1355
Joined: 24 Apr 2020, 16:29

Re: Error on MNL Estimation Result - No covariance matrix to compute.

Post by stephanehess »

Hi

your log-likelihood goes to zero, so the model is purely determinsitic. This points towards a data issue. Can you show some cross-tabs of the attributes, and of the attributes against choices?

Stephane
--------------------------------
Stephane Hess
www.stephanehess.me.uk
dpurba
Posts: 2
Joined: 09 Sep 2024, 17:28

Re: Error on MNL Estimation Result - No covariance matrix to compute.

Post by dpurba »

Hi Stephane,
Thank you for your response. I printed screen my data here. Would it be ok?
Screenshot 2024-09-12 at 12.12.19 PM.png
Screenshot 2024-09-12 at 12.12.19 PM.png (94.37 KiB) Viewed 29288 times
Screenshot 2024-09-12 at 12.06.39 PM.png
Screenshot 2024-09-12 at 12.06.39 PM.png (78.97 KiB) Viewed 29288 times
Thanks.
stephanehess
Site Admin
Posts: 1355
Joined: 24 Apr 2020, 16:29

Re: Error on MNL Estimation Result - No covariance matrix to compute.

Post by stephanehess »

Hi

that's just showing the data. What we need is some cross-tabs, showing how the attributes relate to each other, and some statistics on what is chosen when. So e.g. look at the level of attribute 1 for the chosen vs unchosen alternatives

Stephane
--------------------------------
Stephane Hess
www.stephanehess.me.uk
Post Reply