MNL model: "Warning: Estimation failed. No covariance matrix to compute.”

[Feli]

Dear Prof. Hess,

thank you for answering questions on here! This is my first time using the apollo package (and the first time posting on a forum like this…) and I am having issues with a warning message “Warning: Estimation failed. No covariance matrix to compute.” The model is supposed to be an MNL model predicting the choice of Part of Speech (PoS - 12 levels, categorical but dummy coded as numerical) by the text type the word occurs in (text_type - 2 levels, categorical but dummy coded as numerical). I am using the newest R Version (4.4.1) and the newest version of apollo (0.3.2).
The code and the output of the model are attached below. I would appreciate any suggestion. Thank you!

ReprEx:

#dataset

ID <- c(1,2,3,4,5,6,7,8,9,10,11,12)
PoS <- c(8,2,4,11,5,1,7,9,6,10,12,3)
text_type <- c(1,2,1,2,1,2,1,2,1,2,1,2)
database <- data.frame (ID, PoS, text_type)

rm (ID, PoS, text_type)

#Apollo
library(apollo)
apollo_initialise()

apollo_control = list(
modelName = "Mixed multinomial model",
modelDescr = "Mixed logit model on Refcor (uncorrelated Lognormals in preference space)",
indivID = "ID",
nCores = 4
)

choiceAnalysis_settings <- list(
alternatives = c("noun" = 1,"proper noun"=2, "fingerspelling"=3, "SASS"=4, "pronoun"=5, "spatial reference"=6, "classifier"=7, "constructed action"=8, "list"=9, "mouthing"=10, "zero"=11, "other"=12),
choiceVar = database[,c("PoS")],
explanators = database[,c("text_type")]
)

### Run function to analyse choice data
apollo_choiceAnalysis(choiceAnalysis_settings, apollo_control, database)

apollo_beta = c(asc_1 = 0,
asc_2 = 0,
asc_3 = 0,
asc_4 = 0,
asc_5 = 0,
asc_6 = 0,
asc_7 = 0,
asc_8 = 0,
asc_9 = 0,
asc_10 = 0,
asc_11 = 0,
asc_12 = 0)

apollo_fixed = c()

apollo_inputs = apollo_validateInputs()

apollo_probabilities=function(apollo_beta, apollo_inputs, functionality="estimate"){

### Function initialisation: do not change the following three commands
### Attach inputs and detach after function exit
apollo_attach(apollo_beta, apollo_inputs)
on.exit(apollo_detach(apollo_beta, apollo_inputs))

### Create list of probabilities P
P = list()

### List of utilities: these must use the same names as in mnl_settings, order is irrelevant
V = list()
V[["alt1"]] = asc_1
V[["alt2"]] = asc_2
V[["alt3"]] = asc_3
V[["alt4"]] = asc_4
V[["alt5"]] = asc_5
V[["alt6"]] = asc_6
V[["alt7"]] = asc_7
V[["alt8"]] = asc_8
V[["alt9"]] = asc_9
V[["alt10"]] = asc_10
V[["alt11"]] = asc_11
V[["alt12"]] = asc_12

### Define settings for MNL model component
mnl_settings = list(
alternatives = c(alt1=1, alt2=2, alt3 =3, alt4 = 4, alt5 =5, alt6=6, alt7 = 7, alt8=8, alt9 =9, alt10=10, alt11 = 11, alt12=12),
#avail = list(alt1=1, alt2=1),
choiceVar = PoS,
utilities = V
)

### Compute probabilities using MNL model
P[["model"]] = apollo_mnl(mnl_settings, functionality)

### Take product across observation for same individual
# P = apollo_panelProd(P, apollo_inputs, functionality)

### Average across inter-individual draws
# P = apollo_avgInterDraws(P, apollo_inputs, functionality)

### Prepare and return outputs of function
P = apollo_prepareProb(P, apollo_inputs, functionality)
return(P)
}

model = apollo_estimate(apollo_beta, apollo_fixed, apollo_probabilities, apollo_inputs)

apollo_modelOutput(model)

Output:

> model = apollo_estimate(apollo_beta, apollo_fixed, apollo_probabilities, apollo_inputs)
Preparing user-defined functions.

Testing likelihood function...
Setting "avail" is missing, so full availability is assumed.

Overview of choices for MNL model component :
alt1 alt2 alt3 alt4 alt5 alt6 alt7 alt8 alt9 alt10 alt11 alt12
Times available 12.00 12.00 12.00 12.00 12.00 12.00 12.00 12.00 12.00 12.00 12.00 12.00
Times chosen 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00
Percentage chosen overall 8.33 8.33 8.33 8.33 8.33 8.33 8.33 8.33 8.33 8.33 8.33 8.33
Percentage chosen when available 8.33 8.33 8.33 8.33 8.33 8.33 8.33 8.33 8.33 8.33 8.33 8.33


Pre-processing likelihood function...
Creating cluster...
Preparing workers for multithreading...

Testing influence of parameters
Starting main estimation

BGW using analytic model derivatives supplied by caller...


Iterates will be written to:
*working directory*
it nf F RELDF PRELDF RELDX MODEL stppar
0 1 2.981887980e+01

***** Singular convergence *****

Estimated parameters:
Estimate
asc_1 -0.2741
asc_2 -0.2741
asc_3 -0.2741
asc_4 -0.2741
asc_5 -0.2741
asc_6 -0.2741
asc_7 -0.2741
asc_8 -0.2741
asc_9 -0.2741
asc_10 -0.2741
asc_11 -0.2741
asc_12 -0.2741

Final LL: -29.8189

WARNING: Estimation failed. No covariance matrix to compute.

Current process will resume in 3 seconds unless interrupted by the user...

Calculating log-likelihood at equal shares (LL(0)) for applicable models...
Calculating log-likelihood at observed shares from estimation data (LL(c)) for applicable models...
Calculating LL of each model component...
Calculating other model fit measures

Your model was estimated using the BGW algorithm. Please acknowledge this by citing Bunch et al. (1993) - DOI
10.1145/151271.151279
>
> apollo_modelOutput(model)
Model run by *author* using Apollo 0.3.2 on R 4.4.1 for Darwin.
Please acknowledge the use of Apollo by citing Hess & Palma (2019)
DOI 10.1016/j.jocm.2019.100170
www.ApolloChoiceModelling.com

Model name : Mixed multinomial model
Model description : Mixed logit model on Refcor (uncorrelated Lognormals in preference space)
Model run at : 2024-06-27 11:42:44.826078
Estimation method : bgw
Model diagnosis : Singular convergence
Number of individuals : 12
Number of rows in database : 12
Number of modelled outcomes : 12

Number of cores used : 4
Model without mixing

LL(start) : -29.82
LL at equal shares, LL(0) : -29.82
LL at observed shares, LL(C) : -29.82
LL(final) : -29.82
Rho-squared vs equal shares : 0
Adj.Rho-squared vs equal shares : -0.4024
Rho-squared vs observed shares : 0
Adj.Rho-squared vs observed shares : -0.0335
AIC : 83.64
BIC : 89.46

Estimated parameters : 12
Time taken (hh:mm:ss) : 00:00:10.8
pre-estimation : 00:00:7.62
estimation : 00:00:3.15
post-estimation : 00:00:0.03
Iterations : 1 (Singular convergence)

Unconstrained optimisation.

Estimates:
Estimate s.e. t.rat.(0) Rob.s.e. Rob.t.rat.(0)
asc_1 -0.2741 NA NA NA NA
asc_2 -0.2741 NA NA NA NA
asc_3 -0.2741 NA NA NA NA
asc_4 -0.2741 NA NA NA NA
asc_5 -0.2741 NA NA NA NA
asc_6 -0.2741 NA NA NA NA
asc_7 -0.2741 NA NA NA NA
asc_8 -0.2741 NA NA NA NA
asc_9 -0.2741 NA NA NA NA
asc_10 -0.2741 NA NA NA NA
asc_11 -0.2741 NA NA NA NA
asc_12 -0.2741 NA NA NA NA

[stephanehess]

Hi

you have two problems here:

1. you have overspecified the model. With 12 alternatives, you can only estimate 11 constants
2. your data setup looks odd. you only have 12 observations, with each of your 12 alternatives chosen exactly once (which is why your ASCs all go to the same value)

Stephane

[Feli]

Dear Stephane,

regarding the second problem: the data I posted here is only a minimal example that mirrors the structure of my actual data and that reproduces the issues with the model. My actual data is a table consisting of about 3000 rows and 30 columns (though only two of the columns are relevant to the model at this point).

Regarding the first problem: I'm afraid I don't quite unterstand what you mean. Overspecification of models usually means that there are too many predictor variables, right? I only have one predictor at this point. I do see that there are 12 estimated constants in the output, when there should only be 11, but I'm not sure which part of my code to adjust to fix that. Could you elaborate, please?

Thank you!

Feli

[stephanehess]

Hi

the relevant part is apollo_fixed. This is explained in the manual and online examples too

Stephane