Important: Read this before posting to this forum

  1. This forum is for questions related to the use of Apollo. We will answer some general choice modelling questions too, where appropriate, and time permitting. We cannot answer questions about how to estimate choice models with other software packages.
  2. There is a very detailed manual for Apollo available at http://www.ApolloChoiceModelling.com/manual.html. This contains detailed descriptions of the various Apollo functions, and numerous examples are available at http://www.ApolloChoiceModelling.com/examples.html. In addition, help files are available for all functions, using e.g. ?apollo_mnl
  3. Before asking a question on the forum, users are kindly requested to follow these steps:
    1. Check that the same issue has not already been addressed in the forum - there is a search tool.
    2. Ensure that the correct syntax has been used. For any function, detailed instructions are available directly in Apollo, e.g. by using ?apollo_mnl for apollo_mnl
    3. Check the frequently asked questions section on the Apollo website, which discusses some common issues/failures. Please see http://www.apollochoicemodelling.com/faq.html
    4. Make sure that R is using the latest official release of Apollo.
  4. If the above steps do not resolve the issue, then users should follow these steps when posting a question:
    1. provide full details on the issue, including the entire code and output, including any error messages
    2. posts will not immediately appear on the forum, but will be checked by a moderator first. We check the forum at least twice a week. It may thus take a couple of days for your post to appear and before we reply. There is no need to submit the post multiple times.

Estimation failed on Joint estimation on RP and SP data

Ask questions about model specifications. Ideally include a mathematical explanation of your proposed model.
Post Reply
andikawp
Posts: 2
Joined: 14 Mar 2025, 14:05

Estimation failed on Joint estimation on RP and SP data

Post by andikawp »

Dear Apollo users,

I want to use the MNL model for RP and SP data. It is a forced choice with 3 alternatives. I have prepared my dataset based on this example: https://www.apollochoicemodelling.com/f ... NL_RP_SP.r

Codes

Code: Select all

### Set core controls
apollo_control = list(
  modelName       = "MNL_RP_SP",
  modelDescr      = "RP-SP model on mode choice data",
  indivID         = "id", 
  outputDirectory = "output"
)

# ################################################################# #
#### LOAD DATA AND APPLY ANY TRANSFORMATIONS                     ####
# ################################################################# #

### Loading data from package
### if data is to be loaded from a file (e.g. called data.csv), 
### the code would be: database = read.csv("data.csv",header=TRUE)
database = read.csv("input_MNL.csv", header = TRUE)
database$Panelist_Code <- NULL
database <- database[!is.na(database$choice), ]

### for data dictionary, use ?apollo_modeChoiceData

# ################################################################# #
#### DEFINE MODEL PARAMETERS                                     ####
# ################################################################# #

### Vector of parameters, including any that are kept fixed in estimation (I deleted b_tasting and b_weekly_coffee_spending, because I do not know how to incorporate that)
apollo_beta=c(asc_1                 = 0,
              asc_2                 = 0,
              asc_3                 = 0,
              b_hedonic             = 0, # overall liking
              b_robusta             = 0,
              b_arabica             = 0,
              b_wo_sugar            = 0,
              b_half                = 0,
              b_normal              = 0,
              b_wo_label            = 0,
              b_fairtrade           = 0,
              b_organic             = 0,
              b_reward              = 0, # 3 options of rewards that they chose
              b_cluster             = 0, # 3 consumer segments based on their attitudinal data
              b_age                 = 0, # age
              b_gender              = 0, # Female (Binary)
              b_BMI                 = 0, # 5 categories
              b_urbanization        = 0, # 3 categories
              b_education           = 0, # 6 categories
              b_income              = 0, # 5 categories
              b_frequency           = 0, # 6 categories
              mu_SP                   = 1,
              mu_RP                   = 1)

### Vector with names (in quotes) of parameters to be kept fixed at their starting value in apollo_beta, use apollo_beta_fixed = c() if none
apollo_fixed = c("asc_1", "mu_SP")

# ################################################################# #
#### GROUP AND VALIDATE INPUTS                                   ####
# ################################################################# #

apollo_inputs = apollo_validateInputs()

# ################################################################# #
#### DEFINE MODEL AND LIKELIHOOD FUNCTION                        ####
# ################################################################# #

apollo_probabilities=function(apollo_beta, apollo_inputs, functionality="estimate"){
  
  ### Attach inputs and detach after function exit
  apollo_attach(apollo_beta, apollo_inputs)
  on.exit(apollo_detach(apollo_beta, apollo_inputs))
  
  ### Create list of probabilities P
  P = list()
  
  ### List of utilities (before applying scales): these must use the same names as in mnl_settings, order is irrelevant
  # PS: I have not included b_tasting? not sure how to put it also remove the asc_1
  V = list()
  
  V[["choice1"]]  = asc_1 + b_hedonic  * alt1_value + b_robusta * alt1_robusta + b_arabica * alt1_arabica + b_wo_sugar * alt1_wo_sugar + b_half * alt1_half + b_normal * alt1_normal + b_wo_label * alt1_wo_label + b_fairtrade * alt1_fairtrade + b_organic * alt1_organic + b_reward * reward + b_cluster * cluster + b_age * age + b_gender * gender + b_BMI * BMI + b_urbanization * urbanization + b_education * education + b_income * income + b_frequency * frequency  
  V[["choice2"]]  = asc_2 + b_hedonic  * alt2_value + b_robusta * alt2_robusta + b_arabica * alt2_arabica + b_wo_sugar * alt2_wo_sugar + b_half * alt2_half + b_normal * alt2_normal + b_wo_label * alt2_wo_label + b_fairtrade * alt2_fairtrade + b_organic * alt2_organic + b_reward * reward + b_cluster * cluster + b_age * age + b_gender * gender + b_BMI * BMI + b_urbanization * urbanization + b_education * education + b_income * income + b_frequency * frequency
  V[["choice3"]]  = asc_3 + b_hedonic  * alt3_value + b_robusta * alt3_robusta + b_arabica * alt3_arabica + b_wo_sugar * alt3_wo_sugar + b_half * alt3_half + b_normal * alt3_normal + b_wo_label * alt3_wo_label + b_fairtrade * alt3_fairtrade + b_organic * alt3_organic + b_reward * reward + b_cluster * cluster + b_age * age + b_gender * gender + b_BMI * BMI + b_urbanization * urbanization + b_education * education + b_income * income + b_frequency * frequency
  
  ### Compute probabilities for the RP part of the data using MNL model
  mnl_settings_SP = list(
    alternatives  = c(choice1=1, choice2=2, choice3=3), 
    avail         = list(choice1=1, choice2=1, choice3=1), # change RP into 1 here
    choiceVar     = choice, 
    utilities     = list(choice1  = mu_SP*V[["choice1"]],
                         choice2  = mu_SP*V[["choice2"]],
                         choice3  = mu_SP*V[["choice3"]]),
    rows          = (SP == 1)
  )
  
  P[["SP"]] = apollo_mnl(mnl_settings_SP, functionality)
  
  ### Compute probabilities for the SP part of the data using MNL model
  mnl_settings_RP = list(
    alternatives  = c(choice1=1, choice2=2, choice3=3), 
    avail         = list(choice1=1, choice2=1, choice3=1), 
    choiceVar     = choice, 
    utilities     = list(choice1  = mu_RP*V[["choice1"]],
                         choice2  = mu_RP*V[["choice2"]],
                         choice3  = mu_RP*V[["choice3"]]),
    rows          = (RP == 1)
  )
  
  P[["RP"]] = apollo_mnl(mnl_settings_RP, functionality)
  
  ### Combined model
  P = apollo_combineModels(P, apollo_inputs, functionality)
  
  ### Take product across observation for same individual
  P = apollo_panelProd(P, apollo_inputs, functionality)
  
  ### Prepare and return outputs of function
  P = apollo_prepareProb(P, apollo_inputs, functionality)
  return(P)
}

# ################################################################# #
#### MODEL ESTIMATION                                            ####
# ################################################################# #

model = apollo_estimate(apollo_beta, apollo_fixed, apollo_probabilities, apollo_inputs)

# ################################################################# #
#### MODEL OUTPUTS                                               ####
# ################################################################# #

# ----------------------------------------------------------------- #
#---- FORMATTED OUTPUT (TO SCREEN)                               ----
# ----------------------------------------------------------------- #

apollo_modelOutput(model)
However, I found a warning: Estimation Failed. No covariance matrix to compute. Resulting in estimation for the beta, but not the standard error, and t-statistic.

Results

Code: Select all

> model = apollo_estimate(apollo_beta, apollo_fixed, apollo_probabilities, apollo_inputs)
Preparing user-defined functions.

Testing likelihood function...
Apollo found a model component of type MNL without a componentName. The name was set to "SP"
  by default.

Overview of choices for MNL model component SP:
                                 choice1 choice2 choice3
Times available                   788.00  788.00  788.00
Times chosen                      429.00  168.00  191.00
Percentage chosen overall          54.44   21.32   24.24
Percentage chosen when available   54.44   21.32   24.24

Apollo found a model component of type MNL without a componentName. The name was set to "RP"
  by default.

Overview of choices for MNL model component RP:
                                 choice1 choice2 choice3
Times available                   788.00  788.00  788.00
Times chosen                      399.00  167.00  222.00
Percentage chosen overall          50.63   21.19   28.17
Percentage chosen when available   50.63   21.19   28.17


Pre-processing likelihood function...

Testing influence of parameters
Starting main estimation

BGW using analytic model derivatives supplied by caller...


Iterates will be written to: 
 output/MNL_RP_SP_iterations.csv
    it    nf     F            RELDF    PRELDF    RELDX    MODEL stppar
     0     1 1.731412967e+03
     1     4 1.199075843e+03 3.075e-01 2.709e-01 1.00e+00   G   1.12e-03
     2     5 6.880394007e+02 4.262e-01 4.813e-01 6.78e-01   G   1.12e-03
     3     6 5.359061542e+02 2.211e-01 3.860e-01 3.98e-01   G   7.81e-02
     4     7 5.176915061e+02 3.399e-02 2.865e-02 6.78e-02   G   -6.81e-05
     5     8 5.167308379e+02 1.856e-03 1.560e-03 1.43e-02   G   -6.81e-08
     6    11 5.166600715e+02 1.370e-04 2.645e-04 8.64e-03   S   5.50e-01
     7    12 5.166392782e+02 4.025e-05 3.565e-05 3.12e-03   G   -5.19e-04
     8    13 5.166382971e+02 1.899e-06 1.790e-06 8.75e-03   S   3.17e-13
     9    14 5.166382715e+02 4.944e-08 4.589e-08 8.94e-03   S   5.11e-13
    10    15 5.166382708e+02 1.308e-09 1.442e-09 8.98e-03   S   6.03e-13
    11    16 5.166382708e+02 3.147e-11 3.297e-11 6.04e-03   S   2.01e-13

***** Singular convergence *****

Estimated parameters:
                  Estimate
asc_1              0.00000
asc_2             -0.28649
asc_3             -0.33379
b_hedonic          2.67917
b_robusta          0.06385
b_arabica         -0.06244
b_wo_sugar        -0.22845
b_half             0.32183
b_normal          -0.30687
b_wo_label        -0.90304
b_fairtrade        0.29097
b_organic          0.32886
b_reward           0.13024
b_cluster          0.09047
b_age              1.76265
b_gender           0.05528
b_BMI              0.14068
b_urbanization     0.09226
b_education        0.27037
b_income           0.11091
b_frequency        0.15578
mu_SP              1.00000
mu_RP              0.58764

Final LL: -516.6383

WARNING: Estimation failed. No covariance matrix to compute. 
Calculating log-likelihood at equal shares (LL(0)) for applicable models...
Calculating log-likelihood at observed shares from estimation data (LL(c)) for applicable
  models...
Calculating LL of each model component...
Calculating other model fit measures

Your model was estimated using the BGW algorithm. Please acknowledge this by citing Bunch et
  al. (1993) - DOI 10.1145/151271.151279

Code: Select all

Model name                                  : MNL_RP_SP
Model description                           : RP-SP model on mode choice data
Model run at                                : 2025-03-17 13:58:46.442725
Estimation method                           : bgw
Model diagnosis                             : Singular convergence
Number of individuals                       : 197
Number of rows in database                  : 1576
Number of modelled outcomes                 : 1576
                              SP : 788
                              RP : 788

Number of cores used                        :  1 
Model without mixing

LL(start)                                   : -1731.41
LL (whole model) at equal shares, LL(0)     : -1731.41
LL (whole model) at observed shares, LL(C)  : -1603.06
LL(final, whole model)                      : -516.64
Rho-squared vs equal shares                  :  0.7016 
Adj.Rho-squared vs equal shares              :  0.6895 
Rho-squared vs observed shares               :  0.6777 
Adj.Rho-squared vs observed shares           :  0.6671 
AIC                                         :  1075.28 
BIC                                         :  1173.34 

LL(0,SP)                         : -865.71
LL(final,SP)                     : -194.33
LL(0,RP)                         : -865.71
LL(final,RP)                     : -322.31

Estimated parameters                        : 21
Time taken (hh:mm:ss)                       :  00:00:3.03 
     pre-estimation                         :  00:00:2.64 
     estimation                             :  00:00:0.36 
     post-estimation                        :  00:00:0.02 
Iterations                                  :  11 (Singular convergence) 

Unconstrained optimisation.

Estimates:
                  Estimate        s.e.   t.rat.(0)    Rob.s.e. Rob.t.rat.(0)
asc_1              0.00000          NA          NA          NA            NA
asc_2             -0.28649          NA          NA          NA            NA
asc_3             -0.33379          NA          NA          NA            NA
b_hedonic          2.67917          NA          NA          NA            NA
b_robusta          0.06385          NA          NA          NA            NA
b_arabica         -0.06244          NA          NA          NA            NA
b_wo_sugar        -0.22845          NA          NA          NA            NA
b_half             0.32183          NA          NA          NA            NA
b_normal          -0.30687          NA          NA          NA            NA
b_wo_label        -0.90304          NA          NA          NA            NA
b_fairtrade        0.29097          NA          NA          NA            NA
b_organic          0.32886          NA          NA          NA            NA
b_reward           0.13024          NA          NA          NA            NA
b_cluster          0.09047          NA          NA          NA            NA
b_age              1.76265          NA          NA          NA            NA
b_gender           0.05528          NA          NA          NA            NA
b_BMI              0.14068          NA          NA          NA            NA
b_urbanization     0.09226          NA          NA          NA            NA
b_education        0.27037          NA          NA          NA            NA
b_income           0.11091          NA          NA          NA            NA
b_frequency        0.15578          NA          NA          NA            NA
mu_SP              1.00000          NA          NA          NA            NA
mu_RP              0.58764          NA          NA          NA            NA
Then, I tried to use a simpler MNL model that only included the beta parameters for choice alternatives, omitting the sociodemographic data. However, it still produced the same error.

Simpler code

Code: Select all

database = read.csv("input_MNL.csv", header = TRUE)

### for data dictionary, use ?apollo_modeChoiceData

# ################################################################# #
#### DEFINE MODEL PARAMETERS                                     ####
# ################################################################# #

### Vector of parameters, including any that are kept fixed in estimation (I deleted b_tasting and b_weekly_coffee_spending, because I do not know how to incorporate that)
apollo_beta=c(asc_1                 = 0,
              asc_2                 = 0,
              asc_3                 = 0,
              b_hedonic             = 0,
              b_robusta             = 0,
              b_arabica             = 0,
              b_wo_sugar            = 0,
              b_half                = 0,
              b_normal              = 0,
              b_wo_label            = 0,
              b_fairtrade           = 0,
              b_organic             = 0,
              mu_SP                   = 1,
              mu_RP                   = 1)

### Vector with names (in quotes) of parameters to be kept fixed at their starting value in apollo_beta, use apollo_beta_fixed = c() if none
apollo_fixed = c("asc_1", "mu_SP")

# ################################################################# #
#### GROUP AND VALIDATE INPUTS                                   ####
# ################################################################# #

apollo_inputs = apollo_validateInputs()

# ################################################################# #
#### DEFINE MODEL AND LIKELIHOOD FUNCTION                        ####
# ################################################################# #

apollo_probabilities=function(apollo_beta, apollo_inputs, functionality="estimate"){
  
  ### Attach inputs and detach after function exit
  apollo_attach(apollo_beta, apollo_inputs)
  on.exit(apollo_detach(apollo_beta, apollo_inputs))
  
  ### Create list of probabilities P
  P = list()
  
  ### List of utilities (before applying scales): these must use the same names as in mnl_settings, order is irrelevant
  # PS: I have not included b_tasting? not sure how to put it also remove the asc_1
  V = list()
  
  V[["choice1"]]  = asc_1 + b_hedonic  * alt1_value + b_robusta * alt1_robusta + b_arabica * alt1_arabica + b_wo_sugar * alt1_wo_sugar + b_half * alt1_half + b_normal * alt1_normal + b_wo_label * alt1_wo_label + b_fairtrade * alt1_fairtrade + b_organic * alt1_organic
  V[["choice2"]]  = asc_2 + b_hedonic  * alt2_value + b_robusta * alt2_robusta + b_arabica * alt2_arabica + b_wo_sugar * alt2_wo_sugar + b_half * alt2_half + b_normal * alt2_normal + b_wo_label * alt2_wo_label + b_fairtrade * alt2_fairtrade + b_organic * alt2_organic
  V[["choice3"]]  = asc_3 + b_hedonic  * alt3_value + b_robusta * alt3_robusta + b_arabica * alt3_arabica + b_wo_sugar * alt3_wo_sugar + b_half * alt3_half + b_normal * alt3_normal + b_wo_label * alt3_wo_label + b_fairtrade * alt3_fairtrade + b_organic * alt3_organic
  
  ### Compute probabilities for the SP part of the data using MNL model
  mnl_settings_SP = list(
    alternatives  = c(choice1=1, choice2=2, choice3=3), 
    avail         = list(choice1=1, choice2=1, choice3=1), # change SP into 1 here
    choiceVar     = choice, 
    utilities     = list(choice1  = mu_SP*V[["choice1"]],
                         choice2  = mu_SP*V[["choice2"]],
                         choice3  = mu_SP*V[["choice3"]]),
    rows          = (SP == 1)
  )
  
  P[["SP"]] = apollo_mnl(mnl_settings_SP, functionality)
  
  ### Compute probabilities for the RP part of the data using MNL model
  mnl_settings_RP = list(
    alternatives  = c(choice1=1, choice2=2, choice3=3), 
    avail         = list(choice1=1, choice2=1, choice3=1), 
    choiceVar     = choice, 
    utilities     = list(choice1  = mu_RP*V[["choice1"]],
                         choice2  = mu_RP*V[["choice2"]],
                         choice3  = mu_RP*V[["choice3"]]),
    rows          = (RP == 1)
  )
  
  P[["RP"]] = apollo_mnl(mnl_settings_RP, functionality)
  
  ### Combined model
  P = apollo_combineModels(P, apollo_inputs, functionality)
  
  ### Take product across observation for same individual
  P = apollo_panelProd(P, apollo_inputs, functionality)
  
  ### Prepare and return outputs of function
  P = apollo_prepareProb(P, apollo_inputs, functionality)
  return(P)
}

# ################################################################# #
#### MODEL ESTIMATION                                            ####
# ################################################################# #

model = apollo_estimate(apollo_beta, apollo_fixed, apollo_probabilities, apollo_inputs)
Results

Code: Select all

Testing likelihood function...
Apollo found a model component of type MNL without a componentName. The name was set to "SP"
  by default.

Overview of choices for MNL model component SP:
                                 choice1 choice2 choice3
Times available                   788.00  788.00  788.00
Times chosen                      429.00  168.00  191.00
Percentage chosen overall          54.44   21.32   24.24
Percentage chosen when available   54.44   21.32   24.24

Apollo found a model component of type MNL without a componentName. The name was set to "RP"
  by default.

Overview of choices for MNL model component RP:
                                 choice1 choice2 choice3
Times available                   788.00  788.00  788.00
Times chosen                      399.00  167.00  222.00
Percentage chosen overall          50.63   21.19   28.17
Percentage chosen when available   50.63   21.19   28.17


Pre-processing likelihood function...

Testing influence of parameters
Starting main estimation

BGW using analytic model derivatives supplied by caller...


Iterates will be written to: 
 output/MNL_RP_SP_iterations.csv
    it    nf     F            RELDF    PRELDF    RELDX    MODEL stppar
     0     1 1.731412967e+03
     1     4 1.199075843e+03 3.075e-01 2.709e-01 1.00e+00   G   1.12e-03
     2     5 6.880394007e+02 4.262e-01 4.813e-01 6.78e-01   G   1.12e-03
     3     6 5.359061542e+02 2.211e-01 3.860e-01 3.98e-01   G   7.81e-02
     4     7 5.176915061e+02 3.399e-02 2.865e-02 6.78e-02   G   -6.81e-05
     5     8 5.167308379e+02 1.856e-03 1.560e-03 1.43e-02   G   -6.81e-08
     6    11 5.166600715e+02 1.370e-04 2.645e-04 8.64e-03   S   5.50e-01
     7    12 5.166392782e+02 4.025e-05 3.565e-05 3.12e-03   G   -5.19e-04
     8    13 5.166382971e+02 1.899e-06 1.790e-06 1.17e-03   S   -5.19e-07
     9    14 5.166382715e+02 4.944e-08 4.589e-08 1.48e-04   S   -5.19e-07
    10    15 5.166382708e+02 1.308e-09 1.442e-09 1.21e-05   S   -5.19e-10
    11    16 5.166382708e+02 3.147e-11 3.297e-11 2.82e-06   S   -5.19e-13

***** Singular convergence *****

Estimated parameters:
               Estimate
asc_1           0.00000
asc_2          -0.28649
asc_3          -0.33379
b_hedonic       2.67917
b_robusta       0.06315
b_arabica      -0.06315
b_wo_sugar     -0.23030
b_half          0.31997
b_normal       -0.30872
b_wo_label     -0.90232
b_fairtrade     0.29169
b_organic       0.32958
mu_SP           1.00000
mu_RP           0.58764

Final LL: -516.6383

WARNING: Estimation failed. No covariance matrix to compute. 
Calculating log-likelihood at equal shares (LL(0)) for applicable models...
Calculating log-likelihood at observed shares from estimation data (LL(c)) for applicable
  models...
Calculating LL of each model component...
Calculating other model fit measures
Then, I decided to reduce the parameter only for b_hedonic, b_robusta, and b_arabica. The model works perfectly fine. However, adding another beta parameter caused the estimation to fail, as the covariance matrix could not be computed longer.

Much simpler MNL model

Code: Select all

> apollo_modelOutput(model)
Model run by wzj545 using Apollo 0.3.4 on R 4.3.2 for Windows.
Please acknowledge the use of Apollo by citing Hess & Palma (2019)
  DOI 10.1016/j.jocm.2019.100170
  www.ApolloChoiceModelling.com

Model name                                  : MNL_RP_SP
Model description                           : RP-SP model on mode choice data
Model run at                                : 2025-03-17 14:23:01.783546
Estimation method                           : bgw
Model diagnosis                             : Relative function convergence
Optimisation diagnosis                      : Maximum found
     hessian properties                     : Negative definite
     maximum eigenvalue                     : 0
     reciprocal of condition number         : 4.1813e-14
Number of individuals                       : 197
Number of rows in database                  : 1576
Number of modelled outcomes                 : 1576
                              SP : 788
                              RP : 788

Number of cores used                        :  1 
Model without mixing

LL(start)                                   : -1731.41
LL (whole model) at equal shares, LL(0)     : -1731.41
LL (whole model) at observed shares, LL(C)  : -1603.06
LL(final, whole model)                      : -551.17
Rho-squared vs equal shares                  :  0.6817 
Adj.Rho-squared vs equal shares              :  0.6782 
Rho-squared vs observed shares               :  0.6562 
Adj.Rho-squared vs observed shares           :  0.6549 
AIC                                         :  1114.34 
BIC                                         :  1142.36 

LL(0,SP)                         : -865.71
LL(final,SP)                     : -213.43
LL(0,RP)                         : -865.71
LL(final,RP)                     : -337.74

Estimated parameters                        : 6
Time taken (hh:mm:ss)                       :  00:00:3.02 
     pre-estimation                         :  00:00:1.54 
     estimation                             :  00:00:0.49 
     post-estimation                        :  00:00:0.99 
Iterations                                  :  11  

Unconstrained optimisation.

Estimates:
             Estimate        s.e.   t.rat.(0)    Rob.s.e. Rob.t.rat.(0)
asc_1          0.0000          NA          NA          NA            NA
asc_2         -0.8861     0.15832      -5.597     0.15655     -5.660356
asc_3         -0.7808     0.14624      -5.339     0.16304     -4.788759
b_hedonic      2.7972     0.18319      15.270     0.22262     12.564959
b_robusta      2.0623   1.155e+05   1.786e-05  1275.76885      0.001616
b_arabica      2.1466   1.155e+05   1.859e-05  1275.76885      0.001683
mu_SP          1.0000          NA          NA          NA            NA
mu_RP          0.5748     0.05068      11.343     0.05987      9.601318
I wonder if this issue is due to how my design of experiments. I have randomized my coffee samples based on 3 attributes: coffee type (arabica/robusta), sugar choice(without/half/normal), and eco-label(without/organic/fairtrade). For every first alternative of sugar choice within choice sets, it is always never "normal" sugar, either "without" sugar or "half" sugar. However, the issue was not resolved when I generated random dummy data for alt1_normal.

When I added a b_wo_sugar:

Code: Select all

Preparing user-defined functions.

Testing likelihood function...
Apollo found a model component of type MNL without a componentName. The name was set to "SP"
  by default.

Overview of choices for MNL model component SP:
                                 choice1 choice2 choice3
Times available                   788.00  788.00  788.00
Times chosen                      429.00  168.00  191.00
Percentage chosen overall          54.44   21.32   24.24
Percentage chosen when available   54.44   21.32   24.24

Apollo found a model component of type MNL without a componentName. The name was set to "RP"
  by default.

Overview of choices for MNL model component RP:
                                 choice1 choice2 choice3
Times available                   788.00  788.00  788.00
Times chosen                      399.00  167.00  222.00
Percentage chosen overall          50.63   21.19   28.17
Percentage chosen when available   50.63   21.19   28.17


Pre-processing likelihood function...

Testing influence of parameters
Starting main estimation

BGW using analytic model derivatives supplied by caller...


Iterates will be written to: 
 output/MNL_RP_SP_iterations.csv
    it    nf     F            RELDF    PRELDF    RELDX    MODEL stppar
     0     1 1.731412967e+03
     1     4 1.262663611e+03 2.707e-01 2.453e-01 1.00e+00   G   -1.05e-03
     2     5 7.370061157e+02 4.163e-01 4.553e-01 6.58e-01   G   1.05e-03
     3     6 5.571602593e+02 2.440e-01 4.250e-01 4.31e-01   G   4.72e-02
     4     7 5.465721347e+02 1.900e-02 1.589e-02 3.19e-02   G   -4.40e-05
     5     8 5.459901487e+02 1.065e-03 8.861e-04 1.97e-01   G   0.00e+00
     6     9 5.459574443e+02 5.990e-05 4.654e-05 5.26e-03   G   -7.48e-10
     7    10 5.459544235e+02 5.533e-06 4.155e-06 1.54e-03   G   -7.48e-10
     8    11 5.459540249e+02 7.299e-07 7.577e-07 8.72e-04   S   -7.48e-10
     9    12 5.459540183e+02 1.220e-08 2.529e-08 2.46e-01   S   -6.23e-17
    10    13 5.459540151e+02 5.787e-09 5.605e-09 6.18e-02   G   0.00e+00
    11    14 5.459540150e+02 2.373e-10 2.096e-10 9.10e-06   G   -7.48e-13
    12    15 5.459540150e+02 1.606e-11 1.335e-11 2.53e-02   G   -7.48e-16

***** Singular convergence *****

Estimated parameters:
              Estimate
asc_1           0.0000
asc_2          -0.7424
asc_3          -0.7711
b_hedonic       2.6893
b_robusta       6.3572
b_arabica       6.3305
b_wo_sugar     -0.5776
mu_SP           1.0000
mu_RP           0.5905

Final LL: -545.954

WARNING: Estimation failed. No covariance matrix to compute. 
Calculating log-likelihood at equal shares (LL(0)) for applicable models...
Calculating log-likelihood at observed shares from estimation data (LL(c)) for applicable
  models...
Calculating LL of each model component...
Calculating other model fit measures
Thanks in advance for the help.

Best,
Andika
Top
stephanehess
Site Admin
Posts: 1351
Joined: 24 Apr 2020, 16:29

Re: Estimation failed on Joint estimation on RP and SP data

Post by stephanehess »

Hi

do the separate RP and SP models work?

Best wishes

Stephane
--------------------------------
Stephane Hess
www.stephanehess.me.uk
Top
andikawp
Posts: 2
Joined: 14 Mar 2025, 14:05

Re: Estimation failed on Joint estimation on RP and SP data

Post by andikawp »

Hi Stephane,

The SP model worked when I used parameters for the choice experiments but failed when I added socio-demographic info.

SP model - working

Code: Select all

### Use only RP data
database = subset(database,database$SP==1)

### for data dictionary, use ?apollo_modeChoiceData

# ################################################################# #
#### DEFINE MODEL PARAMETERS                                     ####
# ################################################################# #

### Vector of parameters, including any that are kept fixed in estimation (I deleted b_tasting and b_weekly_coffee_spending, because I do not know how to incorporate that)
apollo_beta=c(asc_1                 = 0,
              asc_2                 = 0,
              asc_3                 = 0,
              b_hedonic             = 0, # overall liking
              b_robusta             = 0,
              b_arabica             = 0,
              b_wo_sugar            = 0,
              b_half                = 0,
              b_normal              = 0,
              b_wo_label            = 0,
              b_fairtrade           = 0,
              b_organic             = 0
)

### Vector with names (in quotes) of parameters to be kept fixed at their starting value in apollo_beta, use apollo_beta_fixed = c() if none
apollo_fixed = c("asc_1")

# ################################################################# #
#### GROUP AND VALIDATE INPUTS                                   ####
# ################################################################# #

apollo_inputs = apollo_validateInputs()

# ################################################################# #
#### DEFINE MODEL AND LIKELIHOOD FUNCTION                        ####
# ################################################################# #

apollo_probabilities=function(apollo_beta, apollo_inputs, functionality="estimate"){
  
  ### Attach inputs and detach after function exit
  apollo_attach(apollo_beta, apollo_inputs)
  on.exit(apollo_detach(apollo_beta, apollo_inputs))
  
  ### Create list of probabilities P
  P = list()
  
  ### List of utilities (before applying scales): these must use the same names as in mnl_settings, order is irrelevant
  # PS: I have not included b_tasting? not sure how to put it also remove the asc_1
  V = list()
  
  V[["choice1"]]  = asc_1 + b_hedonic  * alt1_value + b_robusta * alt1_robusta + b_arabica * alt1_arabica + b_wo_sugar * alt1_wo_sugar + b_half * alt1_half + b_normal * alt1_normal + b_wo_label * alt1_wo_label + b_fairtrade * alt1_fairtrade + b_organic * alt1_organic   
  V[["choice2"]]  = asc_2 + b_hedonic  * alt2_value + b_robusta * alt2_robusta + b_arabica * alt2_arabica + b_wo_sugar * alt2_wo_sugar + b_half * alt2_half + b_normal * alt2_normal + b_wo_label * alt2_wo_label + b_fairtrade * alt2_fairtrade + b_organic * alt2_organic 
  V[["choice3"]]  = asc_3 + b_hedonic  * alt3_value + b_robusta * alt3_robusta + b_arabica * alt3_arabica + b_wo_sugar * alt3_wo_sugar + b_half * alt3_half + b_normal * alt3_normal + b_wo_label * alt3_wo_label + b_fairtrade * alt3_fairtrade + b_organic * alt3_organic
  
  ### Compute probabilities for the SP part of the data using MNL model
  mnl_settings_SP = list(
    alternatives  = c(choice1=1, choice2=2, choice3=3), 
    avail         = list(choice1=1, choice2=1, choice3=1), 
    choiceVar     = choice, 
    utilities     = V
  )
  
  ### Compute probabilities using MNL model
  P[["model"]] = apollo_mnl(mnl_settings_SP, functionality)
  
  ### Take product across observation for same individual
  P = apollo_panelProd(P, apollo_inputs, functionality)
  
  ### Prepare and return outputs of function
  P = apollo_prepareProb(P, apollo_inputs, functionality)
  return(P)
}

# ################################################################# #
#### MODEL ESTIMATION                                            ####
# ################################################################# #

model = apollo_estimate(apollo_beta, apollo_fixed, apollo_probabilities, apollo_inputs)

Code: Select all

> apollo_modelOutput(model)
Model run by wzj545 using Apollo 0.3.4 on R 4.3.2 for Windows.
Please acknowledge the use of Apollo by citing Hess & Palma (2019)
  DOI 10.1016/j.jocm.2019.100170
  www.ApolloChoiceModelling.com

Model name                                  : MNL_SP
Model description                           : SP model on mode choice data
Model run at                                : 2025-03-20 11:21:13.827183
Estimation method                           : bgw
Model diagnosis                             : Relative function convergence
Optimisation diagnosis                      : Non-symmetrical hessian
     hessian properties                     : Complex eigenvalues
     maximum eigenvalue                     : NA
     reciprocal of condition number         : not calculated (Hessian is not negative definite)
Number of individuals                       : 197
Number of rows in database                  : 788
Number of modelled outcomes                 : 788

Number of cores used                        :  1 
Model without mixing

LL(start)                                   : -865.71
LL at equal shares, LL(0)                   : -865.71
LL at observed shares, LL(C)                : -791.19
LL(final)                                   : -192.9
Rho-squared vs equal shares                  :  0.7772 
Adj.Rho-squared vs equal shares              :  0.7645 
Rho-squared vs observed shares               :  0.7562 
Adj.Rho-squared vs observed shares           :  0.7448 
AIC                                         :  407.79 
BIC                                         :  459.16 

Estimated parameters                        : 11
Time taken (hh:mm:ss)                       :  00:00:2.24 
     pre-estimation                         :  00:00:1.09 
     estimation                             :  00:00:0.28 
     post-estimation                        :  00:00:0.86 
Iterations                                  :  13  

Unconstrained optimisation.

Estimates:
               Estimate        s.e.   t.rat.(0)    Rob.s.e. Rob.t.rat.(0)
asc_1           0.00000          NA          NA          NA            NA
asc_2          -0.43516      0.2562    -1.69820      0.2573     -1.691089
asc_3          -0.38624      0.2350    -1.64366      0.2156     -1.791626
b_hedonic       2.72680      0.1910    14.27770      0.2402     11.354364
b_robusta      -3.29233   1.168e+05  -2.819e-05   1.151e+05    -2.861e-05
b_arabica      -3.13772   1.168e+05  -2.686e-05   1.151e+05    -2.727e-05
b_wo_sugar     -0.04184      0.4247    -0.09852      0.4184     -0.100015
b_half          0.38486      0.3446     1.11690      0.3162      1.217064
b_normal       -0.41563      0.3909    -1.06315      0.3703     -1.122432
b_wo_label      0.37951         NaN         NaN    484.8632    7.8271e-04
b_fairtrade     1.56863         NaN         NaN    409.5002      0.003831
b_organic       1.65807         NaN         NaN    409.4998      0.004049
Meanwhile, the RP model did not work. It had the same error where no covariance matrix to compute.

RP model - did not work

Code: Select all

### Use only RP data
database = subset(database,database$RP==1)

### for data dictionary, use ?apollo_modeChoiceData

# ################################################################# #
#### DEFINE MODEL PARAMETERS                                     ####
# ################################################################# #

### Vector of parameters, including any that are kept fixed in estimation (I deleted b_tasting and b_weekly_coffee_spending, because I do not know how to incorporate that)
apollo_beta=c(asc_1                 = 0,
              asc_2                 = 0,
              asc_3                 = 0,
              b_hedonic             = 0, # overall liking
              b_robusta             = 0,
              b_arabica             = 0,
              b_wo_sugar            = 0,
              b_half                = 0,
              b_normal              = 0,
              b_wo_label            = 0,
              b_fairtrade           = 0,
              b_organic             = 0
)

### Vector with names (in quotes) of parameters to be kept fixed at their starting value in apollo_beta, use apollo_beta_fixed = c() if none
apollo_fixed = c("asc_1")

# ################################################################# #
#### GROUP AND VALIDATE INPUTS                                   ####
# ################################################################# #

apollo_inputs = apollo_validateInputs()

# ################################################################# #
#### DEFINE MODEL AND LIKELIHOOD FUNCTION                        ####
# ################################################################# #

apollo_probabilities=function(apollo_beta, apollo_inputs, functionality="estimate"){
  
  ### Attach inputs and detach after function exit
  apollo_attach(apollo_beta, apollo_inputs)
  on.exit(apollo_detach(apollo_beta, apollo_inputs))
  
  ### Create list of probabilities P
  P = list()
  
  ### List of utilities (before applying scales): these must use the same names as in mnl_settings, order is irrelevant
  # PS: I have not included b_tasting? not sure how to put it also remove the asc_1
  V = list()
  
  V[["choice1"]]  = asc_1 + b_hedonic  * alt1_value + b_robusta * alt1_robusta + b_arabica * alt1_arabica + b_wo_sugar * alt1_wo_sugar + b_half * alt1_half + b_normal * alt1_normal + b_wo_label * alt1_wo_label + b_fairtrade * alt1_fairtrade + b_organic * alt1_organic
  V[["choice2"]]  = asc_2 + b_hedonic  * alt2_value + b_robusta * alt2_robusta + b_arabica * alt2_arabica + b_wo_sugar * alt2_wo_sugar + b_half * alt2_half + b_normal * alt2_normal + b_wo_label * alt2_wo_label + b_fairtrade * alt2_fairtrade + b_organic * alt2_organic
  V[["choice3"]]  = asc_3 + b_hedonic  * alt3_value + b_robusta * alt3_robusta + b_arabica * alt3_arabica + b_wo_sugar * alt3_wo_sugar + b_half * alt3_half + b_normal * alt3_normal + b_wo_label * alt3_wo_label + b_fairtrade * alt3_fairtrade + b_organic * alt3_organic
  
  ### Compute probabilities for the RP part of the data using MNL model
  mnl_settings_RP = list(
    alternatives  = c(choice1=1, choice2=2, choice3=3), 
    avail         = list(choice1=1, choice2=1, choice3=1), 
    choiceVar     = choice, 
    utilities     = V
  )
  
  ### Compute probabilities using MNL model
  P[["model"]] = apollo_mnl(mnl_settings_RP, functionality)
  
  ### Take product across observation for same individual
  P = apollo_panelProd(P, apollo_inputs, functionality)
  
  ### Prepare and return outputs of function
  P = apollo_prepareProb(P, apollo_inputs, functionality)
  return(P)
}

# ################################################################# #
#### MODEL ESTIMATION                                            ####
# ################################################################# #

model = apollo_estimate(apollo_beta, apollo_fixed, apollo_probabilities, apollo_inputs)

Code: Select all

> apollo_modelOutput(model)
Model run by wzj545 using Apollo 0.3.4 on R 4.3.2 for Windows.
Please acknowledge the use of Apollo by citing Hess & Palma (2019)
  DOI 10.1016/j.jocm.2019.100170
  www.ApolloChoiceModelling.com

Model name                                  : MNL_RP
Model description                           : RP model on mode choice data
Model run at                                : 2025-03-20 11:24:25.666745
Estimation method                           : bgw
Model diagnosis                             : Singular convergence
Number of individuals                       : 197
Number of rows in database                  : 788
Number of modelled outcomes                 : 788

Number of cores used                        :  1 
Model without mixing

LL(start)                                   : -865.71
LL at equal shares, LL(0)                   : -865.71
LL at observed shares, LL(C)                : -811.87
LL(final)                                   : -320.65
Rho-squared vs equal shares                  :  0.6296 
Adj.Rho-squared vs equal shares              :  0.6169 
Rho-squared vs observed shares               :  0.605 
Adj.Rho-squared vs observed shares           :  0.594 
AIC                                         :  663.3 
BIC                                         :  714.67 

Estimated parameters                        : 11
Time taken (hh:mm:ss)                       :  00:00:0.79 
     pre-estimation                         :  00:00:0.57 
     estimation                             :  00:00:0.21 
     post-estimation                        :  00:00:0.01 
Iterations                                  :  11 (Singular convergence) 

Unconstrained optimisation.

Estimates:
               Estimate        s.e.   t.rat.(0)    Rob.s.e. Rob.t.rat.(0)
asc_1           0.00000          NA          NA          NA            NA
asc_2          -0.12976          NA          NA          NA            NA
asc_3          -0.19175          NA          NA          NA            NA
b_hedonic       1.55309          NA          NA          NA            NA
b_robusta       0.18803          NA          NA          NA            NA
b_arabica      -0.18739          NA          NA          NA            NA
b_wo_sugar     -0.15270          NA          NA          NA            NA
b_half          0.36212          NA          NA          NA            NA
b_normal       -0.06917          NA          NA          NA            NA
b_wo_label     -0.53988          NA          NA          NA            NA
b_fairtrade     0.18807          NA          NA          NA            NA
b_organic       0.18685          NA          NA          NA            NA
I think the problem is not in the randomization of the DoE anymore, but the socio-demographic info?

Best,
Andika
Top
stephanehess
Site Admin
Posts: 1351
Joined: 24 Apr 2020, 16:29

Re: Estimation failed on Joint estimation on RP and SP data

Post by stephanehess »

Hi

your SP model doesn't seem to work either if you look at the standard errors, and esepcially b_robusta and b_arabica

Stephane
--------------------------------
Stephane Hess
www.stephanehess.me.uk
Top
Post Reply

Return to “Model specification”