Important: Read this before posting to this forum

  1. This forum is for questions related to the use of Apollo. We will answer some general choice modelling questions too, where appropriate, and time permitting. We cannot answer questions about how to estimate choice models with other software packages.
  2. There is a very detailed manual for Apollo available at http://www.ApolloChoiceModelling.com/manual.html. This contains detailed descriptions of the various Apollo functions, and numerous examples are available at http://www.ApolloChoiceModelling.com/examples.html. In addition, help files are available for all functions, using e.g. ?apollo_mnl
  3. Before asking a question on the forum, users are kindly requested to follow these steps:
    1. Check that the same issue has not already been addressed in the forum - there is a search tool.
    2. Ensure that the correct syntax has been used. For any function, detailed instructions are available directly in Apollo, e.g. by using ?apollo_mnl for apollo_mnl
    3. Check the frequently asked questions section on the Apollo website, which discusses some common issues/failures. Please see http://www.apollochoicemodelling.com/faq.html
    4. Make sure that R is using the latest official release of Apollo.
  4. If the above steps do not resolve the issue, then users should follow these steps when posting a question:
    1. provide full details on the issue, including the entire code and output, including any error messages
    2. posts will not immediately appear on the forum, but will be checked by a moderator first. We check the forum at least twice a week. It may thus take a couple of days for your post to appear and before we reply. There is no need to submit the post multiple times.

Unexpected sign for remuneration coefficient

Ask questions about the results reported after estimation. If the output includes errors, please include your model code if possible.
Post Reply
nicolas_p
Posts: 3
Joined: 06 Oct 2025, 15:40

Unexpected sign for remuneration coefficient

Post by nicolas_p »

Hi,

I am working on a DCE with a willingness to accept setting. So instead of price there is a hypothetical financial remuneration "b_rmn" . The problem is that I get a negative and significant coefficient for this attribute and I would expect a positive sign. I would expect a positive sign for "b_dc" as it refers to offered discounts but my main concern is not the coefficient for remuneration

This is the code that I am using in the MNL model:

Code: Select all

### Clear memory
rm(list = ls())

### Load Apollo library
library(apollo)

### Initialise code
apollo_initialise()

### Set core controls
apollo_control = list(
  modelName       = "MNL_SC_nwsm",
  modelDescr      = "Simple MNL model on SP data",
  indivID         = "ID",
  outputDirectory = "output"
)


# with ID, you tell R, what is the ID variable 

# ################################################################# #
#### LOAD DATA AND APPLY ANY TRANSFORMATIONS                     ####
# ################################################################# #

### Loading data from package
database <- readRDS("C:\\Users\\posso-gonzalez.n\\Documents\\PhD\\Ch2\\jointdataSC.rds")

class(database$Time)
database$Time <- as.numeric(database$Time)
database <- database %>%
  filter(Time >= 300)
database <- database %>%
  filter(AgeGr != 1)
database$spd_0 <- as.integer(database$spd_0)




### Vector of parameters, including any that are kept fixed in estimation
apollo_beta = c(asc = 0,
                b_vhm   = 0,
                b_elp    = 0,
                b_cpo    = 0,
                b_rn    = 0, 
                b_dc    = 0, 
                b_flx     =0, 
                b_spd    =0, 
                b_exm    =0, 
                b_rmn    =0
)

### Vector with names (in quotes) of parameters to be kept fixed at their starting value in apollo_beta, use apollo_beta_fixed = c() if none
apollo_fixed = c()

# ################################################################# #
#### GROUP AND VALIDATE INPUTS                                   ####
# ################################################################# #

apollo_inputs = apollo_validateInputs()

# ################################################################# #
#### DEFINE MODEL AND LIKELIHOOD FUNCTION                        ####
# ################################################################# #

apollo_probabilities=function(apollo_beta, apollo_inputs, functionality="estimate"){
  
  ### Function initialisation: do not change the following three commands
  ### Attach inputs and detach after function exit
  apollo_attach(apollo_beta, apollo_inputs)
  on.exit(apollo_detach(apollo_beta, apollo_inputs))
  
  ### Create list of probabilities P
  P = list()
  
  ### List of utilities: these must use the same names as in mnl_settings, order is irrelevant
  V = list()
  V[["alt0"]] =                                                                                                                                                                                b_spd * spd_0 
  V[["alt1"]] = asc + b_vhm * (cnt_1 == "vh_m") + b_elp * (cnt_1 == "el_p") + b_cpo * (cnt_1 == "cp_o") + b_rn * (adv_1 == "50_rn") +  b_dc * (adv_1 == "10_dc") +  b_flx * (adv_1 == "flx") + b_spd * spd_1 + b_exm * exm_1 + b_rmn * rmn_1
  V[["alt2"]] = asc + b_vhm * (cnt_2 == "vh_m") + b_elp * (cnt_2 == "el_p") + b_cpo * (cnt_2 == "cp_o") + b_rn * (adv_2 == "50_rn") +  b_dc * (adv_2 == "20_dc") +  b_flx * (adv_2 == "flx") + b_spd * spd_2 + b_exm * exm_2 + b_rmn * rmn_2
  
  #####Im here######
  ### Define settings for MNL model component
  mnl_settings = list(
    alternatives  = c(alt0=0, alt1=1, alt2=2), #choice coding of the variable 
    avail         = list(alt0=1, alt1=1, alt2=1), ####check if 1 means the choice is available
    choiceVar     = choice,
    utilities     = V
  )
  
  ### Compute probabilities using MNL model
  P[["model"]] = apollo_mnl(mnl_settings, functionality)
  
  ### Take product across observation for same individual
  P = apollo_panelProd(P, apollo_inputs, functionality)
  
  ### Prepare and return outputs of function
  P = apollo_prepareProb(P, apollo_inputs, functionality)
  return(P)
}

# ################################################################# #
#### MODEL ESTIMATION                                            ####
# ################################################################# #

model = apollo_estimate(apollo_beta, apollo_fixed,apollo_probabilities, apollo_inputs)

# ################################################################# #
#### MODEL OUTPUTS                                               ####
# ################################################################# #

# ----------------------------------------------------------------- #
#---- FORMATTED OUTPUT (TO SCREEN)                               ----
# ----------------------------------------------------------------- #

apollo_modelOutput(model)

# ----------------------------------------------------------------- #
#---- FORMATTED OUTPUT (TO FILE, using model name)               ----
# ----------------------------------------------------------------- #

apollo_saveOutput(model)



### Print outputs of additional diagnostics to new output file (remember to close file writing when complete)
apollo_sink()
These are the results:

Code: Select all

Model name                                  : MNL_SC_nwsm
Model description                           : Simple MNL model on SP data
Model run at                                : 2025-12-31 11:23:20.252316
Estimation method                           : bgw
Estimation diagnosis                        : Relative function convergence
Optimisation diagnosis                      : Maximum found
     hessian properties                     : Negative definite
     maximum eigenvalue                     : -202.913671
     reciprocal of condition number         : 1.73622e-05
Number of individuals                       : 2357
Number of rows in database                  : 14142
Number of modelled outcomes                 : 14142

Number of cores used                        :  1 
Model without mixing

LL(start)                                   : -15536.57
LL at equal shares, LL(0)                   : -15536.57
LL at observed shares, LL(C)                : -15123.98
LL(final)                                   : -14941.03
Rho-squared vs equal shares                  :  0.0383 
Adj.Rho-squared vs equal shares              :  0.0377 
Rho-squared vs observed shares               :  0.0121 
Adj.Rho-squared vs observed shares           :  0.0116 
AIC                                         :  29902.07 
BIC                                         :  29977.64 

Estimated parameters                        : 10
Time taken (hh:mm:ss)                       :  00:00:4.75 
     pre-estimation                         :  00:00:1.93 
     estimation                             :  00:00:0.59 
     post-estimation                        :  00:00:2.23 
Iterations                                  :  6  

Unconstrained optimisation.

Estimates:
         Estimate        s.e.   t.rat.(0)    Rob.s.e. Rob.t.rat.(0)
asc      0.075520    0.052417      1.4408    0.061428        1.2294
b_vhm    0.005746    0.037777      0.1521    0.037699        0.1524
b_elp    0.070133    0.035211      1.9918    0.037440        1.8732
b_cpo   -0.056804    0.039360     -1.4432    0.036962       -1.5368
b_rn     0.127496    0.033809      3.7711    0.030374        4.1975
b_dc    -0.139167    0.046128     -3.0170    0.047752       -2.9144
b_flx    0.095625    0.033794      2.8296    0.030334        3.1524
b_spd    0.010065  5.6020e-04     17.9674  6.0220e-04       16.7144
b_exm    0.117746    0.024568      4.7927    0.025350        4.6448
b_rmn   -0.010881    0.001682     -6.4687    0.002042       -5.3279


Overview of choices for MNL model component :
                                     alt0     alt1     alt2
Times available                  14142.00 14142.00 14142.00
Times chosen                      6280.00  3538.00  4324.00
Percentage chosen overall           44.41    25.02    30.58
Percentage chosen when available    44.41    25.02    30.58


Thanks a lot in advance
stephanehess
Site Admin
Posts: 1348
Joined: 24 Apr 2020, 16:29

Re: Unexpected sign for remuneration coefficient

Post by stephanehess »

Hi

this could well have to do with your first alternative, which I assume is an opt-out? What were the levels for that?

Stephane
--------------------------------
Stephane Hess
www.stephanehess.me.uk
nicolas_p
Posts: 3
Joined: 06 Oct 2025, 15:40

Re: Unexpected sign for remuneration coefficient

Post by nicolas_p »

Thank you for your reply. So there are two alternatives involving a smart charging contract (alt_1, alt_2) and an opt-out option (alt_0). The levels for the remuneration are 5, 10, 15, and 25 euros as a monthly financial compensation. I know that probably these amounts are not enough to affect much the choices of individuals and this may cause this problem. So I tried with equality constraint latent class to see if but the results are not fully satisfying. With different starting values, it converges to two models, the one with the best LL keeps the significance in the negative sign for remuneration, while in the other model, the remuneration coefficient is not significant. I would really appreciate any recommendation to deal with this problem.

Nicolas
stephanehess
Site Admin
Posts: 1348
Joined: 24 Apr 2020, 16:29

Re: Unexpected sign for remuneration coefficient

Post by stephanehess »

To diagnose it further, can you try a model with a categorical treatment of price, just to see the non-linearity

you might also want to think about a nested logit model
--------------------------------
Stephane Hess
www.stephanehess.me.uk
nicolas_p
Posts: 3
Joined: 06 Oct 2025, 15:40

Re: Unexpected sign for remuneration coefficient

Post by nicolas_p »

Thanks a lot for your reply. These are the reuslts with nest logit with linear remuneration

Code: Select all

Model name                                  : NL_SC
Model description                           : Simple MNL model on SP data
Model run at                                : 2026-02-05 17:56:12.445433
Estimation method                           : bgw
Estimation diagnosis                        : Relative function convergence
Optimisation diagnosis                      : Maximum found
     hessian properties                     : Negative definite
     maximum eigenvalue                     : -71.080336
     reciprocal of condition number         : 4.06846e-06
Number of individuals                       : 2357
Number of rows in database                  : 14142
Number of modelled outcomes                 : 14142

Number of cores used                        :  1 
Model without mixing

LL(start)                                   : -15536.57
LL at equal shares, LL(0)                   : -15536.57
LL at observed shares, LL(C)                : -15123.98
LL(final)                                   : -14934.93
Rho-squared vs equal shares                  :  0.0387 
Adj.Rho-squared vs equal shares              :  0.038 
Rho-squared vs observed shares               :  0.0125 
Adj.Rho-squared vs observed shares           :  0.0119 
AIC                                         :  29891.85 
BIC                                         :  29974.98 

Estimated parameters                        : 11
Time taken (hh:mm:ss)                       :  00:00:13.95 
     pre-estimation                         :  00:00:1.3 
     estimation                             :  00:00:1.7 
     post-estimation                        :  00:00:10.96 
Iterations                                  :  8  

Unconstrained optimisation.

Estimates:
             Estimate        s.e.   t.rat.(0)    Rob.s.e. Rob.t.rat.(0)
asc          0.163237    0.037613      4.3399    0.047780        3.4164
b_vhm        0.006670    0.022742      0.2933    0.022888        0.2914
b_elp        0.041449    0.022632      1.8314    0.023336        1.7761
b_cpo       -0.053589    0.024798     -2.1610    0.023347       -2.2953
b_rn         0.083880    0.025807      3.2502    0.022547        3.7203
b_dc        -0.106425    0.033187     -3.2069    0.029746       -3.5779
b_flx        0.053323    0.023775      2.2428    0.020628        2.5850
b_spd        0.006023    0.001211      4.9716    0.001075        5.6035
b_exm        0.076321    0.019887      3.8377    0.018123        4.2113
b_rmn       -0.006518    0.001575     -4.1384    0.001531       -4.2569
lambda_SC    0.561639    0.113686      4.9403    0.096425        5.8246

Nesting structure for NL model component :
Nest: root (1)
|----Alternative: alt0
'-Nest: alt1_alt2 (0.5616)
   |-Alternative: alt1
   '-Alternative: alt2


Overview of choices for NL model component :
                                     alt0     alt1     alt2
Times available                  14142.00 14142.00 14142.00
Times chosen                      6280.00  3538.00  4324.00
Percentage chosen overall           44.41    25.02    30.58
Percentage chosen when available    44.41    25.02    30.58

These are the results for the nested logit using remuneration as a categorical variable and using 5 as the reference category.

Code: Select all

Model name                                  : NL_SC_rmct
Model description                           : Simple MNL model on SP data
Model run at                                : 2026-02-05 18:35:21.364565
Estimation method                           : bgw
Estimation diagnosis                        : Relative function convergence
Optimisation diagnosis                      : Maximum found
     hessian properties                     : Negative definite
     maximum eigenvalue                     : -67.067856
     reciprocal of condition number         : 3.84292e-06
Number of individuals                       : 2357
Number of rows in database                  : 14142
Number of modelled outcomes                 : 14142

Number of cores used                        :  1 
Model without mixing

LL(start)                                   : -15536.57
LL at equal shares, LL(0)                   : -15536.57
LL at observed shares, LL(C)                : -15123.98
LL(final)                                   : -14929.55
Rho-squared vs equal shares                  :  0.0391 
Adj.Rho-squared vs equal shares              :  0.0382 
Rho-squared vs observed shares               :  0.0129 
Adj.Rho-squared vs observed shares           :  0.0121 
AIC                                         :  29885.09 
BIC                                         :  29983.33 

Estimated parameters                        : 13
Time taken (hh:mm:ss)                       :  00:00:35.34 
     pre-estimation                         :  00:00:2.11 
     estimation                             :  00:00:4.97 
     post-estimation                        :  00:00:28.26 
Iterations                                  :  7  

Unconstrained optimisation.

Estimates:
             Estimate        s.e.   t.rat.(0)    Rob.s.e. Rob.t.rat.(0)
asc          0.110121    0.042012      2.6211    0.050194        2.1939
b_vhm        0.007680    0.022529      0.3409    0.022996        0.3340
b_elp        0.041964    0.023486      1.7868    0.024693        1.6995
b_cpo       -0.041033    0.023803     -1.7239    0.022855       -1.7954
b_rn         0.146890    0.030860      4.7599    0.026234        5.5993
b_dc         0.039595    0.023689      1.6715    0.024894        1.5905
b_flx        0.094086    0.026361      3.5692    0.023142        4.0656
b_spd        0.005882    0.001133      5.1937    0.001057        5.5632
b_exm        0.061085    0.018216      3.3533    0.018239        3.3492
b_rmn10     -0.072739    0.023910     -3.0422    0.021752       -3.3440
b_rmn15     -0.147659    0.031947     -4.6221    0.028739       -5.1379
b_rmn25     -0.113798    0.029390     -3.8720    0.030649       -3.7129
lambda_SC    0.554568    0.108305      5.1204    0.097344        5.6970

Nesting structure for NL model component :
Nest: root (1)
|----Alternative: alt0
'-Nest: alt1_alt2 (0.5546)
   |-Alternative: alt1
   '-Alternative: alt2


Overview of choices for NL model component :
                                     alt0     alt1     alt2
Times available                  14142.00 14142.00 14142.00
Times chosen                      6280.00  3538.00  4324.00
Percentage chosen overall           44.41    25.02    30.58
Percentage chosen when available    44.41    25.02    30.58

In this DCE, there are 3 other non monetary attributes that aim to attract individuals to choose the inside options. Individuals in general chose more times the alternative with lower remuneration between the two inside options. I have some possible causes in mind for the problem with the coefficient. Maybe a kind of warm glowing so individuals do not want to be financially compensated for adapting this technology (smart charging for electric vehicles) or mistrust towards the technology and privacy concerns. Or maybe they value much more the non-monetary values in comparison to the financial compensation. Or maybe there is confounding problem in the design of the DCE or the specification of the utility function. Or a combination of them. The problem is that I do not know how to test or disentangle them.
stephanehess
Site Admin
Posts: 1348
Joined: 24 Apr 2020, 16:29

Re: Unexpected sign for remuneration coefficient

Post by stephanehess »

Hi

this seems more of an issue of how people reacted to the attribute, rather than a modelling issue per se. Did you run a focus group or pilot at all?

Stephane
--------------------------------
Stephane Hess
www.stephanehess.me.uk
Post Reply