ApolloChoiceModelling forum

Dear Stephane,
Thank you for Apollo.

I'm wondering why there is a significant difference in model estimates in WTP space with those in Preference space.

I can not figure out the cause of this difference.

Model 3 (MMNL-WTP) results are very different from Model 1 (MNL) and Model 2(MMNL-Preference)

I would highly appreciate if you could take a look and advise what would cause this and how to fix it.

Thank you very much in advance.

Edgar


CODES

Model 1- MNL IN PREFENCE SPACE

### Set core controls
apollo_control = list(
modelName ="MNL in Preference space-Model",
modelDescr ="Simple MNL model on mode choice SP data",
indivID ="ID"
)

# ####################################################### #
#### 2. Data loading ####
# ####################################################### #
database = read.csv("C:\\Users\\EDGAR/Desktop\\PilotDatas1.csv",header=TRUE)

### Use only SP data


# ####################################################### #
#### 3. Parameter definition ####
# ####################################################### #

### Vector of parameters, including any that are kept fixed
### during estimation
apollo_beta=c(asc_road = 0,
asc_mgr = 0,
asc_sgr = 0,
b_tt = 0,
b_c = 0,
b_rl=0,
b_ra=0

)

### Vector with names (in quotes) of parameters to be
### kept fixed at their starting value in apollo_beta.
### Use apollo_beta_fixed = c() for no fixed parameters.

apollo_fixed = c("asc_road")

# ####################################################### #
#### 4. Input validation ####
# ####################################################### #

apollo_inputs = apollo_validateInputs()

# ####################################################### #
#### 5. Likelihood definition ####
# ####################################################### #

apollo_probabilities=function(apollo_beta, apollo_inputs,
functionality="estimate"){

### Attach inputs and detach after function exit
apollo_attach(apollo_beta, apollo_inputs)
on.exit(apollo_detach(apollo_beta, apollo_inputs))

### Create list of probabilities P
P = list()

### List of utilities: these must use the same names as
### in mnl_settings, order is irrelevant.
V = list()
V[['road']] = asc_road + b_tt*time_road + b_c*cost_road + b_rl*rel_road
V[['mgr']] = asc_mgr + b_tt*time_mgr + b_c*cost_mgr + b_rl*rel_mgr + b_ra*ra_mgr
V[['sgr']] = asc_sgr + b_tt*time_sgr + b_c*cost_sgr+ b_rl*rel_sgr + b_ra* ra_sgr


### Define settings for MNL model component
mnl_settings = list(
alternatives = c(road=1, mgr=2, sgr=3),
choiceVar = choice,
V = V
)

### Compute probabilities using MNL model
P[['model']] = apollo_mnl(mnl_settings, functionality)



### Prepare and return outputs of function
P = apollo_prepareProb(P, apollo_inputs, functionality)
return(P)
}

# ####################################################### #
#### 6. Model estimation and reporting ####
# ####################################################### #

model = apollo_estimate(apollo_beta, apollo_fixed,
apollo_probabilities,
apollo_inputs,
list(writeIter=FALSE))

apollo_modelOutput(model)

apollo_modelOutput(model, modelOutput_settings = list(printPVal=1)) # for one sided t-test
apollo_modelOutput(model, modelOutput_settings = list(printPVal=2)) # for two sided t-test
apollo_saveOutput(model, saveOutput_settings = list(printPVal=1)) # for one sided t-test
apollo_saveOutput(model, saveOutput_settings = list(printPVal=2)) # for two sided t-test
?apollo_modelOutput # for more details




#apollo_saveOutput(model)
apollo_saveOutput(model)
# ####################################################### #
#### 7. Postprocessing of results ####
# ####################################################### #

apollo_saveOutput(model, saveOutput_settings = NA)



### Use the estimated model to make predictions
predictions_base = apollo_prediction(model,
apollo_probabilities,
apollo_inputs)
######apollo_combineResults()

apollo_sink()
apollo_sink()



Model 1 Results

Model name : MNL
Model description : Simple MNL model on mode choice SP data
Model run at : 2023-12-07 08:54:29.888698
Estimation method : bgw
Model diagnosis : Relative function convergence
Optimisation diagnosis : Maximum found
hessian properties : Negative definite
maximum eigenvalue : -1.177771
reciprocal of condition number : 6.62314e-09
Number of individuals : 1260
Number of rows in database : 1260
Number of modelled outcomes : 1260

Number of cores used : 1
Model without mixing

LL(start) : -1384.25
LL at equal shares, LL(0) : -1384.25
LL at observed shares, LL(C) : -1233.12
LL(final) : -823.74
Rho-squared vs equal shares : 0.4049
Adj.Rho-squared vs equal shares : 0.4006
Rho-squared vs observed shares : 0.332
Adj.Rho-squared vs observed shares : 0.3287
AIC : 1659.48
BIC : 1690.32

Estimated parameters : 6
Time taken (hh:mm:ss) : 00:00:1.1
pre-estimation : 00:00:0.61
estimation : 00:00:0.23
initial estimation : 00:00:0.19
estimation after rescaling : 00:00:0.04
post-estimation : 00:00:0.27
Iterations : 11
initial estimation : 10
estimation after rescaling : 1

Unconstrained optimisation.

Estimates:
Estimate s.e. t.rat.(0) p(2-sided) Rob.s.e. Rob.t.rat.(0) p(2-sided)
asc_road 0.000000 NA NA NA NA NA NA
asc_mgr -1.637799 0.72166 -2.269 0.023239 0.76617 -2.138 0.032546
asc_sgr -1.771418 0.59219 -2.991 0.002778 0.61441 -2.883 0.003938
b_tt -0.372319 0.06859 -5.428 5.702e-08 0.07520 -4.951 7.378e-07
b_c -0.001906 1.9895e-04 -9.583 0.000000 2.0424e-04 -9.334 0.000000
b_rl 0.047630 0.02022 2.356 0.018470 0.02082 2.288 0.022166
b_ra 1.334855 0.13954 9.566 0.000000 0.16399 8.140 4.441e-16

> apollo_saveOutput(model, saveOutput_settings = list(printPVal=1)) # for one sided t-test



Model 2 - MMNL-Lognormal preference space

# ####################################################### #
#### 1. Definition of core settings
# ####################################################### #

### Clear memory

rm(list = ls())

### Load libraries
library(apollo)

### Initialise code
apollo_initialise()

### Set core controls
apollo_control = list(
modelName ="MMNL in Preference Space",
modelDescr ="Mixed logit model on mode choice SP data",
indivID ="ID",
nCores = 4,
outputDirectory = "output"
)

# ####################################################### #
#### 2. Data loading ####
# ####################################################### #
database = read.csv("C:\\Users\\EDGAR/Desktop\\PilotDatas1.csv",header=TRUE)



# ####################################################### #
#### 3. Parameter definition ####
# ####################################################### #

### Vector of parameters, including any that are kept fixed
### during estimation

apollo_beta = c(asc_road = 0,
asc_mgr =-1.64,
asc_sgr = -1.77,
mu_b_tt = -5,
sigma_b_tt = 0,
mu_b_c = -5,
sigma_b_c = 0,
b_rl=0.05,
b_ra=1.3349

)



### Vector with names (in quotes) of parameters to be
### kept fixed at their starting value in apollo_beta.
### Use apollo_beta_fixed = c() for no fixed parameters.

apollo_fixed = c("asc_road")



### Set parameters for generating draws
apollo_draws = list(
interDrawsType = "halton",
interNDraws = 500,
interUnifDraws = c(),
interNormDraws = c("draws_tt","draws_c"),
intraDrawsType = "halton",
intraNDraws = 0,
intraUnifDraws = c(),
intraNormDraws = c()
)

### Create random parameters
apollo_randCoeff = function(apollo_beta, apollo_inputs){
randcoeff = list()

randcoeff[["b_tt"]] = -exp( mu_b_tt + sigma_b_tt * draws_tt )
randcoeff[["b_c"]] = -exp( mu_b_c + sigma_b_c * draws_c )

return(randcoeff)
}

# ################################################################# #
#### GROUP AND VALIDATE INPUTS ####
# ################################################################# #

apollo_inputs = apollo_validateInputs()

# ################################################################# #
#### DEFINE MODEL AND LIKELIHOOD FUNCTION ####
# ################################################################# #

apollo_probabilities=function(apollo_beta, apollo_inputs, functionality="estimate"){

### Function initialization: do not change the following three commands
### Attach inputs and detach after function exit
apollo_attach(apollo_beta, apollo_inputs)
on.exit(apollo_detach(apollo_beta, apollo_inputs))

### Create list of probabilities P
P = list()

### List of utilities: these must use the same names as in mnl_settings, order is irrelevant
V = list()
V[['road']] = asc_road + b_tt*time_road + b_c*cost_road + b_rl*rel_road
V[['mgr']] = asc_mgr + b_tt*time_mgr + b_c*cost_mgr + b_rl*rel_mgr + b_ra*ra_mgr
V[['sgr']] = asc_sgr + b_tt*time_sgr + b_c*cost_sgr+ b_rl*rel_sgr + b_ra* ra_sgr


### Define settings for MNL model component
mnl_settings = list(
alternatives = c(road=1, mgr=2, sgr=3),
choiceVar = choice,
V = V
)


### Compute probabilities using MNL model
P[["model"]] = apollo_mnl(mnl_settings, functionality)

### Take product across observation for same individual


### Average across inter-individual draws
P = apollo_avgInterDraws(P, apollo_inputs, functionality)

### Prepare and return outputs of function
P = apollo_prepareProb(P, apollo_inputs, functionality)
return(P)
}

# ################################################################# #
#### MODEL ESTIMATION ####
# ################################################################# #

model = apollo_estimate(apollo_beta, apollo_fixed,
apollo_probabilities, apollo_inputs)


deltaMethod_settings <- list(operation="lognormal", parName1="mu_b_c", parName2="sigma_b_c")
apollo_deltaMethod(model, deltaMethod_settings)


deltaMethod_settings <- list(operation="lognormal", parName1="mu_b_tt", parName2="sigma_b_tt")
apollo_deltaMethod(model, deltaMethod_settings)


# ################################################################# #
#### MODEL OUTPUTS ####
# ################################################################# #

# ----------------------------------------------------------------- #
#---- FORMATTED OUTPUT (TO SCREEN) ----
# ----------------------------------------------------------------- #

apollo_modelOutput(model)

# ----------------------------------------------------------------- #
#---- FORMATTED OUTPUT (TO FILE, using model name) ----
# ----------------------------------------------------------------- #

apollo_saveOutput(model)




#### 7. Postprocessing of results ####
# ####################################################### #
apollo_saveOutput(model, saveOutput_settings = NA)
### Use the estimated model to make predictions
predictions_base = apollo_prediction(model,
apollo_probabilities,
apollo_inputs)

apollo_combineResults(combineResults_settings= NULL)

apollo_sink()
apollo_sink()

Model 2 Results

Model name : MMNL in Preference Space
Model description : Mixed logit model on mode choice SP data
Model run at : 2023-12-07 08:59:43.5022
Estimation method : bgw
Model diagnosis : Relative function convergence
Optimisation diagnosis : Maximum found
hessian properties : Negative definite
maximum eigenvalue : -0.946184
reciprocal of condition number : 7.85808e-06
Number of individuals : 1260
Number of rows in database : 1260
Number of modelled outcomes : 1260

Number of cores used : 4
Number of inter-individual draws : 500 (halton)
WARNING: Inter-individual draws were used
without a panel data structure.

LL(start) : -4129.49
LL at equal shares, LL(0) : -1384.25
LL at observed shares, LL(C) : -1233.12
LL(final) : -818.54
Rho-squared vs equal shares : 0.4087
Adj.Rho-squared vs equal shares : 0.4029
Rho-squared vs observed shares : 0.3362
Adj.Rho-squared vs observed shares : 0.3313
AIC : 1653.08
BIC : 1694.19

Estimated parameters : 8
Time taken (hh:mm:ss) : 00:02:17.3
pre-estimation : 00:00:39.12
estimation : 00:00:30.62
initial estimation : 00:00:28.04
estimation after rescaling : 00:00:2.58
post-estimation : 00:01:7.56
Iterations : 20
initial estimation : 19
estimation after rescaling : 1

Unconstrained optimisation.

Estimates:
Estimate s.e. t.rat.(0) Rob.s.e. Rob.t.rat.(0)
asc_road 0.000000 NA NA NA NA
asc_mgr -1.617589 0.79049 -2.04631 0.843905 -1.9168
asc_sgr -2.254413 0.66557 -3.38721 0.678504 -3.3226
mu_b_tt -0.797232 0.17584 -4.53374 0.193030 -4.1301
sigma_b_tt 0.006742 0.15969 0.04222 0.008151 0.8272
mu_b_c -6.270572 0.12424 -50.47250 0.126086 -49.7324
sigma_b_c 0.495354 0.09304 5.32406 0.090055 5.5006
b_rl 0.056884 0.02263 2.51318 0.023136 2.4586
b_ra 1.528099 0.16308 9.37011 0.186279 8.2033

>
> # ----------------------------------------------------------------- #
> #---- FORMATTED OUTPUT (TO FILE, using model name) ----
> # ----------------------------------------------------------------- #
>

Model 3---MMNL Log Normal in WTP Space

# ####################################################### #
#### 1. Definition of core settings
# ####################################################### #

### Clear memory

rm(list = ls())

### Load libraries
library(apollo)

### Initialise code
apollo_initialise()

### Set core controls
apollo_control = list(
modelName ="MMNL",
modelDescr ="Mixed logit model on mode choice SP data",
indivID ="ID",
nCores = 4,
outputDirectory = "output"
)

# ####################################################### #
#### 2. Data loading ####
# ####################################################### #
database = read.csv("C:\\Users\\EDGAR/Desktop\\PilotDatas1.csv",header=TRUE)



# ####################################################### #
#### 3. Parameter definition ####
# ####################################################### #

### Vector of parameters, including any that are kept fixed
### during estimation

apollo_beta = c(asc_road = 0,
asc_mgr =-1.6,
asc_sgr = -2,
mu_log_b_c=-6,
sigma_log_b_c=0,
mu_log_vtt=5,
sigma_log_vtt=0,
b_rl=0.04,
b_ra=1.33

)



### Vector with names (in quotes) of parameters to be
### kept fixed at their starting value in apollo_beta.
### Use apollo_beta_fixed = c() for no fixed parameters.

apollo_fixed = c("asc_road")



### Set parameters for generating draws
apollo_draws = list(
interDrawsType = "halton",
interNDraws = 500,
interUnifDraws = c(),
interNormDraws = c("draws_vtt","draws_c"),
intraDrawsType = "halton",
intraNDraws = 0,
intraUnifDraws = c(),
intraNormDraws = c()
)

### Create random parameters
apollo_randCoeff = function(apollo_beta, apollo_inputs){
randcoeff = list()

randcoeff[["v_tt"]] = exp(mu_log_vtt + sigma_log_vtt * draws_vtt )
randcoeff[["b_c"]] = -exp( mu_log_b_c + sigma_log_b_c * draws_c )

return(randcoeff)
}

# ################################################################# #
#### GROUP AND VALIDATE INPUTS ####
# ################################################################# #

apollo_inputs = apollo_validateInputs()

# ################################################################# #
#### DEFINE MODEL AND LIKELIHOOD FUNCTION ####
# ################################################################# #

apollo_probabilities=function(apollo_beta, apollo_inputs, functionality="estimate"){

### Function initialization: do not change the following three commands
### Attach inputs and detach after function exit
apollo_attach(apollo_beta, apollo_inputs)
on.exit(apollo_detach(apollo_beta, apollo_inputs))

### Create list of probabilities P
P = list()

### List of utilities: these must use the same names as in mnl_settings, order is irrelevant
V = list()
V[['road']] = asc_road + v_tt*b_c*time_road + b_c*cost_road + b_rl*rel_road
V[['mgr']] = asc_mgr + v_tt*b_c*time_mgr + b_c*cost_mgr + b_rl*rel_mgr + b_ra*ra_mgr
V[['sgr']] = asc_sgr + v_tt*b_c*time_sgr + b_c*cost_sgr+ b_rl*rel_sgr + b_ra* ra_sgr


### Define settings for MNL model component
mnl_settings = list(
alternatives = c(road=1, mgr=2, sgr=3),
choiceVar = choice,
V = V
)


### Compute probabilities using MNL model
P[["model"]] = apollo_mnl(mnl_settings, functionality)

### Take product across observation for same individual


### Average across inter-individual draws
P = apollo_avgInterDraws(P, apollo_inputs, functionality)

### Prepare and return outputs of function
P = apollo_prepareProb(P, apollo_inputs, functionality)
return(P)
}

# ################################################################# #
#### MODEL ESTIMATION ####
# ################################################################# #

model = apollo_estimate(apollo_beta, apollo_fixed,
apollo_probabilities, apollo_inputs )

deltaMethod_settings <- list(operation="lognormal", parName1="mu_log_vtt", parName2="sigma_log_vtt")
apollo_deltaMethod(model, deltaMethod_settings)


# ################################################################# #
#### MODEL OUTPUTS ####
# ################################################################# #

# ----------------------------------------------------------------- #
#---- FORMATTED OUTPUT (TO SCREEN) ----
# ----------------------------------------------------------------- #

apollo_modelOutput(model)

# ----------------------------------------------------------------- #
#---- FORMATTED OUTPUT (TO FILE, using model name) ----
# ----------------------------------------------------------------- #

apollo_saveOutput(model)




#### 7. Postprocessing of results ####
# ####################################################### #

### Use the estimated model to make predictions
predictions_base = apollo_prediction(model,
apollo_probabilities,
apollo_inputs)

apollo_combineResults()

apollo_sink()
apollo_sink()


Model 3 Results


Model name : MMNL
Model description : Mixed logit model on mode choice SP data
Model run at : 2023-12-07 09:20:26.906073
Estimation method : bgw
Model diagnosis : Relative function convergence
Optimisation diagnosis : Maximum found
hessian properties : Negative definite
maximum eigenvalue : -5.209627
reciprocal of condition number : 5.93112e-05
Number of individuals : 1260
Number of rows in database : 1260
Number of modelled outcomes : 1260

Number of cores used : 4
Number of inter-individual draws : 500 (halton)
WARNING: Inter-individual draws were used
without a panel data structure.

LL(start) : -896.08
LL at equal shares, LL(0) : -1384.25
LL at observed shares, LL(C) : -1233.12
LL(final) : -808.02
Rho-squared vs equal shares : 0.4163
Adj.Rho-squared vs equal shares : 0.4105
Rho-squared vs observed shares : 0.3447
Adj.Rho-squared vs observed shares : 0.3399
AIC : 1632.05
BIC : 1673.16

Estimated parameters : 8
Time taken (hh:mm:ss) : 00:02:20.88
pre-estimation : 00:00:44
estimation : 00:00:29.16
initial estimation : 00:00:25.79
estimation after rescaling : 00:00:3.36
post-estimation : 00:01:7.73
Iterations : 16
initial estimation : 15
estimation after rescaling : 1

Unconstrained optimisation.

Estimates:
Estimate s.e. t.rat.(0) Rob.s.e. Rob.t.rat.(0)
asc_road 0.000000 NA NA NA NA
asc_mgr -4.318767 0.40580 -10.64271 0.438765 -9.8430
asc_sgr -0.362105 0.21996 -1.64624 0.208218 -1.7391
mu_log_b_c -5.986683 0.10577 -56.59876 0.116212 -51.5151
sigma_log_b_c -1.020471 0.12333 -8.27434 0.123080 -8.2911
mu_log_vtt 4.545351 0.06908 65.79495 0.062431 72.8055
sigma_log_vtt 0.013708 0.14060 0.09749 0.037246 0.3680
b_rl -0.008711 0.01012 -0.86060 0.009676 -0.9002
b_ra 1.921541 0.17559 10.94312 0.196578 9.7750

Edgar

when working with MNL models, WTP space and preference space are equivalent.

However, it is well known (see also the Apollo FAQ section on the website) that when working with random heterogeneity, preference space and wtp space imply different distributional assumptions for the partial derivatives of the utilities in most cases, and this will lead to differences in results. So it's not a case that wtp space and preference space are two different models (as so many people think) but that you are simply using different distributional assumptions

E.g. in your cae, in preference space, the impacts on utility of time and cost are independent negative lognormals, while in wtp space, the impact of cost and time are still both negative lognormal, but they are now correlated (the impact on cost is -exp( mu_log_b_c + sigma_log_b_c * draws_c ) while the impact on time is -exp( mu_log_b_c + sigma_log_b_c * draws_c + mu_log_vtt + sigma_log_vtt * draws_vtt)

Stephane

Thank you Stephane for the prompt response.

So in terms of reporting WTP estimates, which would I adopt for my study given the mean of the WTP in MMNL is significantly different from that of the MNL.

Not mentioning that some parameter estimates have different signs.

What are my options then?

Hi

you would never expect that MMNL and MNL give the same results. If MMNL is a superior model in terms of fit, then you would of course use that.

also, you say that the parameter estimates have different signs, but I think you might be confused here by the use of the lognormal distribution. Your parameter estimates are for the mean and sd of the log of the beta, not the actual beta. The signs of the WTP is the same

Stephane

Dear Stephane,

My concern is not only the WTP estimate being significantly different ( 190 VS 94) but also the signs on of the non random parameter estimates are different when I use the WTP space .

How else could I calculate WTP estimates in Apollo without using the WTP space given I have 2 random parameters (cost and price), and two non random parameters.

You mean ra and rl? Why don't you also include those in the WTP space spec?

in relation to calculating wtp in preference space, you would just take the ratios of the random parameters