Important: Read this before posting to this forum

  1. This forum is for questions related to the use of Apollo. We will answer some general choice modelling questions too, where appropriate, and time permitting. We cannot answer questions about how to estimate choice models with other software packages.
  2. There is a very detailed manual for Apollo available at http://www.ApolloChoiceModelling.com/manual.html. This contains detailed descriptions of the various Apollo functions, and numerous examples are available at http://www.ApolloChoiceModelling.com/examples.html. In addition, help files are available for all functions, using e.g. ?apollo_mnl
  3. Before asking a question on the forum, users are kindly requested to follow these steps:
    1. Check that the same issue has not already been addressed in the forum - there is a search tool.
    2. Ensure that the correct syntax has been used. For any function, detailed instructions are available directly in Apollo, e.g. by using ?apollo_mnl for apollo_mnl
    3. Check the frequently asked questions section on the Apollo website, which discusses some common issues/failures. Please see http://www.apollochoicemodelling.com/faq.html
    4. Make sure that R is using the latest official release of Apollo.
  4. If the above steps do not resolve the issue, then users should follow these steps when posting a question:
    1. provide full details on the issue, including the entire code and output, including any error messages
    2. posts will not immediately appear on the forum, but will be checked by a moderator first. This may take a day or two at busy times. There is no need to submit the post multiple times.

huge difference between results from a WTP-space and Preference-space model

Ask questions about the results reported after estimation. If the output includes errors, please include your model code if possible.
Post Reply
arohamirai
Posts: 18
Joined: 28 Jun 2020, 16:26

huge difference between results from a WTP-space and Preference-space model

Post by arohamirai »

Dear Stephane,

Thank you very much for inventing the Apollo. I’m also the attendees of your previous summer and winter DCE courses.

I was estimating a MMNL WTP space model and had a problem.

The DCE
Attributes: h (health, number hospital admissions due to air pollution), v (number of bad visibility days) and c(cost of enhanced air pollution policy)
Available options: 2 unlabelled alternatives + 1 status quo

The model
It’s an asymmetric model in which the health parameter is split into two according to whether the attribute is described as an improvement (i.e., h_inc) or a deterioration (i.e., h_dec) from a reference point. The idea is similar to Hess et al. (2008). Other attributes are specified as linear.

Hess, S., Rose, J. M., & Hensher, D. A. (2008). Asymmetric preference formation in willingness to pay estimates in discrete choice models. Transportation Research Part E: Logistics and Transportation Review, 44(5), 847-863.

I want to estimate MMNL WTP space model. What I did was: (a) Model 1: starting from a basic MNL preference space model and calculate the WTP; (b) Model 2: estimate a MNL WTP space model using WTP values from Model 1 as starting values; (c) Model 3: estimate MMNL WTP space model using WTP values from Model 2 as starting values; (d) Model 4: as robustness check, I also estimate the same MMNL with preference space and compare the WTPs with those from Model 3.

Questions
(1) Theoretically, WTPs from Model 1 should be very similar to those from Model 2, but the results suggested that WTPs of Model 2 are nearly doubled compared with those in Model 1. I’m not sure why it has happened?
(2) Again, WTPs from Model 3 and Model 4 should be similar, but they are quite different. I also noticed that the model fit of the WTP-space model is much worse.

I tried a set of different starting values but none of them made the LL or WTPs of the two models similar.

Could you kindly tell me what could be the potential reason of the inconsistency? One observation is that the cost attribute is somehow not significant, could this be reason of the instable WTP estimates?

Please see the CODES below, followed by the estimation RESULTS, I can provide the data if needed . Thank you very much.
Best wishes,
Tim


CODES

# ################################################################# #
# ################################################################# #
# ################################################################# #
##################MODEL 1: MNL_PREFERENCE SPACE######################
# ################################################################# #
# ################################################################# #
# ################################################################# #

# ################################################################# #
#### LOAD LIBRARY AND DEFINE CORE SETTINGS ####
# ################################################################# #

### Clear memory
rm(list = ls())

### Load Apollo library
library(apollo)

### Initialise code
apollo_initialise()

### Set core controls
apollo_control = list(
modelName ="MNL",
modelDescr ="MNL",
indivID ="id1",
nCores = 4,
mixing = FALSE,
noDiagnostics = TRUE
)

# ################################################################# #
#### LOAD DATA AND APPLY ANY TRANSFORMATIONS ####
# ################################################################# #
#load the data
database = read.csv("R_T1T3.csv",header=TRUE)

#rename
names(database)[names(database)=="id"] <- "ID"

names(database)[names(database)=="E_h_1_inc"]<-"h_1_inc"
names(database)[names(database)=="E_h_2_inc"]<-"h_2_inc"
names(database)[names(database)=="E_h_1_dec"]<-"h_1_dec"
names(database)[names(database)=="E_h_2_dec"]<-"h_2_dec"

#rescale data
database$c_1=database$c_1/100
database$c_2=database$c_2/100
database$c_3=database$c_3/100

#only use one of the datasets
database= subset(database, database$same_answersG==0&same_answersQ==0&T3_dummy==0)

# ################################################################# #
#### DEFINE MODEL PARAMETERS ####
# ################################################################# #

### Vector of parameters, including any that are kept fixed in estimation
apollo_beta = c(

asc_III = 0,

hinc = 0,

hdec = 0,

v = 0,

c = 0

)

### Vector with names (in quotes) of parameters to be kept fixed at their starting value in apollo_beta, use apollo_beta_fixed = c() if none
apollo_fixed = c( )

# ################################################################# #
#### GROUP AND VALIDATE INPUTS ####
# ################################################################# #

apollo_inputs = apollo_validateInputs()

# ################################################################# #
#### DEFINE MODEL AND LIKELIHOOD FUNCTION ####
# ################################################################# #

apollo_probabilities=function(apollo_beta, apollo_inputs, functionality="estimate"){

### Attach inputs and detach after function exit
apollo_attach(apollo_beta, apollo_inputs)
on.exit(apollo_detach(apollo_beta, apollo_inputs))

### Create list of probabilities P
P = list()


### Compute utilities
V=list()
V[['alt1']] = hinc*h_1_inc + hdec*h_1_dec + v*v_1 + c*c_1


V[['alt2']] = hinc*h_2_inc + hdec*h_2_dec + v*v_2 + c*c_2

V[['alt3']] = asc_III + v*v_3 + c*c_3

### Define settings for MNL model component that are generic across classes
mnl_settings = list(
alternatives = c(alt1=1, alt2=2,alt3=3),
avail = list(alt1=1, alt2=1,alt3=1),
choiceVar = choice,
V = V

)

### Compute probabilities using MNL model
P[['model']] = apollo_mnl(mnl_settings, functionality)

### Take product across observation for same individual
P = apollo_panelProd(P, apollo_inputs, functionality)

### Average across inter-individual draws
# P = apollo_avgInterDraws(P, apollo_inputs, functionality)

### Prepare and return outputs of function
P = apollo_prepareProb(P, apollo_inputs, functionality)
return(P)
}

# ################################################################# #
#### MODEL ESTIMATION ####
# ################################################################# #

model = apollo_estimate(apollo_beta, apollo_fixed,
apollo_probabilities, apollo_inputs)

# ################################################################# #
#### MODEL OUTPUTS ####
# ################################################################# #

# ----------------------------------------------------------------- #
#---- FORMATTED OUTPUT (TO SCREEN) ----
# ----------------------------------------------------------------- #

apollo_modelOutput(model)

##calculate WTP using delta method and use these values as starting values for Model 2
deltaMethod_settings=list(operation="ratio",parName1="hinc",parName2="c")
apollo_deltaMethod(model, deltaMethod_settings)
deltaMethod_settings=list(operation="ratio",parName1="hdec",parName2="c")
apollo_deltaMethod(model, deltaMethod_settings)
deltaMethod_settings=list(operation="ratio",parName1="v",parName2="c")
apollo_deltaMethod(model, deltaMethod_settings)



# ################################################################# #
# ################################################################# #
# ################################################################# #
##################MODEL 2: MNL_WTP SPACE############################
# ################################################################# #
# ################################################################# #
# ################################################################# #

# ################################################################# #
#### LOAD LIBRARY AND DEFINE CORE SETTINGS ####
# ################################################################# #

### Clear memory
rm(list = ls())

### Load Apollo library
library(apollo)

### Initialise code
apollo_initialise()

### Set core controls
apollo_control = list(
modelName ="MNL_WTP-SPACE",
modelDescr ="MNL_WTP-SPACE",
indivID ="id1",
nCores = 4,
mixing = FALSE,
noDiagnostics = TRUE
)

# ################################################################# #
#### LOAD DATA AND APPLY ANY TRANSFORMATIONS ####
# ################################################################# #
#load the data
database = read.csv("R_T1T3.csv",header=TRUE)

#rename
names(database)[names(database)=="id"] <- "ID"

names(database)[names(database)=="E_h_1_inc"]<-"h_1_inc"
names(database)[names(database)=="E_h_2_inc"]<-"h_2_inc"
names(database)[names(database)=="E_h_1_dec"]<-"h_1_dec"
names(database)[names(database)=="E_h_2_dec"]<-"h_2_dec"

#rescale data
database$c_1=database$c_1/100
database$c_2=database$c_2/100
database$c_3=database$c_3/100

#only use one of the datasets
database= subset(database, database$same_answersG==0&same_answersQ==0&T3_dummy==0)

# ################################################################# #
#### DEFINE MODEL PARAMETERS ####
# ################################################################# #

### Vector of parameters, including any that are kept fixed in estimation
apollo_beta = c(

asc_III = -0.270,

hinc = 20.96,

hdec = -18.1,

v = -3.444,

c = 0.01 # just put a non-zero number here to avoid c approaches to a local optima around zero, is this correct?

)

### Vector with names (in quotes) of parameters to be kept fixed at their starting value in apollo_beta, use apollo_beta_fixed = c() if none
apollo_fixed = c( )

# ################################################################# #
#### GROUP AND VALIDATE INPUTS ####
# ################################################################# #

apollo_inputs = apollo_validateInputs()

# ################################################################# #
#### DEFINE MODEL AND LIKELIHOOD FUNCTION ####
# ################################################################# #

apollo_probabilities=function(apollo_beta, apollo_inputs, functionality="estimate"){

### Attach inputs and detach after function exit
apollo_attach(apollo_beta, apollo_inputs)
on.exit(apollo_detach(apollo_beta, apollo_inputs))

### Create list of probabilities P
P = list()


### Compute utilities
V=list()
V[['alt1']] = c*(hinc*h_1_inc + hdec*h_1_dec + v*v_1 + c_1)


V[['alt2']] = c*(hinc*h_2_inc + hdec*h_2_dec + v*v_2 + c_2)

V[['alt3']] = asc_III + c*(v*v_3 + c_3)

### Define settings for MNL model component that are generic across classes
mnl_settings = list(
alternatives = c(alt1=1, alt2=2,alt3=3),
avail = list(alt1=1, alt2=1,alt3=1),
choiceVar = choice,
V = V

)

### Compute probabilities using MNL model
P[['model']] = apollo_mnl(mnl_settings, functionality)

### Take product across observation for same individual
P = apollo_panelProd(P, apollo_inputs, functionality)

### Average across inter-individual draws
# P = apollo_avgInterDraws(P, apollo_inputs, functionality)

### Prepare and return outputs of function
P = apollo_prepareProb(P, apollo_inputs, functionality)
return(P)
}

# ################################################################# #
#### MODEL ESTIMATION ####
# ################################################################# #

model = apollo_estimate(apollo_beta, apollo_fixed,
apollo_probabilities, apollo_inputs)

# ################################################################# #
#### MODEL OUTPUTS ####
# ################################################################# #

# ----------------------------------------------------------------- #
#---- FORMATTED OUTPUT (TO SCREEN) ----
# ----------------------------------------------------------------- #

apollo_modelOutput(model)


# ################################################################# #
# ################################################################# #
# ################################################################# #
##################MODEL 3: MMNL_WTP SPACE############################
# ################################################################# #
# ################################################################# #
# ################################################################# #


# ################################################################# #
#### LOAD LIBRARY AND DEFINE CORE SETTINGS ####
# ################################################################# #

### Clear memory
rm(list = ls())

### Load Apollo library
library(apollo)

### Initialise code
apollo_initialise()

### Set core controls
apollo_control = list(
modelName ="MMNL_WTP-SPACE",
modelDescr ="MMNL_WTP-SPACE",
indivID ="id1",
nCores = 4,
mixing = TRUE,
noDiagnostics = TRUE
)

# ################################################################# #
#### LOAD DATA AND APPLY ANY TRANSFORMATIONS ####
# ################################################################# #
#load the data
database = read.csv("R_T1T3.csv",header=TRUE)

#rename
names(database)[names(database)=="id"] <- "ID"

names(database)[names(database)=="E_h_1_inc"]<-"h_1_inc"
names(database)[names(database)=="E_h_2_inc"]<-"h_2_inc"
names(database)[names(database)=="E_h_1_dec"]<-"h_1_dec"
names(database)[names(database)=="E_h_2_dec"]<-"h_2_dec"

#rescale data
database$c_1=database$c_1/100
database$c_2=database$c_2/100
database$c_3=database$c_3/100

#only use one of the datasets
database= subset(database, database$same_answersG==0&same_answersQ==0&T3_dummy==0)

# ################################################################# #
#### DEFINE MODEL PARAMETERS ####
# ################################################################# #

### Vector of parameters. Starting values are from the MNL WTP SPACE results in Model 2
apollo_beta = c(


asc_III = -0.281,

hinc_mu = 41.991,
hinc_sig = 0.1,

hdec_mu = -36.799,
hdec_sig = 0.1,


v_mu = -6.034,
v_sig = 0.1,


log_c_mu = -3,
log_c_sig = 0.1


)

### Vector with names (in quotes) of parameters to be kept fixed at their starting value in apollo_beta, use apollo_beta_fixed = c() if none
apollo_fixed = c( )


# ################################################################# #
#### DEFINE RANDOM COMPONENTS ####
# ################################################################# #

### Set parameters for generating draws
apollo_draws = list(
interDrawsType = "mlhs",
interNDraws = 500,
interUnifDraws = c(),
interNormDraws = c("draws_v",
"draws_hinc",
"draws_hdec",
"draws_c"

),
intraDrawsType = "halton",
intraNDraws = 0,
intraUnifDraws = c(),
intraNormDraws = c()
)

## Create random parameters
apollo_randCoeff = function(apollo_beta, apollo_inputs){
randcoeff = list()

randcoeff[["hinc"]] = hinc_mu + hinc_sig*draws_hinc

randcoeff[["hdec"]] = hdec_mu + hdec_sig*draws_hdec

randcoeff[["v"]] = v_mu + v_sig*draws_v

randcoeff[["c"]]= -exp(log_c_mu+log_c_sig*draws_c)

return(randcoeff)
}


# ################################################################# #
#### GROUP AND VALIDATE INPUTS ####
# ################################################################# #

apollo_inputs = apollo_validateInputs()

# ################################################################# #
#### DEFINE MODEL AND LIKELIHOOD FUNCTION ####
# ################################################################# #

apollo_probabilities=function(apollo_beta, apollo_inputs, functionality="estimate"){

### Attach inputs and detach after function exit
apollo_attach(apollo_beta, apollo_inputs)
on.exit(apollo_detach(apollo_beta, apollo_inputs))

### Create list of probabilities P
P = list()

### Compute utilities
V=list()
V[['alt1']] = c*(hinc*h_1_inc + hdec*h_1_dec + v*v_1 + c_1)


V[['alt2']] = c*(hinc*h_2_inc + hdec*h_2_dec + v*v_2 + c_2)

V[['alt3']] = asc_III + c*(v*v_3 + c_3)

### Define settings for MNL model component that are generic across classes
mnl_settings = list(
alternatives = c(alt1=1, alt2=2,alt3=3),
avail = list(alt1=1, alt2=1,alt3=1),
choiceVar = choice,
V = V

)

### Compute probabilities using MNL model
P[['model']] = apollo_mnl(mnl_settings, functionality)

### Take product across observation for same individual
P = apollo_panelProd(P, apollo_inputs, functionality)

### Average across inter-individual draws
P = apollo_avgInterDraws(P, apollo_inputs, functionality)

### Prepare and return outputs of function
P = apollo_prepareProb(P, apollo_inputs, functionality)
return(P)
}

# ################################################################# #
#### MODEL ESTIMATION ####
# ################################################################# #

model = apollo_estimate(apollo_beta, apollo_fixed,
apollo_probabilities, apollo_inputs)

# ################################################################# #
#### MODEL OUTPUTS ####
# ################################################################# #

# ----------------------------------------------------------------- #
#---- FORMATTED OUTPUT (TO SCREEN) ----
# ----------------------------------------------------------------- #

apollo_modelOutput(model)


# ################################################################# #
# ################################################################# #
# ################################################################# #
##################MODEL 4: MMNL_PREFERENCE SPACE#####################
# ################################################################# #
# ################################################################# #
# ################################################################# #


# ################################################################# #
#### LOAD LIBRARY AND DEFINE CORE SETTINGS ####
# ################################################################# #

### Clear memory
rm(list = ls())

### Load Apollo library
library(apollo)

### Initialise code
apollo_initialise()

### Set core controls
apollo_control = list(
modelName ="MMNL",
modelDescr ="MMNL",
indivID ="id1",
nCores = 4,
mixing = TRUE,
noDiagnostics = TRUE
)

# ################################################################# #
#### LOAD DATA AND APPLY ANY TRANSFORMATIONS ####
# ################################################################# #
#load the data
database = read.csv("R_T1T3.csv",header=TRUE)

#rename
names(database)[names(database)=="id"] <- "ID"

names(database)[names(database)=="E_h_1_inc"]<-"h_1_inc"
names(database)[names(database)=="E_h_2_inc"]<-"h_2_inc"
names(database)[names(database)=="E_h_1_dec"]<-"h_1_dec"
names(database)[names(database)=="E_h_2_dec"]<-"h_2_dec"

#rescale data
database$c_1=database$c_1/100
database$c_2=database$c_2/100
database$c_3=database$c_3/100

#only use one of the datasets
database= subset(database, database$same_answersG==0&same_answersQ==0&T3_dummy==0)

# ################################################################# #
#### DEFINE MODEL PARAMETERS ####
# ################################################################# #

### Vector of parameters. Starting values are from the MNL WTP SPACE results in Model 2
apollo_beta = c(


asc_III = -0.270,

hinc_mu = 0.408,
hinc_sig = 0.1,

hdec_mu = -0.352,
hdec_sig = 0.1,


v_mu = -0.067,
v_sig = 0.1,


log_c_mu = -3,
log_c_sig = 1


)

### Vector with names (in quotes) of parameters to be kept fixed at their starting value in apollo_beta, use apollo_beta_fixed = c() if none
apollo_fixed = c( )


# ################################################################# #
#### DEFINE RANDOM COMPONENTS ####
# ################################################################# #

### Set parameters for generating draws
apollo_draws = list(
interDrawsType = "mlhs",
interNDraws = 500,
interUnifDraws = c(),
interNormDraws = c("draws_v",
"draws_hinc",
"draws_hdec",
"draws_c"


),
intraDrawsType = "halton",
intraNDraws = 0,
intraUnifDraws = c(),
intraNormDraws = c()
)

## Create random parameters
apollo_randCoeff = function(apollo_beta, apollo_inputs){
randcoeff = list()

randcoeff[["hinc"]] = hinc_mu + hinc_sig*draws_hinc

randcoeff[["hdec"]] = hdec_mu + hdec_sig*draws_hdec

randcoeff[["v"]] = v_mu + v_sig*draws_v

randcoeff[["c"]]= -exp(log_c_mu+log_c_sig*draws_c)

return(randcoeff)
}


# ################################################################# #
#### GROUP AND VALIDATE INPUTS ####
# ################################################################# #

apollo_inputs = apollo_validateInputs()

# ################################################################# #
#### DEFINE MODEL AND LIKELIHOOD FUNCTION ####
# ################################################################# #

apollo_probabilities=function(apollo_beta, apollo_inputs, functionality="estimate"){

### Attach inputs and detach after function exit
apollo_attach(apollo_beta, apollo_inputs)
on.exit(apollo_detach(apollo_beta, apollo_inputs))

### Create list of probabilities P
P = list()

### Compute utilities
V=list()
V[['alt1']] = hinc*h_1_inc + hdec*h_1_dec + v*v_1 + c*c_1


V[['alt2']] = hinc*h_2_inc + hdec*h_2_dec + v*v_2 + c*c_2

V[['alt3']] = asc_III + v*v_3 + c*c_3

### Define settings for MNL model component that are generic across classes
mnl_settings = list(
alternatives = c(alt1=1, alt2=2,alt3=3),
avail = list(alt1=1, alt2=1,alt3=1),
choiceVar = choice,
V = V

)

### Compute probabilities using MNL model
P[['model']] = apollo_mnl(mnl_settings, functionality)

### Take product across observation for same individual
P = apollo_panelProd(P, apollo_inputs, functionality)

### Average across inter-individual draws
P = apollo_avgInterDraws(P, apollo_inputs, functionality)

### Prepare and return outputs of function
P = apollo_prepareProb(P, apollo_inputs, functionality)
return(P)
}

# ################################################################# #
#### MODEL ESTIMATION ####
# ################################################################# #

model = apollo_estimate(apollo_beta, apollo_fixed,
apollo_probabilities, apollo_inputs)

# ################################################################# #
#### MODEL OUTPUTS ####
# ################################################################# #

# ----------------------------------------------------------------- #
#---- FORMATTED OUTPUT (TO SCREEN) ----
# ----------------------------------------------------------------- #

apollo_modelOutput(model)

######################################################
#############load unconditional parameters############
######################################################
unconditionals=apollo_unconditionals(model,apollo_probabilities,apollo_inputs)
wtp_hinc_Pspace=mean(unconditionals[["hinc"]])/-mean(unconditionals[["c"]])
wtp_hdec_Pspace=mean(unconditionals[["hdec"]])/-mean(unconditionals[["c"]])
wtp_v_Pspace=mean(unconditionals[["v"]])/-mean(unconditionals[["c"]])
wtp_PreferenceSpace=c(wtp_hinc_Pspace,wtp_hdec_Pspace,wtp_v_Pspace)
wtp_PreferenceSpace





RESULTS
Model 1:
Model name : MNL
Model description : MNL
Model run at : 2022-12-31 14:49:27
Estimation method : bfgs
Model diagnosis : successful convergence
Number of individuals : 228
Number of rows in database : 2280
Number of modelled outcomes : 2280

Number of cores used : 4
Model without mixing

LL(start) : -2504.84
LL at equal shares, LL(0) : -2504.84
LL at observed shares, LL(C) : -2482.26
LL(final) : -2372.59
Rho-squared vs equal shares : 0.0528
Adj.Rho-squared vs equal shares : 0.0508
Rho-squared vs observed shares : 0.0442
Adj.Rho-squared vs observed shares : 0.0422
AIC : 4755.19
BIC : 4783.85

Estimated parameters : 5
Time taken (hh:mm:ss) : 00:00:15.02
pre-estimation : 00:00:13.88
estimation : 00:00:0.54
post-estimation : 00:00:0.6
Iterations : 12
Min abs eigenvalue of Hessian : 82.58577

Unconstrained optimisation.

These outputs have had the scaling used in estimation applied to them.
Estimates:
Estimate s.e. t.rat.(0) Rob.s.e. Rob.t.rat.(0)
asc_III -0.27052 0.08686 -3.114 0.10822 -2.500
hinc 0.40828 0.05885 6.937 0.06763 6.037
hdec -0.35248 0.06112 -5.767 0.07528 -4.682
v -0.06707 0.01221 -5.491 0.01320 -5.079
c -0.01948 0.01320 -1.475 0.01592 -1.223

Model 2:
Model name : MNL_WTP-SPACE
Model description : MNL_WTP-SPACE
Model run at : 2022-12-31 14:32:32
Estimation method : bfgs
Model diagnosis : successful convergence
Number of individuals : 228
Number of rows in database : 2280
Number of modelled outcomes : 2280

Number of cores used : 4
Model without mixing

LL(start) : -2395.81
LL at equal shares, LL(0) : -2504.84
LL at observed shares, LL(C) : -2482.26
LL(final) : -2374.95
Rho-squared vs equal shares : 0.0519
Adj.Rho-squared vs equal shares : 0.0499
Rho-squared vs observed shares : 0.0432
Adj.Rho-squared vs observed shares : 0.0412
AIC : 4759.91
BIC : 4788.57

Estimated parameters : 5
Time taken (hh:mm:ss) : 00:00:16.27
pre-estimation : 00:00:13.59
estimation : 00:00:2.1
post-estimation : 00:00:0.58
Iterations : 48
Min abs eigenvalue of Hessian : 0.001004

Unconstrained optimisation.

These outputs have had the scaling used in estimation applied to them.
Estimates:
Estimate s.e. t.rat.(0) Rob.s.e. Rob.t.rat.(0)
asc_III -0.280994 0.087859 -3.198 0.109471 -2.567
hinc 41.991383 23.170909 1.812 9.959715 4.216
hdec -36.799333 22.739927 -1.618 14.553082 -2.529
v -6.034356 3.615529 -1.669 1.851588 -3.259
c 0.008961 0.004929 1.818 0.002568 3.489

Model 3:
Model name : MMNL_WTP-SPACE
Model description : MMNL_WTP-SPACE
Model run at : 2022-12-31 14:35:07
Estimation method : bfgs
Model diagnosis : successful convergence
Number of individuals : 228
Number of rows in database : 2280
Number of modelled outcomes : 2280

Number of cores used : 4
Number of inter-individual draws : 500 (mlhs)

LL(start) : -5556.05
LL at equal shares, LL(0) : -2504.84
LL at observed shares, LL(C) : -2482.26
LL(final) : -2482.41
Rho-squared vs equal shares : 0.009
Adj.Rho-squared vs equal shares : 0.0054
Rho-squared vs observed shares : -1e-04
Adj.Rho-squared vs observed shares : -0.0037
AIC : 4982.82
BIC : 5034.4

Estimated parameters : 9
Time taken (hh:mm:ss) : 00:03:32.58
pre-estimation : 00:00:26.24
estimation : 00:01:36.66
post-estimation : 00:01:29.68
Iterations : 24
Min abs eigenvalue of Hessian : 7e-05
Some eigenvalues of Hessian are positive, indicating potential problems!

Unconstrained optimisation.

These outputs have had the scaling used in estimation applied to them.
Estimates:
Estimate s.e. t.rat.(0) Rob.s.e. Rob.t.rat.(0)
asc_III -0.3095 0.04727 -6.547625 0.07856 -3.93971
hinc_mu 41.8863 151.44462 0.276578 20.91372 2.00281
hinc_sig 0.1915 NaN NaN 18.30019 0.01047
hdec_mu -34.8849 35.57165 -0.980693 35.69305 -0.97736
hdec_sig -0.2863 NaN NaN 26.19673 -0.01093
v_mu -4.9735 58.15338 -0.085524 3.77369 -1.31795
v_sig 0.2767 166.51250 0.001662 7.54384 0.03668
log_c_mu -82.1537 NaN NaN 0.13723 -598.65428
log_c_sig 22.1010 NaN NaN 0.03026 730.41261

Model 4:

Model name : MMNL
Model description : MMNL
Model run at : 2022-12-31 14:50:45
Estimation method : bfgs
Model diagnosis : successful convergence
Number of individuals : 228
Number of rows in database : 2280
Number of modelled outcomes : 2280

Number of cores used : 4
Number of inter-individual draws : 500 (mlhs)

LL(start) : -2331.43
LL at equal shares, LL(0) : -2504.84
LL at observed shares, LL(C) : -2482.26
LL(final) : -2038.47
Rho-squared vs equal shares : 0.1862
Adj.Rho-squared vs equal shares : 0.1826
Rho-squared vs observed shares : 0.1788
Adj.Rho-squared vs observed shares : 0.1752
AIC : 4094.94
BIC : 4146.53

Estimated parameters : 9
Time taken (hh:mm:ss) : 00:05:15.85
pre-estimation : 00:00:27.22
estimation : 00:03:14.65
post-estimation : 00:01:33.98
Iterations : 46
Min abs eigenvalue of Hessian : 3.916915

Unconstrained optimisation.

These outputs have had the scaling used in estimation applied to them.
Estimates:
Estimate s.e. t.rat.(0) Rob.s.e. Rob.t.rat.(0)
asc_III -0.7103 0.11055 -6.425 0.14174 -5.011
hinc_mu 0.5225 0.11268 4.637 0.11022 4.741
hinc_sig 1.2204 0.10955 11.140 0.12114 10.074
hdec_mu -0.9954 0.15516 -6.415 0.16618 -5.990
hdec_sig 1.7294 0.14657 11.799 0.14540 11.894
v_mu -0.1100 0.01935 -5.683 0.02128 -5.168
v_sig 0.1687 0.02332 7.233 0.02774 6.080
log_c_mu -3.7603 0.45998 -8.175 0.46398 -8.104
log_c_sig 1.8217 0.24386 7.471 0.17112 10.646

WTPs from Model 4:
wtp_PreferenceSpace = c(4.2567504 -8.1148622 -0.8970758)
stephanehess
Site Admin
Posts: 974
Joined: 24 Apr 2020, 16:29

Re: huge difference between results from a WTP-space and Preference-space model

Post by stephanehess »

Hi

the MNL results are already a bit worrying. The cost coefficient becomes positive in WTP space, so that's already an issue. Why did you decide on a positive starting value for that?

BTW, even though this is unlabelled, you should really also include an ASC to test for ordering effects. Unfortunately, people keep thinking that you don't need a full set of ASCs in unlabelled data

Stephane
--------------------------------
Stephane Hess
www.stephanehess.me.uk
arohamirai
Posts: 18
Joined: 28 Jun 2020, 16:26

Re: huge difference between results from a WTP-space and Preference-space model

Post by arohamirai »

Hi Stephane,

Thank you for your reply. I agree that the positive cost coefficient is a bit worrying. Indeed, the cost is negative but not significant in the simplest MNL preference-space model (model 1). For the MNL WTP-space model, I did try a negative starting value for the cost parameter, but the estimated coefficient is still positive.

Do you think the weird WTP-space results could be caused by the insignificant cost parameter (i.e., likelihood function is difficult to estimate due to the denominator approximating to zero)? In this case, would you suggest to estimate a preference-space model?

Thank you very much.

Best wishes,
Tim
stephanehess
Site Admin
Posts: 974
Joined: 24 Apr 2020, 16:29

Re: huge difference between results from a WTP-space and Preference-space model

Post by stephanehess »

Tim

I think you need to think about why the cost coefficient is not impacting utility. That is worrying in your first model. Of course, working in WTP space then doesn't make much sense anyway

Is there something wrong with the data, maybe?

Stephane
--------------------------------
Stephane Hess
www.stephanehess.me.uk
Post Reply