Unreasonable estimate for price and other attributes

qcng · Post by **qcng** » 27 Dec 2023, 11:40

Dear Professors,

In my study, a choice experiment includes 5 attribtues: 4 binary attributes flour, fibre, upcycled, texture and price (4 levels: 25, 40, 55, 70).
I have applied MNL as follows

Code: Select all

# clean
rm(list = ls()); graphics.off()

# .libPaths("C:/Rpack")

library(openxlsx)
library(apollo)

# anal. Apollo ------------------------------------------------------------

### Initialise code
apollo_initialise()

### Set core controls
apollo_control = list(
  modelName       = "MNL_preference",
  modelDescr      = "Simple MNL model on mode choice SP data, in preference space",
  indivID         = "ID",
  outputDirectory = "output"
)

# ################################################################# #
#### LOAD DATA AND APPLY ANY TRANSFORMATIONS                     ####
# ################################################################# #

dataApollo <- read.xlsx("dataApollo (biscuits).xlsx", sheet = "data", colNames = T)
database <- dataApollo

# ################################################################# #
#### DEFINE MODEL PARAMETERS                                     ####
# ################################################################# #

### Vector of parameters, including any that are kept fixed in estimation
apollo_beta=c(asc_alt3      = 0,
              
              b_flour       = 0,
              b_fibre       = 0,
              b_upc         = 0,
              b_tex         = 0,
              
              b_price       = 0)

### Vector with names (in quotes) of parameters to be kept fixed at their starting value in apollo_beta, use apollo_beta_fixed = c() if none
apollo_fixed = c()

# ################################################################# #
#### GROUP AND VALIDATE INPUTS                                   ####
# ################################################################# #

apollo_inputs = apollo_validateInputs()

# ################################################################# #
#### DEFINE MODEL AND LIKELIHOOD FUNCTION                        ####
# ################################################################# #

apollo_probabilities=function(apollo_beta, apollo_inputs, functionality="estimate"){
  
  ### Attach inputs and detach after function exit
  apollo_attach(apollo_beta, apollo_inputs)
  on.exit(apollo_detach(apollo_beta, apollo_inputs))
  
  ### Create list of probabilities P
  P = list()
  
  ### List of utilities: these must use the same names as in mnl_settings, order is irrelevant
  V = list()
  V[["alt1"]]  = (           b_flour * flour_alt1 + b_fibre * fibre_alt1 + b_upc * upcycled_alt1 + b_tex * texture_alt1 + b_price * price_alt1)
  V[["alt2"]]  = (           b_flour * flour_alt2 + b_fibre * fibre_alt2 + b_upc * upcycled_alt2 + b_tex * texture_alt2 + b_price * price_alt2)
  V[["alt3"]]  = (asc_alt3 + b_flour * flour_alt3 + b_fibre * fibre_alt3 + b_upc * upcycled_alt3 + b_tex * texture_alt3 + b_price * price_alt3)
  
  ### Define settings for MNL model component
  mnl_settings = list(
    alternatives  = c(alt1=1, alt2=2, alt3=3), 
    avail         = list(alt1=av_alt1, alt2=av_alt2, alt3=av_alt3), 
    choiceVar     = choice,
    utilities     = V
  )
  
  ### Compute probabilities using MNL model
  P[["model"]] = apollo_mnl(mnl_settings, functionality)
  
  ### Take product across observation for same individual
  P = apollo_panelProd(P, apollo_inputs, functionality)
  
  ### Prepare and return outputs of function
  P = apollo_prepareProb(P, apollo_inputs, functionality)
  return(P)
}

# ################################################################# #
#### MODEL ESTIMATION                                            ####
# ################################################################# #

model = apollo_estimate(apollo_beta, apollo_fixed, apollo_probabilities, apollo_inputs)

# ################################################################# #
#### MODEL OUTPUTS                                               ####
# ################################################################# #

# ----------------------------------------------------------------- #
#---- FORMATTED OUTPUT (TO SCREEN)                               ----
# ----------------------------------------------------------------- #

# apollo_modelOutput(model)

apollo_modelOutput(model, list(printPVal = TRUE))  #with p-values

# ----------------------------------------------------------------- #
#---- FORMATTED OUTPUT (TO FILE, using model name)               ----
# ----------------------------------------------------------------- #

# apollo_saveOutput(model)

apollo_saveOutput(model, list(printPVal = TRUE))  #with p-values

The model estimation is as follows

> model = apollo_estimate(apollo_beta, apollo_fixed, apollo_probabilities, apollo_inputs)
Preparing user-defined functions.

Testing likelihood function...

Overview of choices for MNL model component :
alt1 alt2 alt3
Times available 828.00 828.00 828.00
Times chosen 395.00 405.00 28.00
Percentage chosen overall 47.71 48.91 3.38
Percentage chosen when available 47.71 48.91 3.38

Pre-processing likelihood function...

Testing influence of parameters
Starting main estimation
Initial function value: -909.651
Initial gradient value:
asc_alt3 b_flour b_fibre b_upc b_tex b_price
-248 190 180 296 167 11090
initial value 909.650975
iter 2 value 783.621739
iter 3 value 624.381642
iter 4 value 619.698426
iter 5 value 616.699689
iter 6 value 611.004587
iter 7 value 600.080297
iter 8 value 583.911269
iter 9 value 570.421119
iter 10 value 569.048008
iter 11 value 568.798479
iter 12 value 568.661859
iter 13 value 568.625278
iter 14 value 568.624893
iter 14 value 568.624890
final value 568.624890
converged
Additional convergence test using scaled estimation. Parameters will be scaled by their
current estimates and additional iterations will be performed.
initial value 568.624890
iter 2 value 568.624878
iter 3 value 568.624867
iter 3 value 568.624866
iter 3 value 568.624860
final value 568.624860
converged

Estimated parameters with approximate standard errors from BHHH matrix:
Estimate BHHH se BHH t-ratio
asc_alt3 -1.751482 0.216175 -8.102
b_flour 0.381428 0.095018 4.014
b_fibre 0.414557 0.083741 4.950
b_upc 1.008169 0.046842 21.523
b_tex 0.246690 0.106062 2.326
b_price -0.005679 0.002298 -2.471

Final LL: -568.6249

Calculating log-likelihood at equal shares (LL(0)) for applicable models...
Calculating log-likelihood at observed shares from estimation data (LL(c)) for applicable
models...
Calculating LL of each model component...
Calculating other model fit measures
Computing covariance matrix using analytical gradient.
0%....25%....50%....75%.100%
Negative definite Hessian with maximum eigenvalue: -14.111301

As can be seen, the effect of upcycled (b_upc) is extremely high as compared to other attribtues. More importantly, price is not significant.

> apollo_modelOutput(model, list(printPVal = TRUE)) #with p-values
Model run by THINKPAD using Apollo 0.2.9 on R 4.3.2 for Windows.
www.ApolloChoiceModelling.com

Model name : MNL_preference
Model description : Simple MNL model on mode choice SP data, in preference space
Model run at : 2023-12-27 18:26:42.99382
Estimation method : bfgs
Model diagnosis : successful convergence
Optimisation diagnosis : Maximum found
hessian properties : Negative definitive
maximum eigenvalue : -14.111301
Number of individuals : 69
Number of rows in database : 828
Number of modelled outcomes : 828

Number of cores used : 1
Model without mixing

LL(start) : -909.65
LL at equal shares, LL(0) : -909.65
LL at observed shares, LL(C) : -676.81
LL(final) : -568.62
Rho-squared vs equal shares : 0.3749
Adj.Rho-squared vs equal shares : 0.3683
Rho-squared vs observed shares : 0.1598
Adj.Rho-squared vs observed shares : 0.1539
AIC : 1149.25
BIC : 1177.56

Estimated parameters : 6
Time taken (hh:mm:ss) : 00:00:1.42
pre-estimation : 00:00:0.6
estimation : 00:00:0.48
initial estimation : 00:00:0.38
estimation after rescaling : 00:00:0.1
post-estimation : 00:00:0.34
Iterations : 20
initial estimation : 16
estimation after rescaling : 4

Unconstrained optimisation.

Estimates:
Estimate s.e. t.rat.(0) p(1-sided) Rob.s.e. Rob.t.rat.(0) p(1-sided)
asc_alt3 -1.751482 0.263087 -6.657 1.393e-11 0.366168 -4.783 8.623e-07
b_flour 0.381428 0.084193 4.530 2.944e-06 0.081110 4.703 1.284e-06
b_fibre 0.414557 0.085102 4.871 5.543e-07 0.094357 4.394 5.577e-06
b_upc 1.008169 0.084121 11.985 0.000000 0.156663 6.435 6.162e-11
b_tex 0.246690 0.081655 3.021 0.001259 0.069599 3.544 1.9673e-04
b_price -0.005679 0.003125 -1.817 0.034588 0.005353 -1.061 0.1444

It is a strange result from my side.

For WTP space, here is the codes

Code: Select all

# clean
rm(list = ls()); graphics.off()

# .libPaths("C:/Rpack")

library(openxlsx)
library(apollo)

# anal. Apollo ------------------------------------------------------------

### Initialise code
apollo_initialise()

### Set core controls
apollo_control = list(
  modelName       = "MNL_WTP",
  modelDescr      = "MNL model on mode choice SP data, in WTP space",
  indivID         = "ID", 
  outputDirectory = "output"
)

# ################################################################# #
#### LOAD DATA AND APPLY ANY TRANSFORMATIONS                     ####
# ################################################################# #

dataApollo <- read.xlsx("dataApollo (biscuits).xlsx", sheet = "data", colNames = T)

database <- dataApollo

# ################################################################# #
#### DEFINE MODEL PARAMETERS                                     ####
# ################################################################# #

### Vector of parameters, including any that are kept fixed in estimation
apollo_beta=c(asc_alt3      = 0,
              
              w_flour       = 0,
              w_fibre       = 0,
              w_upc         = 0,
              w_tex         = 0,
              
              b_price       = 0)


### Vector with names (in quotes) of parameters to be kept fixed at their starting value in apollo_beta, use apollo_beta_fixed = c() if none
apollo_fixed = c()

# ################################################################# #
#### GROUP AND VALIDATE INPUTS                                   ####
# ################################################################# #

apollo_inputs = apollo_validateInputs()

# ################################################################# #
#### DEFINE MODEL AND LIKELIHOOD FUNCTION                        ####
# ################################################################# #

apollo_probabilities=function(apollo_beta, apollo_inputs, functionality="estimate"){
  
  ### Attach inputs and detach after function exit
  apollo_attach(apollo_beta, apollo_inputs)
  on.exit(apollo_detach(apollo_beta, apollo_inputs))
  
  ### Create list of probabilities P
  P = list()
  
  ### List of utilities: these must use the same names as in mnl_settings, order is irrelevant
  V = list()
  V[["alt1"]]  = (           b_price * (w_flour * flour_alt1 + w_fibre * fibre_alt1 + w_upc * upcycled_alt1 + w_tex * texture_alt1 + price_alt1))
  V[["alt2"]]  = (           b_price * (w_flour * flour_alt2 + w_fibre * fibre_alt2 + w_upc * upcycled_alt2 + w_tex * texture_alt2 + price_alt2))
  V[["alt3"]]  = (asc_alt3 + b_price * (w_flour * flour_alt3 + w_fibre * fibre_alt3 + w_upc * upcycled_alt3 + w_tex * texture_alt3 + price_alt3))
  
  ### Define settings for MNL model component
  mnl_settings = list(
    alternatives  = c(alt1=1, alt2=2, alt3=3), 
    avail         = list(alt1=av_alt1, alt2=av_alt2, alt3=av_alt3), 
    choiceVar     = choice,
    utilities     = V
  )
  
  ### Compute probabilities using MNL model
  P[["model"]] = apollo_mnl(mnl_settings, functionality)
  
  ### Take product across observation for same individual
  P = apollo_panelProd(P, apollo_inputs, functionality)
  
  ### Prepare and return outputs of function
  P = apollo_prepareProb(P, apollo_inputs, functionality)
  return(P)
}

# ################################################################# #
#### MODEL ESTIMATION                                            ####
# ################################################################# #

model = apollo_estimate(apollo_beta, apollo_fixed, apollo_probabilities, apollo_inputs)

# ################################################################# #
#### MODEL OUTPUTS                                               ####
# ################################################################# #

# ----------------------------------------------------------------- #
#---- FORMATTED OUTPUT (TO SCREEN)                               ----
# ----------------------------------------------------------------- #

# apollo_modelOutput(model)

apollo_modelOutput(model, list(printPVal = TRUE))  #with p-values

# ----------------------------------------------------------------- #
#---- FORMATTED OUTPUT (TO FILE, using model name)               ----
# ----------------------------------------------------------------- #

# apollo_saveOutput(model)

apollo_saveOutput(model, list(printPVal = TRUE))  #with p-values

and results

> model = apollo_estimate(apollo_beta, apollo_fixed, apollo_probabilities, apollo_inputs)
Preparing user-defined functions.

Testing likelihood function...

Overview of choices for MNL model component :
alt1 alt2 alt3
Times available 828.00 828.00 828.00
Times chosen 395.00 405.00 28.00
Percentage chosen overall 47.71 48.91 3.38
Percentage chosen when available 47.71 48.91 3.38

Pre-processing likelihood function...

Testing influence of parameters
Starting main estimation
Initial function value: -909.651
Initial gradient value:
asc_alt3 w_flour w_fibre w_upc w_tex b_price
-248 0 0 0 0 11090
initial value 909.650975
iter 2 value 783.898748
iter 3 value 694.075750
iter 4 value 677.364216
iter 5 value 674.574903
iter 6 value 650.466102
iter 7 value 645.366875
iter 8 value 637.398770
iter 9 value 629.478738
iter 10 value 612.279506
iter 11 value 585.479804
iter 12 value 574.342963
iter 13 value 573.642400
iter 14 value 573.201709
iter 15 value 573.167088
iter 16 value 573.104417
iter 17 value 572.995556
iter 18 value 572.333537
iter 19 value 571.654557
iter 20 value 571.135962
iter 21 value 571.076801
iter 22 value 571.058097
iter 23 value 571.019968
iter 24 value 570.955301
iter 25 value 570.785926
iter 26 value 570.403851
iter 27 value 569.636533
iter 28 value 569.626783
iter 29 value 569.619936
iter 30 value 569.610734
iter 31 value 569.361856
iter 32 value 569.306282
iter 33 value 569.297379
iter 34 value 569.296031
iter 35 value 569.293176
iter 36 value 569.287235
iter 37 value 569.271545
iter 38 value 569.236359
iter 39 value 569.161451
iter 40 value 569.050594
iter 41 value 569.044897
iter 42 value 569.036172
iter 43 value 569.032146
iter 44 value 569.027560
iter 45 value 569.011240
iter 46 value 568.984396
iter 47 value 568.946792
iter 48 value 568.920947
iter 49 value 568.914083
iter 50 value 568.913417
iter 51 value 568.913052
iter 52 value 568.911944
iter 53 value 568.909410
iter 54 value 568.908556
iter 55 value 568.908507
iter 56 value 568.908137
iter 57 value 568.907771
iter 58 value 568.907262
iter 59 value 568.905496
iter 60 value 568.901505
iter 61 value 568.891534
iter 62 value 568.869662
iter 63 value 568.828098
iter 64 value 568.800995
iter 65 value 568.798748
iter 66 value 568.793984
iter 66 value 568.793984
final value 568.793984
converged
Additional convergence test using scaled estimation. Parameters will be scaled by their
current estimates and additional iterations will be performed.
initial value 568.793984
iter 2 value 568.777055
iter 3 value 568.768532
iter 4 value 568.753520
iter 5 value 568.732190
iter 6 value 568.714581
iter 7 value 568.689912
iter 8 value 568.658344
iter 9 value 568.642751
iter 10 value 568.634034
iter 11 value 568.628840
iter 12 value 568.625442
iter 13 value 568.624959
iter 14 value 568.624865
iter 14 value 568.624861
final value 568.624861
converged

Estimated parameters with approximate standard errors from BHHH matrix:
Estimate BHHH se BHH t-ratio
asc_alt3 -1.751482 0.216175 -8.102
w_flour -67.085507 38.333175 -1.750
w_fibre -72.936437 27.307235 -2.671
w_upc -177.316117 73.811873 -2.402
w_tex -43.427451 27.435850 -1.583
b_price -0.005686 0.002298 -2.474

Final LL: -568.6249

Calculating log-likelihood at equal shares (LL(0)) for applicable models...
Calculating log-likelihood at observed shares from estimation data (LL(c)) for applicable
models...
Calculating LL of each model component...
Calculating other model fit measures
Computing covariance matrix using analytical gradient.
0%....25%....50%....75%.100%
Negative definite Hessian with maximum eigenvalue: -7.5e-05

with p-values

> apollo_modelOutput(model, list(printPVal = TRUE)) #with p-values
Model run by THINKPAD using Apollo 0.2.9 on R 4.3.2 for Windows.
www.ApolloChoiceModelling.com

Model name : MNL_WTP
Model description : MNL model on mode choice SP data, in WTP space
Model run at : 2023-12-27 18:36:47.142215
Estimation method : bfgs
Model diagnosis : successful convergence
Optimisation diagnosis : Maximum found
hessian properties : Negative definitive
maximum eigenvalue : -7.5e-05
Number of individuals : 69
Number of rows in database : 828
Number of modelled outcomes : 828

Number of cores used : 1
Model without mixing

LL(start) : -909.65
LL at equal shares, LL(0) : -909.65
LL at observed shares, LL(C) : -676.81
LL(final) : -568.62
Rho-squared vs equal shares : 0.3749
Adj.Rho-squared vs equal shares : 0.3683
Rho-squared vs observed shares : 0.1598
Adj.Rho-squared vs observed shares : 0.1539
AIC : 1149.25
BIC : 1177.56

Estimated parameters : 6
Time taken (hh:mm:ss) : 00:00:2.25
pre-estimation : 00:00:0.56
estimation : 00:00:1.36
initial estimation : 00:00:1.12
estimation after rescaling : 00:00:0.23
post-estimation : 00:00:0.33
Iterations : 82
initial estimation : 68
estimation after rescaling : 14

Unconstrained optimisation.

Estimates:
Estimate s.e. t.rat.(0) p(1-sided) Rob.s.e. Rob.t.rat.(0) p(1-sided)
asc_alt3 -1.751482 0.263458 -6.648 1.485e-11 0.366664 -4.777 8.905e-07
w_flour -67.085507 42.722521 -1.570 0.05818 63.486870 -1.057 0.1453
w_fibre -72.936437 39.498740 -1.847 0.03241 68.260375 -1.069 0.1426
w_upc -177.316117 98.509385 -1.800 0.03593 163.200671 -1.086 0.1386
w_tex -43.427451 27.694722 -1.568 0.05843 39.565351 -1.098 0.1362
b_price -0.005686 0.003139 -1.812 0.03502 0.005397 -1.053 0.1461

As can be seen, all WTP are not significant.

These results are strange for me. Maybe I have set someting wrong.

Could please have a look and give me some advices?

Thanks very much.

Best regards,

Cuong

Post by **stephanehess** » 19 Jan 2024, 01:25

Hi

apologies for the slow reply, David and I have been away.

There is nothing wrong with your code at face value. Can you tell us more about the experiment? You might want to test the model with all attributes treated as categorical as opposed to linear.

Stephane

qcng · Post by **qcng** » 03 Feb 2024, 01:58

Hi Stephane,

Thanks for your reply.

In this choice experiment, I would like to test the effects of 5 attributes on consumer WTP of biscuits

type of flour: wheat flour, BSG flour

fibre information: no label, high fibre

upcycled information: no label, upcycled logo

texture description: no label, crispy description

price: 25, 40, 55, 70

Could you please provide me some further advices?

Thanks and best regards,

Cuong

Post by **stephanehess** » 06 Feb 2024, 19:56

Hi

first, you should always use a full set (i.e. J-1) alternative specific constants.

But is your third alternative systematically different from the other two which could lead to the issues you're seeing

Stephane

qcng · Post by **qcng** » 19 Feb 2024, 10:01

Hi Prof. Stephane,

I do not get the idea from your response.

"first, you should always use a full set (i.e. J-1) alternative specific constants." With this comment, does it mean that I need to assign, for example, asc of alternative 3 = 0?

"But is your third alternative systematically different from the other two which could lead to the issues you're seeing" You mean that the significance of asc alt3 will lead issues in the estimation of beta or wtp.

In addition, you mentioned "the model with all attributes treated as categorical as opposed to linear.". Could you please give me further details about it? Could we run this with Apollo?

Thanks and best regards,

Cuong

Post by **stephanehess** » 23 Feb 2024, 11:32

Hi Cuong

You have 3 alternatives, but you are only estimating one constant. You can estimate J-1 constants, i.e. 2

My question was whether your third alternative in the survey is different from the other two, i.e. does it use different levels, etc?

Regarding categorical, what I mean is that you dummy code all your attributes

Stephane

ApolloChoiceModelling forum

Unreasonable estimate for price and other attributes

Unreasonable estimate for price and other attributes

Re: Unreasonable estimate for price and other attributes

Re: Unreasonable estimate for price and other attributes

Re: Unreasonable estimate for price and other attributes

Re: Unreasonable estimate for price and other attributes

Re: Unreasonable estimate for price and other attributes