Important: Read this before posting to this forum

  1. This forum is for questions related to the use of Apollo. We will answer some general choice modelling questions too, where appropriate, and time permitting. We cannot answer questions about how to estimate choice models with other software packages.
  2. There is a very detailed manual for Apollo available at http://www.ApolloChoiceModelling.com/manual.html. This contains detailed descriptions of the various Apollo functions, and numerous examples are available at http://www.ApolloChoiceModelling.com/examples.html. In addition, help files are available for all functions, using e.g. ?apollo_mnl
  3. Before asking a question on the forum, users are kindly requested to follow these steps:
    1. Check that the same issue has not already been addressed in the forum - there is a search tool.
    2. Ensure that the correct syntax has been used. For any function, detailed instructions are available directly in Apollo, e.g. by using ?apollo_mnl for apollo_mnl
    3. Check the frequently asked questions section on the Apollo website, which discusses some common issues/failures. Please see http://www.apollochoicemodelling.com/faq.html
    4. Make sure that R is using the latest official release of Apollo.
  4. If the above steps do not resolve the issue, then users should follow these steps when posting a question:
    1. provide full details on the issue, including the entire code and output, including any error messages
    2. posts will not immediately appear on the forum, but will be checked by a moderator first. This may take a day or two at busy times. There is no need to submit the post multiple times.

Comparing LL of the ICLV choice component with MMNL

Ask questions about post-estimation functions (e.g. prediction, conditionals, etc) or other processing of results.
Post Reply
chrismayer
Posts: 3
Joined: 21 Feb 2021, 15:04

Comparing LL of the ICLV choice component with MMNL

Post by chrismayer »

Hi Stephane and David,

I have a question regarding the comparison of LL value of the choice component of the ICLV model with the MMNL model. I use the same demographic variables for both the random variables and the latent variable as follows:

### Create random parameters
apollo_randCoeff = function(apollo_beta, apollo_inputs){
randcoeff = list()


randcoeff[["LV"]] = draws_LV+ gamma_Female_LV*Female + gamma_Age_20_30_LV*Age_20_30
randcoeff[["quality"]] = quality_mu + draws_quality + gamma_Female_quality*Female + gamma_Age_20_30_quality*Age_20_30
randcoeff[["rating"]] = rating_mu + draws_rating + gamma_Female_rating*Female + gamma_Age_20_30_rating*Age_20_30

return(randcoeff)
}

I incorporate the latent variable in the utility functions as follows:

quality_val = quality + lambda_quality*LV

### List of utilities
V1: ASC1 + alt1.quality*quality_val + alt1.rating*rating + ...
V2: ASC2 + alt2.quality*quality_val + alt2.rating*rating +...

When I estimate the reduced ICLV model without the measurement equations, the reduced model LL is about 20 units of LL better than that of the MMNL model. With the specification above, is the comparison correct?

Thank you for your help!
Chris
stephanehess
Site Admin
Posts: 974
Joined: 24 Apr 2020, 16:29

Re: Comparing LL of the ICLV choice component with MMNL

Post by stephanehess »

Hi Chris

can you show the full specification for both models, please. Also, when you say "the reduced model LL is about 20 units of LL better than that of the MMNL model", do you mean the reduced form model LL is better than the choice model part of the ICLV?

Stephane
--------------------------------
Stephane Hess
www.stephanehess.me.uk
chrismayer
Posts: 3
Joined: 21 Feb 2021, 15:04

Re: Comparing LL of the ICLV choice component with MMNL

Post by chrismayer »

Hi Stephan,

Thank you for the prompt reply. I compared the final LL of the reduced ICLV with the final LL of the MMNL. My understanding was that the two should be approximately the same. However, for me, the reduced ICLV performs better than the MMNL. Please see the specifications below:

MMNL:
# ################################################################# #
#### DEFINE MODEL PARAMETERS ####
# ################################################################# #

### Vector of parameters, including any that are kept fixed in estimation
apollo_beta = c(ASC_NO = 0,
ASC_1 = 0,
ASC_2 0,

quality_mu=0,
quality_sig=0.1,
brand_mu=0,
brand_sig=0.1,
rating_mu=0,
rating_sig=0.1,
price_mu=-3,
price_sig=0.1,

gamma_Age_20_30_brand = 0,
gamma_Age_20_30_quality = 0,
gamma_Age_20_30_rating = 0,
gamma_Age_20_30_price = 0,

gamma_Female_brand = 0,
gamma_Female_quality = 0,
gamma_Female_rating = 0,
gamma_Female_price = 0,
)

### Vector with names (in quotes) of parameters to be kept fixed at their starting value in apollo_beta, use apollo_beta_fixed = c() if none
apollo_fixed = c("ASC2")

# ################################################################# #
#### DEFINE RANDOM COMPONENTS ####
# ################################################################# #

### Set parameters for generating draws
apollo_draws = list(
interDrawsType = "mlhs",
interNDraws = 500,
interUnifDraws = c(),
interNormDraws = c("draws_price","draws_quality","draws_rating","draws_brand"),
intraDrawsType = "halton",
intraNDraws = 0,
intraUnifDraws = c(),
intraNormDraws = c()
)

### Create random parameters
apollo_randCoeff = function(apollo_beta, apollo_inputs){
randcoeff = list()

randcoeff[["quality"]] = quality_mu + draws_quality + gamma_Female_quality*Female + gamma_Age_20_30_quality*Age_20_30
randcoeff[["rating"]] = rating_mu + draws_rating + gamma_Female_rating*Female + gamma_Age_20_30_rating*Age_20_30
randcoeff[["brand"]] = brand_mu + draws_brand + gamma_Female_brand*Female + gamma_Age_20_30_brand*Age_20_30
randcoeff[["price"]] = -exp(price_mu + draws_price + gamma_Female_price*Female + gamma_Age_20_30_price*Age_20_30)

return(randcoeff)
}

# ################################################################# #
#### GROUP AND VALIDATE INPUTS ####
# ################################################################# #

apollo_inputs = apollo_validateInputs()

# ################################################################# #
#### DEFINE MODEL AND LIKELIHOOD FUNCTION ####
# ################################################################# #

apollo_probabilities=function(apollo_beta, apollo_inputs, functionality="estimate"){

### Function initialisation: do not change the following three commands
### Attach inputs and detach after function exit
apollo_attach(apollo_beta, apollo_inputs)
on.exit(apollo_detach(apollo_beta, apollo_inputs))

### Create list of probabilities P
P = list()


### List of utilities: these must use the same names as in mnl_settings, order is irrelevant
V = list()

V[['alt1']] = ASC_1 + alt1.quality*quality + alt1.rating*rating + alt1.brand*brand + alt1.price*price
V[['alt2']] = ASC_2 + alt2.quality*quality + alt2.rating*rating + alt2.brand*brand + alt2.price*price
V[['NO']] = ASC_NO


### Define settings for MNL model component
mnl_settings = list(
alternatives = c(alt1=1, alt2=2,NO=3),
avail = list(alt1=1,alt2=1,NO=1),
choiceVar = Choice,
V = V
)

### Compute probabilities using MNL model
P[['model']] = apollo_mnl(mnl_settings, functionality)

### Average across inter-individual draws
P = apollo_avgInterDraws(P, apollo_inputs, functionality)

### Prepare and return outputs of function
P = apollo_prepareProb(P, apollo_inputs, functionality)
return(P)
}

ICLV:

# ################################################################# #
#### DEFINE MODEL PARAMETERS ####
# ################################################################# #

### Vector of parameters, including any that are kept fixed in estimation
apollo_beta = c(ASC_NO = 0,
ASC_1 = 0,
ASC_2 0,

quality_mu=0,
quality_sig=0.1,
lambda_quality = 0.1,
brand_mu=0,
brand_sig=0.1,
lambda_brand = 0.1,
rating_mu=0,
rating_sig=0.1,
price_mu=-3,
price_sig=0.1,

gamma_Age_20_30_brand = 0,
gamma_Age_20_30_quality = 0,
gamma_Age_20_30_rating = 0,
gamma_Age_20_30_price = 0,

gamma_Female_brand = 0,
gamma_Female_quality = 0,
gamma_Female_rating = 0,
gamma_Female_price = 0,

gamma_Age_20_30_LV = 0,
gamma_Female_LV = 0,

zeta_ind_1 = 0,
tau_ind_1.1= -3,
tau_ind_1.2= -1,
tau_ind_1.3= 1,
tau_ind_1.4= 3,

zeta_ind_2 = 0,
tau_ind_2.1= -3,
tau_ind_2.2= -1,
tau_ind_2.3= 1,
tau_ind_2.4= 3,

zeta_ind_3 = 0,
tau_ind_3.1= -3,
tau_ind_3.2= -1,
tau_ind_3.3= 1,
tau_ind_3.4= 3
)

### Vector with names (in quotes) of parameters to be kept fixed at their starting value in apollo_beta, use apollo_beta_fixed = c() if none
apollo_fixed = c("ASC2")

# ################################################################# #
#### DEFINE RANDOM COMPONENTS ####
# ################################################################# #

### Set parameters for generating draws
apollo_draws = list(
interDrawsType = "mlhs",
interNDraws = 500,
interUnifDraws = c(),
interNormDraws = c("draws_price","draws_quality","draws_rating","draws_brand","draws_LV"),
intraDrawsType = "halton",
intraNDraws = 0,
intraUnifDraws = c(),
intraNormDraws = c()
)

### Create random parameters
apollo_randCoeff = function(apollo_beta, apollo_inputs){
randcoeff = list()


randcoeff[["LV"]] = draws_LV+ gamma_Female_LV*Female + gamma_Age_20_30_LV*Age_20_30
randcoeff[["quality"]] = quality_mu + draws_quality + gamma_Female_quality*Female + gamma_Age_20_30_quality*Age_20_30
randcoeff[["rating"]] = rating_mu + draws_rating + gamma_Female_rating*Female + gamma_Age_20_30_rating*Age_20_30
randcoeff[["brand"]] = brand_mu + draws_brand + gamma_Female_brand*Female + gamma_Age_20_30_brand*Age_20_30
randcoeff[["price"]] = -exp(price_mu + draws_price + gamma_Female_price*Female + gamma_Age_20_30_price*Age_20_30)

return(randcoeff)
}

# ################################################################# #
#### GROUP AND VALIDATE INPUTS ####
# ################################################################# #

apollo_inputs = apollo_validateInputs()

# ################################################################# #
#### DEFINE MODEL AND LIKELIHOOD FUNCTION ####
# ################################################################# #

apollo_probabilities=function(apollo_beta, apollo_inputs, functionality="estimate"){

### Function initialisation: do not change the following three commands
### Attach inputs and detach after function exit
apollo_attach(apollo_beta, apollo_inputs)
on.exit(apollo_detach(apollo_beta, apollo_inputs))

### Create list of probabilities P
P = list()


### Likelihood of indicators
ol_settings1 = list(outcomeOrdered=ind_1,
V=zeta_ind_1*LV,
tau=c(tau_ind_1.1, tau_ind_1.2, tau_ind_1.3, tau_ind_1.4),
rows=(task==1))
ol_settings2 = list(outcomeOrdered=ind_2,
V=zeta_ind_2*LV,
tau=c(tau_ind_2.1, tau_ind_2.2, tau_ind_2.3, tau_ind_2.4),
rows=(task==1))
ol_settings2 = list(outcomeOrdered=ind_3,
V=zeta_ind_3*LV,
tau=c(tau_ind_3.1, tau_ind_3.2, tau_ind_3.3, tau_ind_3.4),
rows=(task==1))

P[["ind_1"]] = apollo_ol(ol_settings1, functionality)
P[["ind_2"]] = apollo_ol(ol_settings2, functionality)
P[["ind_3"]] = apollo_ol(ol_settings3, functionality)


### Incorporating LVs in utilities

quality_val = quality + lambda_quality*LV
brand_val = brand + lambda_brand*LV

### List of utilities: these must use the same names as in mnl_settings, order is irrelevant
V = list()

V[['alt1']] = ASC_1 + alt1.quality*quality_val + alt1.rating*rating + alt1.brand*brand_val + alt1.price*price
V[['alt2']] = ASC_2 + alt2.quality*quality_val + alt2.rating*rating + alt2.brand*brand_val + alt2.price*price
V[['NO']] = ASC_NO


### Define settings for MNL model component
mnl_settings = list(
alternatives = c(alt1=1, alt2=2,NO=3),
avail = list(alt1=1,alt2=1,NO=1),
choiceVar = Choice,
V = V
)

### Compute probabilities using MNL model
P[['choice']] = apollo_mnl(mnl_settings, functionality)

### Likelihood of the whole model
P = apollo_combineModels(P, apollo_inputs, functionality)

### Take product across observation for same individual
P = apollo_panelProd(P, apollo_inputs, functionality)

### Average across inter-individual draws
P = apollo_avgInterDraws(P, apollo_inputs, functionality)

### Prepare and return outputs of function
P = apollo_prepareProb(P, apollo_inputs, functionality)
return(P)
}

Thank you for your help!

Chris
stephanehess
Site Admin
Posts: 974
Joined: 24 Apr 2020, 16:29

Re: Comparing LL of the ICLV choice component with MMNL

Post by stephanehess »

Hi

are you purposefully fixing the standard deviation of your random terms to 1? That is the difference between your MMNL and ICLV models.

Also, what you are showing here are the MMNL and ICLV model, not a reduced form model of the ICLV. The reduced form model of the ICLV would a model that has the same flexibility for the choice model as the ICLV model but does not include the measurement model in estimation (that's different from just calculating the LL for the choice part for a jointly estimated ICLV)


Stephane
--------------------------------
Stephane Hess
www.stephanehess.me.uk
chrismayer
Posts: 3
Joined: 21 Feb 2021, 15:04

Re: Comparing LL of the ICLV choice component with MMNL

Post by chrismayer »

Hi Stephane,

Thank you for looking at my code and sharing your opinion. I have a question based on the comments you provided as follows:

I was not aware that I purposefully fixed the standard deviations to 1. Could you please help me understand which part of the code leads to fixing the standard deviations to 1, and what is the implication of this on the estimated parameters?

Thank you!

Chris
stephanehess
Site Admin
Posts: 974
Joined: 24 Apr 2020, 16:29

Re: Comparing LL of the ICLV choice component with MMNL

Post by stephanehess »

Hi Chris

for example, you have:

Code: Select all

randcoeff[["quality"]] = quality_mu + draws_quality + gamma_Female_quality*Female + gamma_Age_20_30_quality*Age_20_30
you are not multiplying draws_quality by a parameter, and draws_quality is N(0,1).

Have a look at some of the MMNL examples on the Apollo website

Best wishes

Stephane
--------------------------------
Stephane Hess
www.stephanehess.me.uk
Post Reply