Important: Read this before posting to this forum

  1. This forum is for questions related to the use of Apollo. We will answer some general choice modelling questions too, where appropriate, and time permitting. We cannot answer questions about how to estimate choice models with other software packages.
  2. There is a very detailed manual for Apollo available at http://www.ApolloChoiceModelling.com/manual.html. This contains detailed descriptions of the various Apollo functions, and numerous examples are available at http://www.ApolloChoiceModelling.com/examples.html. In addition, help files are available for all functions, using e.g. ?apollo_mnl
  3. Before asking a question on the forum, users are kindly requested to follow these steps:
    1. Check that the same issue has not already been addressed in the forum - there is a search tool.
    2. Ensure that the correct syntax has been used. For any function, detailed instructions are available directly in Apollo, e.g. by using ?apollo_mnl for apollo_mnl
    3. Check the frequently asked questions section on the Apollo website, which discusses some common issues/failures. Please see http://www.apollochoicemodelling.com/faq.html
    4. Make sure that R is using the latest official release of Apollo.
  4. If the above steps do not resolve the issue, then users should follow these steps when posting a question:
    1. provide full details on the issue, including the entire code and output, including any error messages
    2. posts will not immediately appear on the forum, but will be checked by a moderator first. This may take a day or two at busy times. There is no need to submit the post multiple times.

Interpreting tables of coefficients

Ask general questions about model specification and estimation that are not Apollo specific but relevant to Apollo users.
Post Reply
marah
Posts: 15
Joined: 13 Jan 2022, 03:06

Interpreting tables of coefficients

Post by marah »

Hi,
I have a table of results for best-worst scaling (using the apollo_combineModels method) and a table of results for a discrete choice experiment. In both tables, I provide coefficients, standard errors and p-values. My PhD supervisor has asked me to interpret the tables on a general level, including both what the dependent variable is and what the coefficients mean. Is the following correct?
I am looking at the relative importance of barriers to investing in (say) energy efficiency for the BWS. In the DCE, I look at characteristics (attributes and levels) of different policy options.

For best-worst scaling:
"In the table, the dependent variable is a respondent's BWS score for each barrier, which is a measure of the number of times the respondent chooses the barrier as the most important (the best option) versus the least important (the worst option). The best and worst components of the BWS score are both given values of 1,2,3,4, or 5 corresponding to the orders of the barriers chosen as most and least important in a given choice task. The coefficients represent higher (positive coefficients) or lower (negative coefficients) best-worst scaling scores for each barrier compared to the “not considered” barrier. The models allow for different scale parameters for the choices of most and least important barriers. Scale parameters measure variance of the error term, with a higher scale parameter implying lower variance."

For DCE:
"In these tables, the dependent variable is binary, equalling one for a preferred policy in a given choice task and zero otherwise. The coefficients can be interpreted as marginal utilities, with positive coefficients implying that policy options with these characteristics are more likely to be chosen and the opposite for negative coefficients, all else equal."

It is my first time trying to analyse my own choice experiments and my supervisor unfortunately has even less experience with choice experiments so any help with my interpretation would be greatly appreciated. Thank you.
stephanehess
Site Admin
Posts: 974
Joined: 24 Apr 2020, 16:29

Re: Interpreting tables of coefficients

Post by stephanehess »

Marah

the part about the discrete choice model seems fine. For the BWS one, can you show us the model?

Stephane
--------------------------------
Stephane Hess
www.stephanehess.me.uk
marah
Posts: 15
Joined: 13 Jan 2022, 03:06

Re: Interpreting tables of coefficients

Post by marah »

Thanks for your reply. Sorry but what do you mean by the model?

In the survey, respondents saw each barrier five times, completed eleven choice sets, and barriers were presented in groups of five. In each question, respondents were asked: “When you think about reasons that you haven’t [made specific energy efficiency improvements], which is the MOST and LEAST important reason from the list below?”.

I can also send you an example table but I don't think attachments or screenshots are possible.
stephanehess
Site Admin
Posts: 974
Joined: 24 Apr 2020, 16:29

Re: Interpreting tables of coefficients

Post by stephanehess »

Thanks. And how did you model that data? Using all the possible best-worst combination as alternatives?
--------------------------------
Stephane Hess
www.stephanehess.me.uk
marah
Posts: 15
Joined: 13 Jan 2022, 03:06

Re: Interpreting tables of coefficients

Post by marah »

Thanks, that makes sense. I'm quite sure I did use all the possible best-worst combination as alternatives.

I ran the following codes for my main table of BWS results:

Code: Select all

# ################################################################# #
#### LOAD LIBRARY AND DEFINE CORE SETTINGS                       ####
# ################################################################# #

### Clear memory
rm(list = ls())

### Load Apollo library
library(apollo)

### Initialise code
apollo_initialise()

### Set core controls
apollo_control = list(
  modelName       = "Apollo BW simultaneous",
  modelDescr      = "Best-worst model simultaneous",
  indivID         = "id", 
  outputDirectory = "output"
)

# ################################################################# #
#### LOAD DATA AND APPLY ANY TRANSFORMATIONS                     ####
# ################################################################# #

### Loading data from package
### if data is to be loaded from a file (e.g. called data.csv), 
### the code would be: database = read.csv("data.csv",header=TRUE)
library(readxl)
database = read_excel("...")
#View(database)
#attach(database)
### for data dictionary, use ?apollo_drugChoiceData

# ################################################################# #
#### DEFINE MODEL PARAMETERS                                     ####
# ################################################################# #

### Vector of parameters, including any that are kept fixed in estimation
apollo_beta=c(bafford     = 0,
              bpayback      = 0,
              bmargins      = 0,
              brentalmarket     = 0,
              bsavings     = 0,
              brent    = 0,
              bresale     = 0,
              bnoinstall     = 0,
              btime       = 0,
              bnoconsider  = 0,
              bgov       = 0,
              asc_alt1 = 0,
              asc_alt2 = 0,
              asc_alt3 = 0,
              asc_alt4 = 0,
              asc_alt5 = 0,
              mu_worst       = 1)

### Vector with names (in quotes) of parameters to be kept fixed at their starting value in apollo_beta, use apollo_beta_fixed = c() if none
apollo_fixed = c("asc_alt5","bnoconsider")

# ################################################################# #
#### GROUP AND VALIDATE INPUTS                                   ####
# ################################################################# #

apollo_inputs = apollo_validateInputs()

# ################################################################# #
#### DEFINE MODEL AND LIKELIHOOD FUNCTION                        ####
# ################################################################# #

apollo_probabilities=function(apollo_beta, apollo_inputs, functionality="estimate"){
  
  ### Attach inputs and detach after function exit
  apollo_attach(apollo_beta, apollo_inputs)
  on.exit(apollo_detach(apollo_beta, apollo_inputs))
  
  ### Create list of probabilities P
  P = list()
  
  ### List of utilities: these must use the same names as in mnl_settings, order is irrelevant
  V = list()
  V[["alt1"]] = ( asc_alt1 + bafford*afford1 + bpayback*payback1 + bmargins*margins1 + brentalmarket*rentalmarket1 + bsavings*savings1 + brent*rent1 + bresale*resale1 + bnoinstall*noinstall1 + btime*time1 + bnoconsider*noconsider1 + bgov*gov1)
  
  V[["alt2"]] = ( asc_alt2 + bafford*afford2 + bpayback*payback2 + bmargins*margins2 + brentalmarket*rentalmarket2 + bsavings*savings2 + brent*rent2 + bresale*resale2 + bnoinstall*noinstall2 + btime*time2 + bnoconsider*noconsider2 + bgov*gov2)
  
  V[["alt3"]] = ( asc_alt3 + bafford*afford3 + bpayback*payback3 + bmargins*margins3 + brentalmarket*rentalmarket3 + bsavings*savings3 + brent*rent3 + bresale*resale3 + bnoinstall*noinstall3 + btime*time3 + bnoconsider*noconsider3 + bgov*gov3)
  
  V[["alt4"]] = ( asc_alt4 + bafford*afford4 + bpayback*payback4 + bmargins*margins4 + brentalmarket*rentalmarket4 + bsavings*savings4 + brent*rent4 + bresale*resale4 + bnoinstall*noinstall4 + btime*time4 + bnoconsider*noconsider4 + bgov*gov4)
  
  V[["alt5"]] = ( asc_alt5 + bafford*afford5 + bpayback*payback5 + bmargins*margins5 + brentalmarket*rentalmarket5 + bsavings*savings5 + brent*rent5 + bresale*resale5 + bnoinstall*noinstall5 + btime*time5 + bnoconsider*noconsider5 + bgov*gov5)
  
  ### Compute probabilities for combined "best" and "worst" choice using MNL model
  mnl_settings = list(
    alternatives  = c(alt_b1_w2=12,
                      alt_b1_w3=13, 
                      alt_b1_w4=14,
                      alt_b1_w5=15,
                      alt_b2_w1=21, 
                      alt_b2_w3=23,
                      alt_b2_w4=24,
                      alt_b2_w5=25,
                      alt_b3_w1=31,
                      alt_b3_w2=32, 
                      alt_b3_w4=34,
                      alt_b3_w5=35,
                      alt_b4_w1=41,
                      alt_b4_w2=42,
                      alt_b4_w3=43,
                      alt_b4_w5=45,
                      alt_b5_w1=51,
                      alt_b5_w2=52,
                      alt_b5_w3=53,
                      alt_b5_w4=54),
    choiceVar     = 10*best1+worst1, 
    utilities     = list(alt_b1_w2=V[["alt1"]]-mu_worst*V[["alt2"]],
                         alt_b1_w3=V[["alt1"]]-mu_worst*V[["alt3"]],
                         alt_b1_w4=V[["alt1"]]-mu_worst*V[["alt4"]],
                         alt_b1_w5=V[["alt1"]]-mu_worst*V[["alt5"]],
                         alt_b2_w1=V[["alt2"]]-mu_worst*V[["alt1"]],
                         alt_b2_w3=V[["alt2"]]-mu_worst*V[["alt3"]],
                         alt_b2_w4=V[["alt2"]]-mu_worst*V[["alt4"]],
                         alt_b2_w5=V[["alt2"]]-mu_worst*V[["alt5"]],
                         alt_b3_w1=V[["alt3"]]-mu_worst*V[["alt1"]],
                         alt_b3_w2=V[["alt3"]]-mu_worst*V[["alt2"]],
                         alt_b3_w4=V[["alt3"]]-mu_worst*V[["alt4"]],
                         alt_b3_w5=V[["alt3"]]-mu_worst*V[["alt5"]],
                         alt_b4_w1=V[["alt4"]]-mu_worst*V[["alt1"]],
                         alt_b4_w2=V[["alt4"]]-mu_worst*V[["alt2"]],
                         alt_b4_w3=V[["alt4"]]-mu_worst*V[["alt3"]],
                         alt_b4_w5=V[["alt4"]]-mu_worst*V[["alt5"]],
                         alt_b5_w1=V[["alt5"]]-mu_worst*V[["alt1"]],
                         alt_b5_w2=V[["alt5"]]-mu_worst*V[["alt2"]],
                         alt_b5_w3=V[["alt5"]]-mu_worst*V[["alt3"]],
                         alt_b5_w4=V[["alt5"]]-mu_worst*V[["alt4"]])
  )
  
  P[["model"]] = apollo_mnl(mnl_settings, functionality)
  
  P=apollo_panelProd(P,apollo_inputs,functionality)
  P=apollo_prepareProb(P,apollo_inputs,functionality)
  return(P)
}
  
  # ################################################################# #
  #### MODEL ESTIMATION                                            ####
  # ################################################################# #
  
  model = apollo_estimate(apollo_beta, apollo_fixed, apollo_probabilities, apollo_inputs)

  # ################################################################# #
  #### MODEL OUTPUTS                                               ####
  # ################################################################# #
  
# ----------------------------------------------------------------- #
#---- FORMATTED OUTPUT (TO SCREEN)                               ----
# ----------------------------------------------------------------- #

modelOutput_settings = list(printPVal=2)

apollo_modelOutput(model,modelOutput_settings)

deltaMethod_settings=list(expression=c(isdiff = "brent-bafford"))
apollo_deltaMethod(model, deltaMethod_settings)

# ----------------------------------------------------------------- #
#---- FORMATTED OUTPUT (TO FILE, using model name)               ----
# ----------------------------------------------------------------- #


saveOutput_settings = list(printPVal=2)

apollo_saveOutput(model, saveOutput_settings)  
And:

Code: Select all

# ################################################################# #
#### LOAD LIBRARY AND DEFINE CORE SETTINGS                       ####
# ################################################################# #

### Clear memory
rm(list = ls())

### Load Apollo library
library(apollo)

### Initialise code
apollo_initialise()

### Set core controls
apollo_control = list(
  modelName       = "Apollo BW separate fixed",
  modelDescr      = "Best-worst model separate",
  indivID         = "id", 
  outputDirectory = "output"
)

# ################################################################# #
#### LOAD DATA AND APPLY ANY TRANSFORMATIONS                     ####
# ################################################################# #

### Loading data from package
### if data is to be loaded from a file (e.g. called data.csv), 
### the code would be: database = read.csv("data.csv",header=TRUE)
library(readxl)
database = read_excel("...")
#View(database)
#attach(database)

# ################################################################# #
#### DEFINE MODEL PARAMETERS                                     ####
# ################################################################# #

### Vector of parameters, including any that are kept fixed in estimation
apollo_beta=c(bafford     = 0,
              bpayback      = 0,
              bmargins      = 0,
              brentalmarket     = 0,
              bsavings     = 0,
              brent    = 0,
              bresale     = 0,
              bnoinstall     = 0,
              btime       = 0,
              bnoconsider  = 0,
              bgov       = 0,
              asc_alt1 = 0,
              asc_alt2 = 0,
              asc_alt3 = 0,
              asc_alt4 = 0,
              asc_alt5 = 0,
              mu_worst       = 1)

### Vector with names (in quotes) of parameters to be kept fixed at their starting value in apollo_beta, use apollo_beta_fixed = c() if none
apollo_fixed = c("asc_alt5","bnoconsider")

# ################################################################# #
#### GROUP AND VALIDATE INPUTS                                   ####
# ################################################################# #

apollo_inputs = apollo_validateInputs()

# ################################################################# #
#### DEFINE MODEL AND LIKELIHOOD FUNCTION                        ####
# ################################################################# #

apollo_probabilities=function(apollo_beta, apollo_inputs, functionality="estimate"){
  
  ### Attach inputs and detach after function exit
  apollo_attach(apollo_beta, apollo_inputs)
  on.exit(apollo_detach(apollo_beta, apollo_inputs))
  
  ### Create list of probabilities P
  P = list()
  
  ### List of utilities: these must use the same names as in mnl_settings, order is irrelevant
  V = list()
  V[["alt1"]] = asc_alt1 + bafford*afford1 + bpayback*payback1 + bmargins*margins1 + brentalmarket*rentalmarket1 + bsavings*savings1 + brent*rent1 + bresale*resale1 + bnoinstall*noinstall1 + btime*time1 + bnoconsider*noconsider1 + bgov*gov1
  
  V[["alt2"]] = asc_alt2 + bafford*afford2 + bpayback*payback2 + bmargins*margins2 + brentalmarket*rentalmarket2 + bsavings*savings2 + brent*rent2 + bresale*resale2 + bnoinstall*noinstall2 + btime*time2 + bnoconsider*noconsider2 + bgov*gov2
  
  V[["alt3"]] = asc_alt3 + bafford*afford3 + bpayback*payback3 + bmargins*margins3 + brentalmarket*rentalmarket3 + bsavings*savings3 + brent*rent3 + bresale*resale3 + bnoinstall*noinstall3 + btime*time3 + bnoconsider*noconsider3 + bgov*gov3
  
  V[["alt4"]] = asc_alt4 + bafford*afford4 + bpayback*payback4 + bmargins*margins4 + brentalmarket*rentalmarket4 + bsavings*savings4 + brent*rent4 + bresale*resale4 + bnoinstall*noinstall4 + btime*time4 + bnoconsider*noconsider4 + bgov*gov4
  
  V[["alt5"]] = asc_alt5 + bafford*afford5 + bpayback*payback5 + bmargins*margins5 + brentalmarket*rentalmarket5 + bsavings*savings5 + brent*rent5 + bresale*resale5 + bnoinstall*noinstall5 + btime*time5 + bnoconsider*noconsider5 + bgov*gov5
  
  ### Compute probabilities for "best" choice using MNL model
  mnl_settings_best = list(
    alternatives  = c(alt1=1, alt2=2, alt3=3, alt4=4, alt5=5),
    avail         = list(alt1=1, alt2=1, alt3=1, alt4=1, alt5=1),
    choiceVar     = best1,
    utilities     = V,
    componentName = "best"
  )
  P[["choice_best"]] = apollo_mnl(mnl_settings_best, functionality)
  
  ### Compute probabilities for "worst" choice using MNL model
  mnl_settings_best = list(
    alternatives  = c(alt1=1, alt2=2, alt3=3, alt4=4, alt5=5),
    avail         = list(alt1=(best1!=1), alt2=(best1!=2), alt3=(best1!=3), alt4=(best1!=4), alt5=(best1!=5)),
    choiceVar     = worst1,
    utilities     = list(alt1=-mu_worst*V[["alt1"]],
                         alt2=-mu_worst*V[["alt2"]],
                         alt3=-mu_worst*V[["alt3"]],
                         alt4=-mu_worst*V[["alt4"]],
                         alt5=-mu_worst*V[["alt5"]]),
    componentName = "worst"
  )
  P[["choice_worst"]] = apollo_mnl(mnl_settings_best, functionality)
  
  P=apollo_combineModels(P,apollo_inputs,functionality)
  P=apollo_panelProd(P,apollo_inputs,functionality)
  P=apollo_prepareProb(P,apollo_inputs,functionality)
  return(P)
}
  
  # ################################################################# #
  #### MODEL ESTIMATION                                            ####
  # ################################################################# #
  
  model = apollo_estimate(apollo_beta, apollo_fixed, apollo_probabilities, apollo_inputs)

  # ################################################################# #
  #### MODEL OUTPUTS                                               ####
  # ################################################################# #
  
  # ----------------------------------------------------------------- #
  #---- FORMATTED OUTPUT (TO SCREEN)                               ----
  # ----------------------------------------------------------------- #

modelOutput_settings = list(printPVal=2)

apollo_modelOutput(model,modelOutput_settings)

deltaMethod_settings=list(expression=c(isdiff = "brent-bafford"))
apollo_deltaMethod(model, deltaMethod_settings)
  
  # ----------------------------------------------------------------- #
  #---- FORMATTED OUTPUT (TO FILE, using model name)               ----
  # ----------------------------------------------------------------- #


saveOutput_settings = list(printPVal=2)

apollo_saveOutput(model, saveOutput_settings)
Thank you!
marah
Posts: 15
Joined: 13 Jan 2022, 03:06

Re: Interpreting tables of coefficients

Post by marah »

I just realised what you were asking for with the model (I think). I used simple multinomial logit models.
stephanehess
Site Admin
Posts: 974
Joined: 24 Apr 2020, 16:29

Re: Interpreting tables of coefficients

Post by stephanehess »

Mara

sorry for the slow reply.

The code looks correct, but I don't think you can describe this as modelling the best-worst scores. What you are doing (which I believe is correct) is that you model the choice of best and worst at the same time. So your "dependent" variable is not the BWS score, but the best-worst pair, and a positive coefficient simply implies that an item is more likely to be chosen as best, with the opposite for a negative coeffiicent

Stephane
--------------------------------
Stephane Hess
www.stephanehess.me.uk
Post Reply