Apollo_lrTest

[anhntl]

Dear Professor,
I am using Apollo_lrtest to test the restricted and general model with same sample but got this error "Error in apollo_lrTest("MIXL_Pool", "MIXL_Width_Test") :
Number of observations not found in MIXL_Pool_output.txt".
It can be calculated manually but I would like to ask why I got this error. Thank you very much.

[stephanehess]

Hi

can you confirm which version of Apollo you are using and whether both models were estimated using that version

Thanks

Stephane

[anhntl]

Dear Professor,
I am using version 0.2.2, but I have just updated to 0.2.3 and the function can be run. Thank you.

[anhntl]

Dear Professor,
For my own estimation, I am focusing on the difference of preference between visitors and households. I want to control for differences in both mean sensitivity and sigmas.
My concerns are:
- How to choose the attribute having the same sensitivity between two groups to fix it in the final model.
- How to conclude that there is a difference between two groups on a particular attribute.

I intended to do it by LL ratio test:

My restricted model (all attributes homogenous between 2 groups):
V = beta_x * x + ...
My general model (testing if there is a difference of preference for only one attribute, other attributes homogenous between two groups):
V= (beta_xvisitor * visitor +beta_xhousehold *household) * x +....

- I use 500 MLHS draws
- I use different draws for tourist and residents, starting value of general model from restricted model.

But then I encountered the worse LL for the general model. I read another post in forum with same questions as mine: viewtopic.php?f=15&t=57

The reason of worsening LL is that I use different draws for two parameters. But this case is a little different than mine, and I dont know what is the reason behind:
- In his case: the two treatments are not specific to individual people but to individual choices, and he is losing the correlation for coefficient between choice tasks by applying separate draws. And you suggest estimating a correlated random parameters
- In my case: two parameters are specific to different respondents. And it might not be necessary to control for the correlation between visitors and household parameters?
- When I estimate model with the correlated parameter of tourists and visitors:
randcoeff[["b.width.h"]] = mu.b.width.h + sigma.b.width.h * draws.width.h
randcoeff[["b.width.v"]] = mu.b.width.v + sigma.b.width.h.v * draws.width.h + sigma.b.width.v * draws.width.v
the LL is still worse than the restricted model


My code is below. I look forward to hearing about your suggestion.
Thank you very much.

My restricted model
################ MIXL POOL #################

apollo_initialise()
apollo_control = list(
modelName ="MIXL_Pool500",
modelDescr ="Pool Residents-Tourists",
indivID ="id",
mixing=TRUE,panelData=TRUE,
nCores=3
)
database<- read.csv("D:\\Research\\Theses\\Second Chapter\\Working\\Data\\Final\\apollo_pool1.csv", header=T)
### Vector of parameters, including any that are kept fixed in estimation
apollo_beta=c( mu.b.width=1, sigma.b.width=0.1,# sigma.hsk.h=0.1,
sigma.hsk=0.1,
mu.b.asc = 1, mu.b.access=1,mu.b.tax=-1,
mu.b.facR=1, mu.b.facRT=1, mu.b.facT=1,
mu.b.strucG=1, mu.b.strucSB=1, mu.b.strucRC=1, mu.b.strucRStairs=1,
sigma.b.asc = 0.1, sigma.b.access=0.1,sigma.b.tax=0.1,
sigma.b.facR=0.1, sigma.b.facRT=0.1, sigma.b.facT=0.1,
sigma.b.strucG=0.1, sigma.b.strucSB=0.1, sigma.b.strucRC=0.1, sigma.b.strucRStairs=0.1)

### Vector with names (in quotes) of parameters to be kept fixed at their starting value in apollo_beta, use apollo_beta_fixed = c() if none
apollo_fixed = c()

### Read in starting values for at least some parameters from existing model output file
apollo_beta=apollo_readBeta(apollo_beta,apollo_fixed,"MNL_Pool",overwriteFixed = FALSE)

# ################################################################# #
#### DEFINE RANDOM COMPONENTS ####
# ################################################################# #

### Set parameters for generating draws
apollo_draws = list(
interDrawsType = "mlhs",
interNDraws = 500,
interUnifDraws = c(),
interNormDraws = c("draws.width",
"draws.asc","draws.access","draws.facR",
"draws.facRT","draws.facT", "draws.strucSB","draws.strucG",
"draws.strucRC","draws.strucRStairs","draws.tax", #"draws.hsk.h",
"draws.hsk1","draws.hsk2","draws.hsk3"),
intraDrawsType = "halton",
intraNDraws = 0,
intraUnifDraws = c(),
intraNormDraws = c()
)

### Create random parameters
apollo_randCoeff = function(apollo_beta, apollo_inputs){
randcoeff = list()
randcoeff[["b.width"]] = mu.b.width + sigma.b.width * draws.width
#randcoeff[["ec.h"]] = sigma.hsk.h * draws.hsk.h
randcoeff[["ec1"]] = sigma.hsk * draws.hsk1
randcoeff[["ec2"]] = sigma.hsk * draws.hsk2
randcoeff[["ec3"]] = sigma.hsk * draws.hsk3

randcoeff[["b.asc"]] = mu.b.asc + sigma.b.asc * draws.asc
randcoeff[["b.access"]] = mu.b.access + sigma.b.access * draws.access
randcoeff[["b.tax"]] = -exp( mu.b.tax + sigma.b.tax * draws.tax)
randcoeff[["b.facR"]] = mu.b.facR + sigma.b.facR * draws.facR
randcoeff[["b.facRT"]] = mu.b.facRT + sigma.b.facRT * draws.facRT
randcoeff[["b.facT"]] = mu.b.facT + sigma.b.facT * draws.facT
randcoeff[["b.strucSB"]] = mu.b.strucSB + sigma.b.strucSB * draws.strucSB
randcoeff[["b.strucRC"]] = mu.b.strucRC + sigma.b.strucRC * draws.strucRC
randcoeff[["b.strucG"]] = mu.b.strucG + sigma.b.strucG * draws.strucG
randcoeff[["b.strucRStairs"]] = mu.b.strucRStairs + sigma.b.strucRStairs * draws.strucRStairs

return(randcoeff)
}

# ################################################################# #
#### GROUP AND VALIDATE INPUTS ####
# ################################################################# #

apollo_inputs = apollo_validateInputs()


# ################################################################# #
#### DEFINE MODEL AND LIKELIHOOD FUNCTION ####
# ################################################################# #

apollo_probabilities=function(apollo_beta, apollo_inputs, functionality="estimate"){

### Function initialisation: do not change the following three commands
### Attach inputs and detach after function exit
apollo_attach(apollo_beta, apollo_inputs)
on.exit(apollo_detach(apollo_beta, apollo_inputs))

### Create list of probabilities P
P = list()

### List of utilities: these must use the same names as in mnl_settings, order is irrelevant
V = list()
V[['alt.1']] = b.asc+ b.access* access.1+b.facR* facR.1+b.width*width.1+
b.facRT* facRT.1+b.facT* facT.1+b.strucRC* strucRC.1+
b.strucSB* strucSB.1+#ec.h*hh+
ec1*hh

V[['alt.2']] = b.access* access.2+b.facR* facR.2+b.width*width.2+
b.facRT* facRT.2+b.facT* facT.2+b.strucG* strucG.2+b.strucRC* strucRC.2+
b.strucRStairs* strucRStairs.2+b.strucSB* strucSB.2+b.tax* tax.2+ec2*hh

V[['alt.3']] = b.width*width.3+
b.access* access.3+b.facR* facR.3+
b.facRT* facRT.3+b.facT* facT.3+b.strucG* strucG.3+b.strucRC* strucRC.3+
b.strucRStairs* strucRStairs.3+b.strucSB* strucSB.3+b.tax* tax.3+ec3*hh

### Define settings for MNL model component
mnl_settings = list(
alternatives = c(alt.1=1, alt.2=2,alt.3=3),
avail = list(alt.1=1, alt.2=1,alt.3=1),
choiceVar = choice,
V = V
)

### Compute probabilities using MNL model
P[['model']] = apollo_mnl(mnl_settings, functionality)

### Take product across observation for same individual
P = apollo_panelProd(P, apollo_inputs, functionality)

### Average across inter-individual draws
P = apollo_avgInterDraws(P, apollo_inputs, functionality)

### Prepare and return outputs of function
P = apollo_prepareProb(P, apollo_inputs, functionality)
return(P)
}

# ################################################################# #
#### MODEL ESTIMATION ####
# ################################################################# #

model = apollo_estimate(apollo_beta, apollo_fixed,
apollo_probabilities, apollo_inputs)#, estimate_settings=list(hessianRoutine="maxLik"))

apollo_modelOutput(model)
apollo_saveOutput(model)
# ################################################################# #


My general model

################ Width Test #################

apollo_initialise()
apollo_control = list(
modelName ="MIXL_Width_Test",
modelDescr ="Pool Residents-Tourists",
indivID ="id",
mixing=TRUE,panelData=TRUE,
nCores=3
)
database<- read.csv("D:\\Research\\Theses\\Second Chapter\\Working\\Data\\Final\\apollo_pool1.csv", header=T)
### Vector of parameters, including any that are kept fixed in estimation
apollo_beta=c( mu.b.width.h=1, sigma.b.width.h=0.1, sigma.hsk=0.1,
mu.b.width.v=1, sigma.b.width.v=0.1,
mu.b.asc = 1, mu.b.access=1,mu.b.tax=-1,
mu.b.facR=1, mu.b.facRT=1, mu.b.facT=1,
mu.b.strucG=1, mu.b.strucSB=1, mu.b.strucRC=1, mu.b.strucRStairs=1,
sigma.b.asc = 0.1, sigma.b.access=0.1,sigma.b.tax=0.1,
sigma.b.facR=0.1, sigma.b.facRT=0.1, sigma.b.facT=0.1,
sigma.b.strucG=0.1, sigma.b.strucSB=0.1, sigma.b.strucRC=0.1, sigma.b.strucRStairs=0.1)

### Vector with names (in quotes) of parameters to be kept fixed at their starting value in apollo_beta, use apollo_beta_fixed = c() if none
apollo_fixed = c()

### Read in starting values for at least some parameters from existing model output file
apollo_beta=apollo_readBeta(apollo_beta,apollo_fixed,"MIXL_Pool500",overwriteFixed = FALSE)

# ################################################################# #
#### DEFINE RANDOM COMPONENTS ####
# ################################################################# #

### Set parameters for generating draws
apollo_draws = list(
interDrawsType = "mlhs",
interNDraws = 500,
interUnifDraws = c(),
interNormDraws = c("draws.width.h","draws.width.v",
"draws.asc","draws.access","draws.facR",
"draws.facRT","draws.facT", "draws.strucSB","draws.strucG",
"draws.strucRC","draws.strucRStairs","draws.tax", #"draws.hsk.h",
"draws.hsk1","draws.hsk2","draws.hsk3"),
intraDrawsType = "halton",
intraNDraws = 0,
intraUnifDraws = c(),
intraNormDraws = c()
)

### Create random parameters
apollo_randCoeff = function(apollo_beta, apollo_inputs){
randcoeff = list()
randcoeff[["b.width.h"]] = mu.b.width.h + sigma.b.width.h * draws.width.h
randcoeff[["b.width.v"]] = mu.b.width.v + sigma.b.width.v * draws.width.v
randcoeff[["ec1"]] = sigma.hsk * draws.hsk1
randcoeff[["ec2"]] = sigma.hsk * draws.hsk2
randcoeff[["ec3"]] = sigma.hsk * draws.hsk3

randcoeff[["b.asc"]] = mu.b.asc + sigma.b.asc * draws.asc
randcoeff[["b.access"]] = mu.b.access + sigma.b.access * draws.access
randcoeff[["b.tax"]] = -exp( mu.b.tax + sigma.b.tax * draws.tax)
randcoeff[["b.facR"]] = mu.b.facR + sigma.b.facR * draws.facR
randcoeff[["b.facRT"]] = mu.b.facRT + sigma.b.facRT * draws.facRT
randcoeff[["b.facT"]] = mu.b.facT + sigma.b.facT * draws.facT
randcoeff[["b.strucSB"]] = mu.b.strucSB + sigma.b.strucSB * draws.strucSB
randcoeff[["b.strucRC"]] = mu.b.strucRC + sigma.b.strucRC * draws.strucRC
randcoeff[["b.strucG"]] = mu.b.strucG + sigma.b.strucG * draws.strucG
randcoeff[["b.strucRStairs"]] = mu.b.strucRStairs + sigma.b.strucRStairs * draws.strucRStairs

return(randcoeff)
}

# ################################################################# #
#### GROUP AND VALIDATE INPUTS ####
# ################################################################# #

apollo_inputs = apollo_validateInputs()


# ################################################################# #
#### DEFINE MODEL AND LIKELIHOOD FUNCTION ####
# ################################################################# #

apollo_probabilities=function(apollo_beta, apollo_inputs, functionality="estimate"){

### Function initialisation: do not change the following three commands
### Attach inputs and detach after function exit
apollo_attach(apollo_beta, apollo_inputs)
on.exit(apollo_detach(apollo_beta, apollo_inputs))

### Create list of probabilities P
P = list()

### List of utilities: these must use the same names as in mnl_settings, order is irrelevant
V = list()
V[['alt.1']] = b.asc+ b.access* access.1+b.facR* facR.1+
b.facRT* facRT.1+b.facT* facT.1+b.strucRC* strucRC.1+
b.strucSB* strucSB.1+b.width.h*width.1*hh+b.width.v*width.1*vit+
ec1*hh

V[['alt.2']] = b.access* access.2+b.facR* facR.2+b.width.h*width.2*hh+b.width.v*width.2*vit+
b.facRT* facRT.2+b.facT* facT.2+b.strucG* strucG.2+b.strucRC* strucRC.2+
b.strucRStairs* strucRStairs.2+b.strucSB* strucSB.2+b.tax* tax.2+ec2*hh

V[['alt.3']] = b.width.h*width.3*hh+b.width.v*width.3*vit+
b.access* access.3+b.facR* facR.3+
b.facRT* facRT.3+b.facT* facT.3+b.strucG* strucG.3+b.strucRC* strucRC.3+
b.strucRStairs* strucRStairs.3+b.strucSB* strucSB.3+b.tax* tax.3+ec3*hh

### Define settings for MNL model component
mnl_settings = list(
alternatives = c(alt.1=1, alt.2=2,alt.3=3),
avail = list(alt.1=1, alt.2=1,alt.3=1),
choiceVar = choice,
V = V
)

### Compute probabilities using MNL model
P[['model']] = apollo_mnl(mnl_settings, functionality)

### Take product across observation for same individual
P = apollo_panelProd(P, apollo_inputs, functionality)

### Average across inter-individual draws
P = apollo_avgInterDraws(P, apollo_inputs, functionality)

### Prepare and return outputs of function
P = apollo_prepareProb(P, apollo_inputs, functionality)
return(P)
}

# ################################################################# #
#### MODEL ESTIMATION ####
# ################################################################# #

model = apollo_estimate(apollo_beta, apollo_fixed,
apollo_probabilities, apollo_inputs)#, estimate_settings=list(hessianRoutine="maxLik"))

apollo_modelOutput(model)
apollo_saveOutput(model)
apollo_lrTest("MIXL_Pool500","MIXL_Width_Test")

[stephanehess]

Hi

I would estimate two separate models first, and then see which parameter is the most similar in the two models and use that as the shared one. You also need to allow for scale differences between the two samples. Then you can do t-ratios for differences between the other parameters

Stephane

[anhntl]

Dear Professor Stephane,
Thank you very much for your suggestion. I still have a question to make clear my understanding:
In our shared model that I accounted for scale difference, parameters specified for each group are random distributed. In case that both mean and sigma of both groups' parameter for a particular attribute are significant, can I conclude that preference for this attribute is different (mean of sensitivity is different and distribution is different).
Or do I have to take further test:
- should I take a Poe test to see the difference of two distributions or
- LL ratio test the restricted model and general model

Thank you very much for your time.
Best,
Anh

[stephanehess]

Hi

when you say "In case that both mean and sigma of both groups' parameter for a particular attribute are significant", this does not tell us anything about the difference between the two groups. For that, you should make comparisons of the parameters, using for example the delta method

Stephane

[anhntl]

Dear Professor Stephane,
Thank you very much for your help.