Hi everyone,
I am rather new to R and Apollo, however, based on your examples I managed to got an MNL estimation for my data set running. I did a split sample approach and for each sample I estimated the parameters. Now I wanted to see, if my estimations for the attributes differ significantly between the two samples. For the analysis I wanted to use the Poe test (R Package "mded"), which measures the difference between two independent or nonindependent empirical distributions and should return a significance level of the difference.
However, from my understanding, for this test I need to have the individual estimations for the respondents for each attribute. And currently I have really no idea, how to get these from my MNL estimations. Is there any chance that you could help me out on this and give me a hint, how to get the attribute estimations for each individual? Or is that not necessary?
I am really looking for you answer! Thank you very much in advance!
Best
Sutsch
Important: Read this before posting to this forum
 This forum is for questions related to the use of Apollo. We will answer some general choice modelling questions too, where appropriate, and time permitting. We cannot answer questions about how to estimate choice models with other software packages.
 There is a very detailed manual for Apollo available at http://www.ApolloChoiceModelling.com/manual.html. This contains detailed descriptions of the various Apollo functions, and numerous examples are available at http://www.ApolloChoiceModelling.com/examples.html.
 Before asking a question on the forum, users are kindly requested to follow these steps:
 Check that the same issue has not already been addressed in the forum  there is a search tool.
 Ensure that the correct syntax has been used. For any function, detailed instructions are available directly in Apollo, e.g. by using ?apollo_mnl for apollo_mnl
 Check the frequently asked questions section on the Apollo website, which discusses some common issues/failures. Please see http://www.apollochoicemodelling.com/faq.html
 Make sure that R is using the latest official release of Apollo.
 Users can check which version they are running by entering packageVersion("apollo").
 Then check what is the latest full release (not development version) at http://www.ApolloChoiceModelling.com/code.html.
 To update to the latest official version, just enter install.packages("apollo"). To update to a development version, download the appropriate binary file from http://www.ApolloChoiceModelling.com/code.html, and install the package from file
 If the above steps do not resolve the issue, then users should follow these steps when posting a question:
 provide full details on the issue, including the entire code and output, including any error messages
 posts will not immediately appear on the forum, but will be checked by a moderator first. This may take a day or two at busy times. There is no need to submit the post multiple times.
Test for difference, split sample MNL, Poe test

 Site Admin
 Posts: 558
 Joined: 24 Apr 2020, 16:29
Re: Test for difference, split sample MNL, Poe test
Sutsch
let me first clear up a confusion here. There is no such thing as individualspecific estimates with a choice model, unless you estimated a separate model for each person. What you maybe refer to here are Bayesian posteriors for a Mixed Logit model, but they are still driven by a sample level model.
Either way, none of these applies in an MNL context, where your parameters are fixed across the sample within a model.
In your case, you are interested in comparing the estimates from two models on nonoverlapping samples. The problem is that in that case, there are potential scale differences between the two models (differential role of the variance of the extreme value error terms) and as a result, the parameters cannot be directly compared. What you could do is compare marginal rates of substitution such as WTP measures between the two models. You could calculate these using apollo_deltaMethod for each of the models, and then you can do a simple ttest for the difference between the two samples (MRS_model1MRS_model2)/(s.e.MRS_model_1+s.e.MRS_model_2).
Stephane
let me first clear up a confusion here. There is no such thing as individualspecific estimates with a choice model, unless you estimated a separate model for each person. What you maybe refer to here are Bayesian posteriors for a Mixed Logit model, but they are still driven by a sample level model.
Either way, none of these applies in an MNL context, where your parameters are fixed across the sample within a model.
In your case, you are interested in comparing the estimates from two models on nonoverlapping samples. The problem is that in that case, there are potential scale differences between the two models (differential role of the variance of the extreme value error terms) and as a result, the parameters cannot be directly compared. What you could do is compare marginal rates of substitution such as WTP measures between the two models. You could calculate these using apollo_deltaMethod for each of the models, and then you can do a simple ttest for the difference between the two samples (MRS_model1MRS_model2)/(s.e.MRS_model_1+s.e.MRS_model_2).
Stephane
Re: Test for difference, split sample MNL, Poe test
Dear Stephane,
thank you very much for your answer and the clarification! That helps! I will look into apollo_deltaMethod, try to apply it and get back at you with my expericences or (hopefully no) questions!
Thank you!
Best wishes
Sutsch
thank you very much for your answer and the clarification! That helps! I will look into apollo_deltaMethod, try to apply it and get back at you with my expericences or (hopefully no) questions!
Thank you!
Best wishes
Sutsch
Re: Test for difference, split sample MNL, Poe test
Hi Stephane, hi all,
I am wondering why you propose a ttest here. I have seen many people using the test proposed by Poe et al. 2005, when ratios of coefficients are to be tested. Can you maybe briefly explain what the advantages and disadvantages of the tests are, and if there is any easy way to do the Poe test in Apollo. I know how it works, and could write the code for it, but if there are shortcuts/functions, I would like to save time reinventing the wheel .
Thanks a lot and kind regards,
ChoiceOne
Poe, G.L., Giraud, K.L. and Loomis, J.B. (2005), Computational Methods for Measuring the Difference of Empirical Distributions. American Journal of Agricultural Economics, 87: 353365. https://doi.org/10.1111/j.14678276.2005.00727.x
I am wondering why you propose a ttest here. I have seen many people using the test proposed by Poe et al. 2005, when ratios of coefficients are to be tested. Can you maybe briefly explain what the advantages and disadvantages of the tests are, and if there is any easy way to do the Poe test in Apollo. I know how it works, and could write the code for it, but if there are shortcuts/functions, I would like to save time reinventing the wheel .
Thanks a lot and kind regards,
ChoiceOne
Poe, G.L., Giraud, K.L. and Loomis, J.B. (2005), Computational Methods for Measuring the Difference of Empirical Distributions. American Journal of Agricultural Economics, 87: 353365. https://doi.org/10.1111/j.14678276.2005.00727.x

 Site Admin
 Posts: 558
 Joined: 24 Apr 2020, 16:29
Re: Test for difference, split sample MNL, Poe test
Hi
the use of ttest is pretty standard for what we're talking about here. Maybe the Poe test is used in env econ, but I don't really see the need for it. You could compare the findings with the two
Stephane
the use of ttest is pretty standard for what we're talking about here. Maybe the Poe test is used in env econ, but I don't really see the need for it. You could compare the findings with the two
Stephane