Important: Read this before posting to this forum

  1. This forum is for questions related to the use of Apollo. We will answer some general choice modelling questions too, where appropriate, and time permitting. We cannot answer questions about how to estimate choice models with other software packages.
  2. There is a very detailed manual for Apollo available at http://www.ApolloChoiceModelling.com/manual.html. This contains detailed descriptions of the various Apollo functions, and numerous examples are available at http://www.ApolloChoiceModelling.com/examples.html.
  3. Before asking a question on the forum, users are kindly requested to follow these steps:
    1. Check that the same issue has not already been addressed in the forum - there is a search tool.
    2. Ensure that the correct syntax has been used. For any function, detailed instructions are available directly in Apollo, e.g. by using ?apollo_mnl for apollo_mnl
    3. Check the frequently asked questions section on the Apollo website, which discusses some common issues/failures. Please see http://www.apollochoicemodelling.com/faq.html
    4. Make sure that R is using the latest official release of Apollo.
  4. If the above steps do not resolve the issue, then users should follow these steps when posting a question:
    1. provide full details on the issue, including the entire code and output, including any error messages
    2. posts will not immediately appear on the forum, but will be checked by a moderator first. This may take a day or two at busy times. There is no need to submit the post multiple times.

Test for difference, split sample MNL, Poe test

Ask questions about post-estimation functions (e.g. prediction, conditionals, etc) or other processing of results.
Post Reply
Sutsch
Posts: 8
Joined: 13 Sep 2021, 12:51

Test for difference, split sample MNL, Poe test

Post by Sutsch » 15 Sep 2021, 20:10

Hi everyone,

I am rather new to R and Apollo, however, based on your examples I managed to got an MNL estimation for my data set running. I did a split sample approach and for each sample I estimated the parameters. Now I wanted to see, if my estimations for the attributes differ significantly between the two samples. For the analysis I wanted to use the Poe test (R Package "mded"), which measures the difference between two independent or non-independent empirical distributions and should return a significance level of the difference.

However, from my understanding, for this test I need to have the individual estimations for the respondents for each attribute. And currently I have really no idea, how to get these from my MNL estimations. Is there any chance that you could help me out on this and give me a hint, how to get the attribute estimations for each individual? Or is that not necessary?

I am really looking for you answer! Thank you very much in advance!

Best
Sutsch

stephanehess
Site Admin
Posts: 558
Joined: 24 Apr 2020, 16:29

Re: Test for difference, split sample MNL, Poe test

Post by stephanehess » 17 Sep 2021, 15:05

Sutsch

let me first clear up a confusion here. There is no such thing as individual-specific estimates with a choice model, unless you estimated a separate model for each person. What you maybe refer to here are Bayesian posteriors for a Mixed Logit model, but they are still driven by a sample level model.

Either way, none of these applies in an MNL context, where your parameters are fixed across the sample within a model.

In your case, you are interested in comparing the estimates from two models on non-overlapping samples. The problem is that in that case, there are potential scale differences between the two models (differential role of the variance of the extreme value error terms) and as a result, the parameters cannot be directly compared. What you could do is compare marginal rates of substitution such as WTP measures between the two models. You could calculate these using apollo_deltaMethod for each of the models, and then you can do a simple t-test for the difference between the two samples (MRS_model1-MRS_model2)/(s.e.MRS_model_1+s.e.MRS_model_2).

Stephane
--------------------------------
Stephane Hess
www.stephanehess.me.uk

Sutsch
Posts: 8
Joined: 13 Sep 2021, 12:51

Re: Test for difference, split sample MNL, Poe test

Post by Sutsch » 17 Sep 2021, 16:40

Dear Stephane,

thank you very much for your answer and the clarification! That helps! I will look into apollo_deltaMethod, try to apply it and get back at you with my expericences or (hopefully no) questions!

Thank you!

Best wishes
Sutsch

ChoiceOne
Posts: 1
Joined: 22 Sep 2021, 11:39

Re: Test for difference, split sample MNL, Poe test

Post by ChoiceOne » 22 Sep 2021, 11:46

Hi Stephane, hi all,

I am wondering why you propose a t-test here. I have seen many people using the test proposed by Poe et al. 2005, when ratios of coefficients are to be tested. Can you maybe briefly explain what the advantages and disadvantages of the tests are, and if there is any easy way to do the Poe test in Apollo. I know how it works, and could write the code for it, but if there are shortcuts/functions, I would like to save time reinventing the wheel :) .

Thanks a lot and kind regards,

ChoiceOne

Poe, G.L., Giraud, K.L. and Loomis, J.B. (2005), Computational Methods for Measuring the Difference of Empirical Distributions. American Journal of Agricultural Economics, 87: 353-365. https://doi.org/10.1111/j.1467-8276.2005.00727.x

stephanehess
Site Admin
Posts: 558
Joined: 24 Apr 2020, 16:29

Re: Test for difference, split sample MNL, Poe test

Post by stephanehess » 22 Sep 2021, 15:49

Hi

the use of t-test is pretty standard for what we're talking about here. Maybe the Poe test is used in env econ, but I don't really see the need for it. You could compare the findings with the two

Stephane
--------------------------------
Stephane Hess
www.stephanehess.me.uk

Post Reply