Important: Read this before posting to this forum

  1. This forum is for questions related to the use of Apollo. We will answer some general choice modelling questions too, where appropriate, and time permitting. We cannot answer questions about how to estimate choice models with other software packages.
  2. There is a very detailed manual for Apollo available at http://www.ApolloChoiceModelling.com/manual.html. This contains detailed descriptions of the various Apollo functions, and numerous examples are available at http://www.ApolloChoiceModelling.com/examples.html.
  3. Before asking a question on the forum, users are kindly requested to follow these steps:
    1. Check that the same issue has not already been addressed in the forum - there is a search tool.
    2. Ensure that the correct syntax has been used. For any function, detailed instructions are available directly in Apollo, e.g. by using ?apollo_mnl for apollo_mnl
    3. Check the frequently asked questions section on the Apollo website, which discusses some common issues/failures. Please see http://www.apollochoicemodelling.com/faq.html
    4. Make sure that R is using the latest official release of Apollo.
  4. If the above steps do not resolve the issue, then users should follow these steps when posting a question:
    1. provide full details on the issue, including the entire code and output, including any error messages
    2. posts will not immediately appear on the forum, but will be checked by a moderator first. This may take a day or two at busy times. There is no need to submit the post multiple times.

Predictions and marginal effects

Ask questions about post-estimation functions (e.g. prediction, conditionals, etc) or other processing of results.
Post Reply
Patrick_K
Posts: 6
Joined: 14 May 2022, 14:20

Predictions and marginal effects

Post by Patrick_K » 14 May 2022, 14:44

Hi there,

I am new to apollo and have two questions:
1. Is there a way of reporting a p-value for the model itself? I just find the p-values for my parameters but not for the model itself. I am estimating a MNL model.

2. I have one price attribute and two dummy coded other attributes; my experiment is unlabeled with a fixed status quo. I took a look at the example for predictions in the manual but couldn´t figure out how the following can be estimated:

Marginal effects in comparison to status quo or other attribute levels:
2.1 How much higher/lower is the probability of Attribute2_Level_1 (dummy coded) in comparison to Attribute2_Level_Status Quo (dummy coded; reference level)?
2.2 How much higher/lower is the probability of Attribute2_Level_1 (dummy coded) in comparison to Attribute2_Level_2 (dummy coded)?
Marginal efffects of price attribute:
2.3 How much higher/lower is the probability of price attribute (linear coded) if price attribute rises by 0.1% in comparison to status quo?

Are there any specific tipps that you could give on how to model that in predictions or would you recommend other functions?
Also any specific examples are highly appreciated.

Thank you,
Patrick

stephanehess
Site Admin
Posts: 672
Joined: 24 Apr 2020, 16:29

Re: Predictions and marginal effects

Post by stephanehess » 16 May 2022, 07:40

Hi
Is there a way of reporting a p-value for the model itself? I just find the p-values for my parameters but not for the model itself. I am estimating a MNL model
It's not clear what you mean here. A p-value is the outcome of a statistical test. So what would the test be for your model. A comparison against another model, a comparison against a random model, etc?
I have one price attribute and two dummy coded other attributes; my experiment is unlabeled with a fixed status quo.
I think what you want to do here is to make two predictions each time, one for a base case and one for a case where you change the data, and compare the two. See for example http://apollochoicemodelling.com/files/examples/1 MNL/MNL_SP_covariates.r

Stephane
--------------------------------
Stephane Hess
www.stephanehess.me.uk

Patrick_K
Posts: 6
Joined: 14 May 2022, 14:20

Re: Predictions and marginal effects

Post by Patrick_K » 16 May 2022, 17:41

Hi Stephane,

thank you very much for your prompt reply.
I will try to do the predictions with the recommended example.

With regards to the p-value I was looking for an indicator for the correlation of the individual decisions among themselves, so how much of the variance can be explained by the random effect. Also for a p-value for a comparison against a random model.
I was also wondering why there is Rho² reported instead of pseudo R²? I could not find anything about it in the manual.

Kind regards,
Patrick

Patrick_K
Posts: 6
Joined: 14 May 2022, 14:20

Re: Predictions and marginal effects

Post by Patrick_K » 21 May 2022, 16:12

When working with the predictions function I had some questions:

I.
I tried to compute predictions for a 10% price change in comparison to fixed status quo price level.

I used the following code (once with Price_SQ, once with Price_Alt1, once with Price_Alt2):
_____
predictions_base = apollo_prediction(model, apollo_probabilities, apollo_inputs)

### Prediction for 10% higher price in Alt1
database$Price_Alt1 = 1.10*database$Price_Alt1

### Rerun predictions with the new data
apollo_inputs = apollo_validateInputs()
predictions_new = apollo_prediction(model, apollo_probabilities, apollo_inputs)

### Return to original data
database$Price_Alt1 = 1/1.10*database$Price_Alt1
apollo_inputs = apollo_validateInputs()

### work with predictions at estimates
predictions_base=predictions_base[["at_estimates"]]

### Compute change in probabilities
change=(predictions_new-predictions_base)/predictions_base
____

Please see below the results:
For Price_SQ:
status quo Alt1 Alt2
Min. :-0.07174 Min. :0.003361 Min. :0.003361
1st Qu.:-0.07010 1st Qu.:0.005132 1st Qu.:0.005132
Median :-0.06840 Median :0.006969 Median :0.006969
Mean :-0.06745 Mean :0.007996 Mean :0.007996
3rd Qu.:-0.06555 3rd Qu.:0.010053 3rd Qu.:0.010053
Max. :-0.05813 Max. :0.018071 Max. :0.018071
For Price_Alt1:
status quo Alt1 Alt2
Min. :0.01436 Min. :-0.19678 Min. :0.01436
1st Qu.:0.02865 1st Qu.:-0.16037 1st Qu.:0.02865
Median :0.04620 Median :-0.11720 Median :0.04620
Mean :0.05065 Mean :-0.10322 Mean :0.05065
3rd Qu.:0.06924 3rd Qu.:-0.04147 3rd Qu.:0.06924
Max. :0.09354 Max. :-0.01165 Max. :0.09354
For Price_Alt2:
status quo Alt1 Alt2
Min. :0.01676 Min. :0.01676 Min. :-0.19488
1st Qu.:0.04657 1st Qu.:0.04657 1st Qu.:-0.10858
Median :0.05879 Median :0.05879 Median :-0.08658
Mean :0.07465 Mean :0.07465 Mean :-0.08372
3rd Qu.:0.07806 3rd Qu.:0.07806 3rd Qu.:-0.03238
Max. :0.17682 Max. :0.17682 Max. :-0.01228
________

1. As my experiment is unlabelled, is there a possibility to compute general predictions telling: "If the price is increased by 10% a choice scenario is x% less probable to be chosen." Do I therefore have to combine the different results from above?
2. If I take the results in bold it can be interpreted that in case of a price increase of 10% in Alt2, Alt2 is 8,372% less probable to be chosen and Alt1 and status quo are 7,465% more probable to be chosen?
_______________________________________________________
II. For dummy variables I did not find any example in your manual or examples but I´ve tried the following:

### Prediction for a change from level 2 of dummy variable1 to level 1 of dummy variable1
database$DumVar1_Alt1==2 = database$DumVar1_Alt1==1

### Rerun predictions with the new data
apollo_inputs = apollo_validateInputs()
predictions_new = apollo_prediction(model, apollo_probabilities, apollo_inputs)

### Return to original data
database$DumVar1_Alt1==2 = database$DumVar1_Alt1==2
apollo_inputs = apollo_validateInputs()

### work with predictions at estimates
predictions_base=predictions_base[["at_estimates"]]

### Compute change in probabilities
change=(predictions_new-predictions_base)/predictions_base

Results only showing NAs:

status quo Alt1 Alt2
Min. : NA Min. : NA Min. : NA
1st Qu.: NA 1st Qu.: NA 1st Qu.: NA
Median : NA Median : NA Median : NA
Mean :NaN Mean :NaN Mean :NaN
3rd Qu.: NA 3rd Qu.: NA 3rd Qu.: NA
Max. : NA Max. : NA Max. : NA
____
1. Are there any examples which I maybe did not find or could you briefly tell what I did wrong? Also wondering (as above) how to compute: "If there is a change from level 2 of dummy variable1 to level 1 of dummy variable1 a choice scenario is x% less/more probable to be chosen."
2. Could you maybe tell how the code for elasticities needs to be adjusted when using for unlabelled experiment and dummy coded variables?
3. In my MNL model DumVar1==4 is the reference level, is it therefore even possible to compute the probabilities from a change from DumVar1==4 to DumVar1==1 as I do not have a parameter for DumVar1==4.

Any hints are highly appreciated.
Thank you very much in advance!
Best wishes,
Patrick

stephanehess
Site Admin
Posts: 672
Joined: 24 Apr 2020, 16:29

Re: Predictions and marginal effects

Post by stephanehess » 09 Aug 2022, 16:39

Patrick

Regarding your first question, what you could maybe consider doing is taking the average across the three cases to get an overall picture of the impact of a price increase. You can still calculate the elasticities as log((P_after_inc/P_before_inc)/1.1)

Regarding the second question, what I would do is to make predictions at all the possible levels of the dummy coded variable, so one where the level is always 1, one where it is always 2, etc and then compare them to each other. The code

Code: Select all

database$DumVar1_Alt1==2 = database$DumVar1_Alt1==1
doesn't really work that way.

Stephane
--------------------------------
Stephane Hess
www.stephanehess.me.uk

stephanehess
Site Admin
Posts: 672
Joined: 24 Apr 2020, 16:29

Re: Predictions and marginal effects

Post by stephanehess » 09 Aug 2022, 16:42

Hi,

in relation to this earlier question:

1. You could compute a likelihood ratio test against the purely random model (LL0) and then calculate the p-value for that.

2. Please see https://doi.org/10.1016/j.jocm.2016.02.001
Patrick_K wrote:
16 May 2022, 17:41
Hi Stephane,

thank you very much for your prompt reply.
I will try to do the predictions with the recommended example.

With regards to the p-value I was looking for an indicator for the correlation of the individual decisions among themselves, so how much of the variance can be explained by the random effect. Also for a p-value for a comparison against a random model.
I was also wondering why there is Rho² reported instead of pseudo R²? I could not find anything about it in the manual.

Kind regards,
Patrick
--------------------------------
Stephane Hess
www.stephanehess.me.uk

Post Reply