Hi there,
I am new to apollo and have two questions:
1. Is there a way of reporting a pvalue for the model itself? I just find the pvalues for my parameters but not for the model itself. I am estimating a MNL model.
2. I have one price attribute and two dummy coded other attributes; my experiment is unlabeled with a fixed status quo. I took a look at the example for predictions in the manual but couldn´t figure out how the following can be estimated:
Marginal effects in comparison to status quo or other attribute levels:
2.1 How much higher/lower is the probability of Attribute2_Level_1 (dummy coded) in comparison to Attribute2_Level_Status Quo (dummy coded; reference level)?
2.2 How much higher/lower is the probability of Attribute2_Level_1 (dummy coded) in comparison to Attribute2_Level_2 (dummy coded)?
Marginal efffects of price attribute:
2.3 How much higher/lower is the probability of price attribute (linear coded) if price attribute rises by 0.1% in comparison to status quo?
Are there any specific tipps that you could give on how to model that in predictions or would you recommend other functions?
Also any specific examples are highly appreciated.
Thank you,
Patrick
Important: Read this before posting to this forum
 This forum is for questions related to the use of Apollo. We will answer some general choice modelling questions too, where appropriate, and time permitting. We cannot answer questions about how to estimate choice models with other software packages.
 There is a very detailed manual for Apollo available at http://www.ApolloChoiceModelling.com/manual.html. This contains detailed descriptions of the various Apollo functions, and numerous examples are available at http://www.ApolloChoiceModelling.com/examples.html. In addition, help files are available for all functions, using e.g. ?apollo_mnl
 Before asking a question on the forum, users are kindly requested to follow these steps:
 Check that the same issue has not already been addressed in the forum  there is a search tool.
 Ensure that the correct syntax has been used. For any function, detailed instructions are available directly in Apollo, e.g. by using ?apollo_mnl for apollo_mnl
 Check the frequently asked questions section on the Apollo website, which discusses some common issues/failures. Please see http://www.apollochoicemodelling.com/faq.html
 Make sure that R is using the latest official release of Apollo.
 Users can check which version they are running by entering packageVersion("apollo").
 Then check what is the latest full release (not development version) at http://www.ApolloChoiceModelling.com/code.html.
 To update to the latest official version, just enter install.packages("apollo"). To update to a development version, download the appropriate binary file from http://www.ApolloChoiceModelling.com/code.html, and install the package from file
 If the above steps do not resolve the issue, then users should follow these steps when posting a question:
 provide full details on the issue, including the entire code and output, including any error messages
 posts will not immediately appear on the forum, but will be checked by a moderator first. This may take a day or two at busy times. There is no need to submit the post multiple times.
Predictions and marginal effects

 Site Admin
 Posts: 896
 Joined: 24 Apr 2020, 16:29
Re: Predictions and marginal effects
Hi
Stephane
It's not clear what you mean here. A pvalue is the outcome of a statistical test. So what would the test be for your model. A comparison against another model, a comparison against a random model, etc?Is there a way of reporting a pvalue for the model itself? I just find the pvalues for my parameters but not for the model itself. I am estimating a MNL model
I think what you want to do here is to make two predictions each time, one for a base case and one for a case where you change the data, and compare the two. See for example http://apollochoicemodelling.com/files/examples/1 MNL/MNL_SP_covariates.rI have one price attribute and two dummy coded other attributes; my experiment is unlabeled with a fixed status quo.
Stephane
Re: Predictions and marginal effects
Hi Stephane,
thank you very much for your prompt reply.
I will try to do the predictions with the recommended example.
With regards to the pvalue I was looking for an indicator for the correlation of the individual decisions among themselves, so how much of the variance can be explained by the random effect. Also for a pvalue for a comparison against a random model.
I was also wondering why there is Rho² reported instead of pseudo R²? I could not find anything about it in the manual.
Kind regards,
Patrick
thank you very much for your prompt reply.
I will try to do the predictions with the recommended example.
With regards to the pvalue I was looking for an indicator for the correlation of the individual decisions among themselves, so how much of the variance can be explained by the random effect. Also for a pvalue for a comparison against a random model.
I was also wondering why there is Rho² reported instead of pseudo R²? I could not find anything about it in the manual.
Kind regards,
Patrick
Re: Predictions and marginal effects
When working with the predictions function I had some questions:
I.
I tried to compute predictions for a 10% price change in comparison to fixed status quo price level.
I used the following code (once with Price_SQ, once with Price_Alt1, once with Price_Alt2):
_____
predictions_base = apollo_prediction(model, apollo_probabilities, apollo_inputs)
### Prediction for 10% higher price in Alt1
database$Price_Alt1 = 1.10*database$Price_Alt1
### Rerun predictions with the new data
apollo_inputs = apollo_validateInputs()
predictions_new = apollo_prediction(model, apollo_probabilities, apollo_inputs)
### Return to original data
database$Price_Alt1 = 1/1.10*database$Price_Alt1
apollo_inputs = apollo_validateInputs()
### work with predictions at estimates
predictions_base=predictions_base[["at_estimates"]]
### Compute change in probabilities
change=(predictions_newpredictions_base)/predictions_base
____
Please see below the results:
For Price_SQ:
status quo Alt1 Alt2
Min. :0.07174 Min. :0.003361 Min. :0.003361
1st Qu.:0.07010 1st Qu.:0.005132 1st Qu.:0.005132
Median :0.06840 Median :0.006969 Median :0.006969
Mean :0.06745 Mean :0.007996 Mean :0.007996
3rd Qu.:0.06555 3rd Qu.:0.010053 3rd Qu.:0.010053
Max. :0.05813 Max. :0.018071 Max. :0.018071
For Price_Alt1:
status quo Alt1 Alt2
Min. :0.01436 Min. :0.19678 Min. :0.01436
1st Qu.:0.02865 1st Qu.:0.16037 1st Qu.:0.02865
Median :0.04620 Median :0.11720 Median :0.04620
Mean :0.05065 Mean :0.10322 Mean :0.05065
3rd Qu.:0.06924 3rd Qu.:0.04147 3rd Qu.:0.06924
Max. :0.09354 Max. :0.01165 Max. :0.09354
For Price_Alt2:
status quo Alt1 Alt2
Min. :0.01676 Min. :0.01676 Min. :0.19488
1st Qu.:0.04657 1st Qu.:0.04657 1st Qu.:0.10858
Median :0.05879 Median :0.05879 Median :0.08658
Mean :0.07465 Mean :0.07465 Mean :0.08372
3rd Qu.:0.07806 3rd Qu.:0.07806 3rd Qu.:0.03238
Max. :0.17682 Max. :0.17682 Max. :0.01228
________
1. As my experiment is unlabelled, is there a possibility to compute general predictions telling: "If the price is increased by 10% a choice scenario is x% less probable to be chosen." Do I therefore have to combine the different results from above?
2. If I take the results in bold it can be interpreted that in case of a price increase of 10% in Alt2, Alt2 is 8,372% less probable to be chosen and Alt1 and status quo are 7,465% more probable to be chosen?
_______________________________________________________
II. For dummy variables I did not find any example in your manual or examples but I´ve tried the following:
### Prediction for a change from level 2 of dummy variable1 to level 1 of dummy variable1
database$DumVar1_Alt1==2 = database$DumVar1_Alt1==1
### Rerun predictions with the new data
apollo_inputs = apollo_validateInputs()
predictions_new = apollo_prediction(model, apollo_probabilities, apollo_inputs)
### Return to original data
database$DumVar1_Alt1==2 = database$DumVar1_Alt1==2
apollo_inputs = apollo_validateInputs()
### work with predictions at estimates
predictions_base=predictions_base[["at_estimates"]]
### Compute change in probabilities
change=(predictions_newpredictions_base)/predictions_base
Results only showing NAs:
status quo Alt1 Alt2
Min. : NA Min. : NA Min. : NA
1st Qu.: NA 1st Qu.: NA 1st Qu.: NA
Median : NA Median : NA Median : NA
Mean :NaN Mean :NaN Mean :NaN
3rd Qu.: NA 3rd Qu.: NA 3rd Qu.: NA
Max. : NA Max. : NA Max. : NA
____
1. Are there any examples which I maybe did not find or could you briefly tell what I did wrong? Also wondering (as above) how to compute: "If there is a change from level 2 of dummy variable1 to level 1 of dummy variable1 a choice scenario is x% less/more probable to be chosen."
2. Could you maybe tell how the code for elasticities needs to be adjusted when using for unlabelled experiment and dummy coded variables?
3. In my MNL model DumVar1==4 is the reference level, is it therefore even possible to compute the probabilities from a change from DumVar1==4 to DumVar1==1 as I do not have a parameter for DumVar1==4.
Any hints are highly appreciated.
Thank you very much in advance!
Best wishes,
Patrick
I.
I tried to compute predictions for a 10% price change in comparison to fixed status quo price level.
I used the following code (once with Price_SQ, once with Price_Alt1, once with Price_Alt2):
_____
predictions_base = apollo_prediction(model, apollo_probabilities, apollo_inputs)
### Prediction for 10% higher price in Alt1
database$Price_Alt1 = 1.10*database$Price_Alt1
### Rerun predictions with the new data
apollo_inputs = apollo_validateInputs()
predictions_new = apollo_prediction(model, apollo_probabilities, apollo_inputs)
### Return to original data
database$Price_Alt1 = 1/1.10*database$Price_Alt1
apollo_inputs = apollo_validateInputs()
### work with predictions at estimates
predictions_base=predictions_base[["at_estimates"]]
### Compute change in probabilities
change=(predictions_newpredictions_base)/predictions_base
____
Please see below the results:
For Price_SQ:
status quo Alt1 Alt2
Min. :0.07174 Min. :0.003361 Min. :0.003361
1st Qu.:0.07010 1st Qu.:0.005132 1st Qu.:0.005132
Median :0.06840 Median :0.006969 Median :0.006969
Mean :0.06745 Mean :0.007996 Mean :0.007996
3rd Qu.:0.06555 3rd Qu.:0.010053 3rd Qu.:0.010053
Max. :0.05813 Max. :0.018071 Max. :0.018071
For Price_Alt1:
status quo Alt1 Alt2
Min. :0.01436 Min. :0.19678 Min. :0.01436
1st Qu.:0.02865 1st Qu.:0.16037 1st Qu.:0.02865
Median :0.04620 Median :0.11720 Median :0.04620
Mean :0.05065 Mean :0.10322 Mean :0.05065
3rd Qu.:0.06924 3rd Qu.:0.04147 3rd Qu.:0.06924
Max. :0.09354 Max. :0.01165 Max. :0.09354
For Price_Alt2:
status quo Alt1 Alt2
Min. :0.01676 Min. :0.01676 Min. :0.19488
1st Qu.:0.04657 1st Qu.:0.04657 1st Qu.:0.10858
Median :0.05879 Median :0.05879 Median :0.08658
Mean :0.07465 Mean :0.07465 Mean :0.08372
3rd Qu.:0.07806 3rd Qu.:0.07806 3rd Qu.:0.03238
Max. :0.17682 Max. :0.17682 Max. :0.01228
________
1. As my experiment is unlabelled, is there a possibility to compute general predictions telling: "If the price is increased by 10% a choice scenario is x% less probable to be chosen." Do I therefore have to combine the different results from above?
2. If I take the results in bold it can be interpreted that in case of a price increase of 10% in Alt2, Alt2 is 8,372% less probable to be chosen and Alt1 and status quo are 7,465% more probable to be chosen?
_______________________________________________________
II. For dummy variables I did not find any example in your manual or examples but I´ve tried the following:
### Prediction for a change from level 2 of dummy variable1 to level 1 of dummy variable1
database$DumVar1_Alt1==2 = database$DumVar1_Alt1==1
### Rerun predictions with the new data
apollo_inputs = apollo_validateInputs()
predictions_new = apollo_prediction(model, apollo_probabilities, apollo_inputs)
### Return to original data
database$DumVar1_Alt1==2 = database$DumVar1_Alt1==2
apollo_inputs = apollo_validateInputs()
### work with predictions at estimates
predictions_base=predictions_base[["at_estimates"]]
### Compute change in probabilities
change=(predictions_newpredictions_base)/predictions_base
Results only showing NAs:
status quo Alt1 Alt2
Min. : NA Min. : NA Min. : NA
1st Qu.: NA 1st Qu.: NA 1st Qu.: NA
Median : NA Median : NA Median : NA
Mean :NaN Mean :NaN Mean :NaN
3rd Qu.: NA 3rd Qu.: NA 3rd Qu.: NA
Max. : NA Max. : NA Max. : NA
____
1. Are there any examples which I maybe did not find or could you briefly tell what I did wrong? Also wondering (as above) how to compute: "If there is a change from level 2 of dummy variable1 to level 1 of dummy variable1 a choice scenario is x% less/more probable to be chosen."
2. Could you maybe tell how the code for elasticities needs to be adjusted when using for unlabelled experiment and dummy coded variables?
3. In my MNL model DumVar1==4 is the reference level, is it therefore even possible to compute the probabilities from a change from DumVar1==4 to DumVar1==1 as I do not have a parameter for DumVar1==4.
Any hints are highly appreciated.
Thank you very much in advance!
Best wishes,
Patrick

 Site Admin
 Posts: 896
 Joined: 24 Apr 2020, 16:29
Re: Predictions and marginal effects
Patrick
Regarding your first question, what you could maybe consider doing is taking the average across the three cases to get an overall picture of the impact of a price increase. You can still calculate the elasticities as log((P_after_inc/P_before_inc)/1.1)
Regarding the second question, what I would do is to make predictions at all the possible levels of the dummy coded variable, so one where the level is always 1, one where it is always 2, etc and then compare them to each other. The code
doesn't really work that way.
Stephane
Regarding your first question, what you could maybe consider doing is taking the average across the three cases to get an overall picture of the impact of a price increase. You can still calculate the elasticities as log((P_after_inc/P_before_inc)/1.1)
Regarding the second question, what I would do is to make predictions at all the possible levels of the dummy coded variable, so one where the level is always 1, one where it is always 2, etc and then compare them to each other. The code
Code: Select all
database$DumVar1_Alt1==2 = database$DumVar1_Alt1==1
Stephane

 Site Admin
 Posts: 896
 Joined: 24 Apr 2020, 16:29
Re: Predictions and marginal effects
Hi,
in relation to this earlier question:
1. You could compute a likelihood ratio test against the purely random model (LL0) and then calculate the pvalue for that.
2. Please see https://doi.org/10.1016/j.jocm.2016.02.001
in relation to this earlier question:
1. You could compute a likelihood ratio test against the purely random model (LL0) and then calculate the pvalue for that.
2. Please see https://doi.org/10.1016/j.jocm.2016.02.001
Patrick_K wrote: ↑16 May 2022, 17:41 Hi Stephane,
thank you very much for your prompt reply.
I will try to do the predictions with the recommended example.
With regards to the pvalue I was looking for an indicator for the correlation of the individual decisions among themselves, so how much of the variance can be explained by the random effect. Also for a pvalue for a comparison against a random model.
I was also wondering why there is Rho² reported instead of pseudo R²? I could not find anything about it in the manual.
Kind regards,
Patrick
Re: Predictions and marginal effects
What's the code for estimating marginal effects in apollo after MNL or MMNL?

 Site Admin
 Posts: 896
 Joined: 24 Apr 2020, 16:29
Re: Predictions and marginal effects
Hi
there is no "code" for this. What you would need to do is to run multiple predictions, changing one attribute at a time
Stephane
there is no "code" for this. What you would need to do is to run multiple predictions, changing one attribute at a time
Stephane