Important: Read this before posting to this forum
- This forum is for questions related to the use of Apollo. We will answer some general choice modelling questions too, where appropriate, and time permitting. We cannot answer questions about how to estimate choice models with other software packages.
- There is a very detailed manual for Apollo available at http://www.ApolloChoiceModelling.com/manual.html. This contains detailed descriptions of the various Apollo functions, and numerous examples are available at http://www.ApolloChoiceModelling.com/examples.html. In addition, help files are available for all functions, using e.g. ?apollo_mnl
- Before asking a question on the forum, users are kindly requested to follow these steps:
- Check that the same issue has not already been addressed in the forum - there is a search tool.
- Ensure that the correct syntax has been used. For any function, detailed instructions are available directly in Apollo, e.g. by using ?apollo_mnl for apollo_mnl
- Check the frequently asked questions section on the Apollo website, which discusses some common issues/failures. Please see http://www.apollochoicemodelling.com/faq.html
- Make sure that R is using the latest official release of Apollo.
- Users can check which version they are running by entering packageVersion("apollo").
- Then check what is the latest full release (not development version) at http://www.ApolloChoiceModelling.com/code.html.
- To update to the latest official version, just enter install.packages("apollo"). To update to a development version, download the appropriate binary file from http://www.ApolloChoiceModelling.com/code.html, and install the package from file
- If the above steps do not resolve the issue, then users should follow these steps when posting a question:
- provide full details on the issue, including the entire code and output, including any error messages
- posts will not immediately appear on the forum, but will be checked by a moderator first. This may take a day or two at busy times. There is no need to submit the post multiple times.
estimation in WTP-space
estimation in WTP-space
I have a problem With NANs and NAs for std.error and rob.std.error in the output when I run the model in WTP-Space. This typically appears when I work With random cost parameter and specify the distribution of the cost-parameter to be negative lognormal:
randcoeff[["cost_B"]] = -exp(cost_B_mu + cost_B_sig * draws_cost)
If I use a normal distribution for the cost parameter, like the one below, the results are fine With no NANs or NAs.
randcoeff[["cost_T"]] = cost_B_mu + cost_B_sig * draws_cost
Is this a problem in my data or a misspecification of the distribution?
randcoeff[["cost_B"]] = -exp(cost_B_mu + cost_B_sig * draws_cost)
If I use a normal distribution for the cost parameter, like the one below, the results are fine With no NANs or NAs.
randcoeff[["cost_T"]] = cost_B_mu + cost_B_sig * draws_cost
Is this a problem in my data or a misspecification of the distribution?
-
- Site Admin
- Posts: 1042
- Joined: 24 Apr 2020, 16:29
Re: estimation in WTP-space
Hi
there could be many reasons for this, and it's difficult to diagnose without seeing your outputs, and also how they compare to the model with Normals. A Normal distribution for cost of course makes no sense, so you're right to try negative Lognormals. One thing to consider is whether your starting values for the negative Lognormal are appropriate. I would advise starting cost_B_mu at something like -3 or even -5
Best wishes
Stephane
there could be many reasons for this, and it's difficult to diagnose without seeing your outputs, and also how they compare to the model with Normals. A Normal distribution for cost of course makes no sense, so you're right to try negative Lognormals. One thing to consider is whether your starting values for the negative Lognormal are appropriate. I would advise starting cost_B_mu at something like -3 or even -5
Best wishes
Stephane
Re: estimation in WTP-space
Thanks. It worked With higher starring value for the mean (-3). However, I kept the starting value for the std.dev at 0, and this resulted in a very large and significant std.deviation, but insignificant mean cost estimate. Should I use a higher (absolute) starting value for the std.dev. of the cost parameter as well?
-
- Site Admin
- Posts: 1042
- Joined: 24 Apr 2020, 16:29
Re: estimation in WTP-space
Hi
again, it's not possible for us to really help without seeing output. Also, when you say an insignificant mean, do you mean the estimate for cost_B_mu - this is not the estimate of the mean of cost_B, but the estimate of the mean of log(-cost_B). The fact that cost_B_mu is not significantly different from zero does not imply that the mean of cost_B is not significantly different from zero. See Table 2 in http://www.stephanehess.me.uk/papers/jo ... B_2012.pdf
Stephane
again, it's not possible for us to really help without seeing output. Also, when you say an insignificant mean, do you mean the estimate for cost_B_mu - this is not the estimate of the mean of cost_B, but the estimate of the mean of log(-cost_B). The fact that cost_B_mu is not significantly different from zero does not imply that the mean of cost_B is not significantly different from zero. See Table 2 in http://www.stephanehess.me.uk/papers/jo ... B_2012.pdf
Stephane
Re: estimation in WTP-space
This is the output when the cost parameter is specified as negative lognormal Distributed With starting value -3. What I do is estimating a merged dataset, consisting of data collected in two different situations (baseline and treatment) to test for possible differences in stated preferences across the situations. Thus two estimated parameters for each attribute. I am aware that I need to correct for differences in scale across the two situations. I will look into the paper you sent and find out how to transform the estimates for cost_B_mu and cost_T_mu into mean values for the cost attribute parameters.
Model name : MMNL_sim
Model description : MMNL model for simultaneous estimation of coeff
Model run at : 2020-08-18 12:25:07
Estimation method : bfgs
Model diagnosis : successful convergence
Number of individuals : 293
Number of observations : 2599
Number of cores used : 3
Number of inter-person draws : 100 (halton)
LL(start) : -2855.802
LL(0) : -2855.293
LL(final) : -1466.954
Rho-square (0) : 0.4862
Adj.Rho-square (0) : 0.4785
AIC : 2977.91
BIC : 3106.89
Estimated parameters : 22
Time taken (hh:mm:ss) : 00:14:3.27
Iterations : 93
Min abs eigenvalue of hessian : 1.196175
Estimates:
Estimate Std.err. t.ratio(0) Rob.std.err. Rob.t.ratio(0)
asc -4.8354 0.7656 -6.32 1.0061 -4.81
cost_B_mu -0.3268 0.2470 -1.32 0.2765 -1.18
cost_B_sig 3.1010 0.2563 12.10 0.2760 11.24
torsk_B_mu -0.2918 0.0411 -7.11 0.0421 -6.93
torsk_B_sig 0.2589 0.0427 6.07 0.0484 5.35
laks_B_mu -0.2584 0.0570 -4.54 0.0555 -4.66
laks_B_sig 0.3049 0.0542 5.62 0.0586 5.21
bunn_B_mu -0.4219 0.1863 -2.26 0.1639 -2.57
bunn_B_sig -0.1682 0.2753 -0.61 0.2282 -0.74
land_B_mu -0.0213 0.0104 -2.05 0.0096 -2.22
land_B_sig 0.0463 0.0150 3.09 0.0149 3.10
cost_T_mu 0.4443 0.3106 1.43 0.3417 1.30
cost_T_sig 2.4355 0.1621 15.03 0.1412 17.25
torsk_T_mu -0.4622 0.1483 -3.12 0.1850 -2.50
torsk_T_sig 0.4676 0.1244 3.76 0.1536 3.04
laks_T_mu -0.5971 0.2136 -2.79 0.2548 -2.34
laks_T_sig -0.7057 0.1955 -3.61 0.2487 -2.84
bunn_T_mu -0.7102 0.4612 -1.54 0.4438 -1.60
bunn_T_sig -1.5096 0.4573 -3.30 0.5148 -2.93
land_T_mu -0.0742 0.0310 -2.39 0.0327 -2.27
land_T_sig -0.1331 0.0396 -3.36 0.0504 -2.64
scale_base 1.0000 NA NA NA NA
scale_treat 0.6339 0.1681 3.77 0.2115 3.00
Overview of choices for model component "MNL"
alt1 alt2 alt3
Times available 2599.00 2599.00 2599.00
Times chosen 429.00 1056.00 1114.00
Percentage chosen overall 16.51 40.63 42.86
Percentage chosen when available 16.51 40.63 42.86
Model name : MMNL_sim
Model description : MMNL model for simultaneous estimation of coeff
Model run at : 2020-08-18 12:25:07
Estimation method : bfgs
Model diagnosis : successful convergence
Number of individuals : 293
Number of observations : 2599
Number of cores used : 3
Number of inter-person draws : 100 (halton)
LL(start) : -2855.802
LL(0) : -2855.293
LL(final) : -1466.954
Rho-square (0) : 0.4862
Adj.Rho-square (0) : 0.4785
AIC : 2977.91
BIC : 3106.89
Estimated parameters : 22
Time taken (hh:mm:ss) : 00:14:3.27
Iterations : 93
Min abs eigenvalue of hessian : 1.196175
Estimates:
Estimate Std.err. t.ratio(0) Rob.std.err. Rob.t.ratio(0)
asc -4.8354 0.7656 -6.32 1.0061 -4.81
cost_B_mu -0.3268 0.2470 -1.32 0.2765 -1.18
cost_B_sig 3.1010 0.2563 12.10 0.2760 11.24
torsk_B_mu -0.2918 0.0411 -7.11 0.0421 -6.93
torsk_B_sig 0.2589 0.0427 6.07 0.0484 5.35
laks_B_mu -0.2584 0.0570 -4.54 0.0555 -4.66
laks_B_sig 0.3049 0.0542 5.62 0.0586 5.21
bunn_B_mu -0.4219 0.1863 -2.26 0.1639 -2.57
bunn_B_sig -0.1682 0.2753 -0.61 0.2282 -0.74
land_B_mu -0.0213 0.0104 -2.05 0.0096 -2.22
land_B_sig 0.0463 0.0150 3.09 0.0149 3.10
cost_T_mu 0.4443 0.3106 1.43 0.3417 1.30
cost_T_sig 2.4355 0.1621 15.03 0.1412 17.25
torsk_T_mu -0.4622 0.1483 -3.12 0.1850 -2.50
torsk_T_sig 0.4676 0.1244 3.76 0.1536 3.04
laks_T_mu -0.5971 0.2136 -2.79 0.2548 -2.34
laks_T_sig -0.7057 0.1955 -3.61 0.2487 -2.84
bunn_T_mu -0.7102 0.4612 -1.54 0.4438 -1.60
bunn_T_sig -1.5096 0.4573 -3.30 0.5148 -2.93
land_T_mu -0.0742 0.0310 -2.39 0.0327 -2.27
land_T_sig -0.1331 0.0396 -3.36 0.0504 -2.64
scale_base 1.0000 NA NA NA NA
scale_treat 0.6339 0.1681 3.77 0.2115 3.00
Overview of choices for model component "MNL"
alt1 alt2 alt3
Times available 2599.00 2599.00 2599.00
Times chosen 429.00 1056.00 1114.00
Percentage chosen overall 16.51 40.63 42.86
Percentage chosen when available 16.51 40.63 42.86
Re: estimation in WTP-space
Hi,
You can obtain the mean and the s.d. of your lognormal parameters using the following code snippet:
This will also show the standard error of these values, so you will be able to tell if they are significant or not.
Remember that -as you mentioned- the base and treatment parameters still have a different scale, so they cannot be compared directly if you are working in preference space.
Best
David
You can obtain the mean and the s.d. of your lognormal parameters using the following code snippet:
Code: Select all
deltaMethod_settings <- list(operation="lognormal", parName1="cost_B_mu", parName2="cost_B_sig")
apollo_deltaMethod(model, deltaMethod_settings)
deltaMethod_settings <- list(operation="lognormal", parName1="cost_T_mu", parName2="cost_T_sig")
apollo_deltaMethod(model, deltaMethod_settings)
Remember that -as you mentioned- the base and treatment parameters still have a different scale, so they cannot be compared directly if you are working in preference space.
Best
David
Re: estimation in WTP-space
Thanks a lot for this! Worked fine.
I tried to run the same model in WTP-Space, but got very strange results:
Model name : MMNL_sim
Model description : MMNL model for simultaneous estimation of coeff
Model run at : 2020-08-19 11:35:52
Estimation method : bfgs
Model diagnosis : successful convergence
Number of individuals : 293
Number of observations : 2599
Number of cores used : 3
Number of inter-person draws : 100 (halton)
LL(start) : -2855.802
LL(0) : -2855.293
LL(final) : -1848.092
Rho-square (0) : 0.3527
Adj.Rho-square (0) : 0.3454
AIC : 3738.18
BIC : 3861.3
Estimated parameters : 21
Time taken (hh:mm:ss) : 00:14:58.33
Iterations : 170
Min abs eigenvalue of hessian : 0.012457
Estimates:
Estimate Std.err. t.ratio(0) Rob.std.err. Rob.t.ratio(0)
asc -3.7459 0.2032 -18.43 0.2171 -17.26
cost_B_mu -13.1867 1.4974 -8.81 2.1216 -6.22
cost_B_sig -7.9258 1.3837 -5.73 2.0412 -3.88
torsk_B_mu -0.2325 3.4799 -0.07 6.0949 -0.04
torsk_B_sig -10.0244 3.1396 -3.19 5.5856 -1.79
laks_B_mu -8.5628 NaN NaN 0.8000 -10.70
laks_B_sig 9.6588 3.0225 3.20 5.6484 1.71
bunn_B_mu -10.1113 3.9581 -2.55 7.7880 -1.30
bunn_B_sig -18.5782 2.8551 -6.51 3.8263 -4.86
land_B_mu -9.0525 NaN NaN 0.5508 -16.43
land_B_sig 6.9587 0.4527 15.37 0.3451 20.16
cost_T_mu -9.8385 0.6132 -16.04 0.3676 -26.77
cost_T_sig 7.9208 0.8280 9.57 0.5399 14.67
torsk_T_mu 4.7083 0.7837 6.01 1.6621 2.83
torsk_T_sig 13.0653 1.5839 8.25 2.8747 4.54
laks_T_mu 4.3214 1.9341 2.23 3.8115 1.13
laks_T_sig 0.4718 0.4190 1.13 0.2942 1.60
bunn_T_mu 0.0409 2.0169 0.02 4.2892 0.01
bunn_T_sig 11.9307 2.3452 5.09 3.1985 3.73
land_T_mu -19.8303 0.6090 -32.56 0.6813 -29.11
land_T_sig -25.0320 0.6740 -37.14 0.4242 -59.01
The model specification I used was as follows, With normal non-cost attributes and lognormal cost:
V = list()
V[['alt1']] = Certain*(asc + cost_B*(torsk_B * KT1 + Cost1 + laks_B * VL1 + bunn_B * HB1 + land_B * KL1))+
(1-Certain)*(asc + cost_T*(torsk_T * KT1 + Cost1 + laks_T * VL1 + bunn_T * HB1 + land_T * KL1))
V[['alt2']] = Certain*(cost_B*(torsk_B * KT2 + Cost2 + laks_B * VL2 + bunn_B * HB2 + land_B * KL2))+
(1-Certain)*(cost_T*(torsk_T * KT2 + Cost2 + laks_T * VL2 + bunn_T * HB2 + land_T * KL2))
V[['alt3']] = Certain*(cost_B*(torsk_B * KT3 + Cost3 + laks_B * VL3 + bunn_B * HB3 + land_B * KL3))+
(1-Certain)*(cost_T*(torsk_T * KT3 + Cost3 + laks_T * VL3 + bunn_T * HB3 + land_T * KL3))
Is this specification correct? If yes, then it might be the very low number of draws (100) that caused the WTP-estimates??
Overview of choices for model component "MNL"
alt1 alt2 alt3
Times available 2599.00 2599.00 2599.00
Times chosen 429.00 1056.00 1114.00
Percentage chosen overall 16.51 40.63 42.86
Percentage chosen when available 16.51 40.63 42.86
I tried to run the same model in WTP-Space, but got very strange results:
Model name : MMNL_sim
Model description : MMNL model for simultaneous estimation of coeff
Model run at : 2020-08-19 11:35:52
Estimation method : bfgs
Model diagnosis : successful convergence
Number of individuals : 293
Number of observations : 2599
Number of cores used : 3
Number of inter-person draws : 100 (halton)
LL(start) : -2855.802
LL(0) : -2855.293
LL(final) : -1848.092
Rho-square (0) : 0.3527
Adj.Rho-square (0) : 0.3454
AIC : 3738.18
BIC : 3861.3
Estimated parameters : 21
Time taken (hh:mm:ss) : 00:14:58.33
Iterations : 170
Min abs eigenvalue of hessian : 0.012457
Estimates:
Estimate Std.err. t.ratio(0) Rob.std.err. Rob.t.ratio(0)
asc -3.7459 0.2032 -18.43 0.2171 -17.26
cost_B_mu -13.1867 1.4974 -8.81 2.1216 -6.22
cost_B_sig -7.9258 1.3837 -5.73 2.0412 -3.88
torsk_B_mu -0.2325 3.4799 -0.07 6.0949 -0.04
torsk_B_sig -10.0244 3.1396 -3.19 5.5856 -1.79
laks_B_mu -8.5628 NaN NaN 0.8000 -10.70
laks_B_sig 9.6588 3.0225 3.20 5.6484 1.71
bunn_B_mu -10.1113 3.9581 -2.55 7.7880 -1.30
bunn_B_sig -18.5782 2.8551 -6.51 3.8263 -4.86
land_B_mu -9.0525 NaN NaN 0.5508 -16.43
land_B_sig 6.9587 0.4527 15.37 0.3451 20.16
cost_T_mu -9.8385 0.6132 -16.04 0.3676 -26.77
cost_T_sig 7.9208 0.8280 9.57 0.5399 14.67
torsk_T_mu 4.7083 0.7837 6.01 1.6621 2.83
torsk_T_sig 13.0653 1.5839 8.25 2.8747 4.54
laks_T_mu 4.3214 1.9341 2.23 3.8115 1.13
laks_T_sig 0.4718 0.4190 1.13 0.2942 1.60
bunn_T_mu 0.0409 2.0169 0.02 4.2892 0.01
bunn_T_sig 11.9307 2.3452 5.09 3.1985 3.73
land_T_mu -19.8303 0.6090 -32.56 0.6813 -29.11
land_T_sig -25.0320 0.6740 -37.14 0.4242 -59.01
The model specification I used was as follows, With normal non-cost attributes and lognormal cost:
V = list()
V[['alt1']] = Certain*(asc + cost_B*(torsk_B * KT1 + Cost1 + laks_B * VL1 + bunn_B * HB1 + land_B * KL1))+
(1-Certain)*(asc + cost_T*(torsk_T * KT1 + Cost1 + laks_T * VL1 + bunn_T * HB1 + land_T * KL1))
V[['alt2']] = Certain*(cost_B*(torsk_B * KT2 + Cost2 + laks_B * VL2 + bunn_B * HB2 + land_B * KL2))+
(1-Certain)*(cost_T*(torsk_T * KT2 + Cost2 + laks_T * VL2 + bunn_T * HB2 + land_T * KL2))
V[['alt3']] = Certain*(cost_B*(torsk_B * KT3 + Cost3 + laks_B * VL3 + bunn_B * HB3 + land_B * KL3))+
(1-Certain)*(cost_T*(torsk_T * KT3 + Cost3 + laks_T * VL3 + bunn_T * HB3 + land_T * KL3))
Is this specification correct? If yes, then it might be the very low number of draws (100) that caused the WTP-estimates??
Overview of choices for model component "MNL"
alt1 alt2 alt3
Times available 2599.00 2599.00 2599.00
Times chosen 429.00 1056.00 1114.00
Percentage chosen overall 16.51 40.63 42.86
Percentage chosen when available 16.51 40.63 42.86
Re: estimation in WTP-space
Hi,
The formulation looks fine, but I would recommend using a different ASC for the base a treatment scenarios. Also, why does only one alternative have an ASC? Are the second and third alternatives unlabelled while the first one is labelled?
Keep in mind that mixed logits potentially have multiple optima, so the estimates you obtain main be dependent on the starting value of your parameters. I would recommend:
David
The formulation looks fine, but I would recommend using a different ASC for the base a treatment scenarios. Also, why does only one alternative have an ASC? Are the second and third alternatives unlabelled while the first one is labelled?
Keep in mind that mixed logits potentially have multiple optima, so the estimates you obtain main be dependent on the starting value of your parameters. I would recommend:
- Increase the number of draws
- Use the apollo_searchStart function. This function will generate random starting values for your parameters and return the parameter values that seem to lead to the best optima. You can see more details by typing ?apollo_searchStart, look at the manual (http://www.apollochoicemodelling.com/files/Apollo.pdf), or look at examples 18 and 19.
David
Re: estimation in WTP-space
This was helpful.
Does it matter where in the script I insert the searchStart command?
I tried to figure out from the manual where to insert it, but could not find out??
Does it matter where in the script I insert the searchStart command?
I tried to figure out from the manual where to insert it, but could not find out??
Re: estimation in WTP-space
You should call apollo_searchStart just before calling apollo_estimate, as in:
Be aware that apollo_searchStart can take a long time to run, and it doesn't guarantee you getting the best solution possible (global optima of the LL), but it can help finding a better solution.
Code: Select all
apollo_beta=apollo_searchStart(apollo_beta, apollo_fixed,apollo_probabilities, apollo_inputs)
model = apollo_estimate(apollo_beta, apollo_fixed, apollo_probabilities, apollo_inputs)