Important: Read this before posting to this forum

  1. This forum is for questions related to the use of Apollo. We will answer some general choice modelling questions too, where appropriate, and time permitting. We cannot answer questions about how to estimate choice models with other software packages.
  2. There is a very detailed manual for Apollo available at http://www.ApolloChoiceModelling.com/manual.html. This contains detailed descriptions of the various Apollo functions, and numerous examples are available at http://www.ApolloChoiceModelling.com/examples.html. In addition, help files are available for all functions, using e.g. ?apollo_mnl
  3. Before asking a question on the forum, users are kindly requested to follow these steps:
    1. Check that the same issue has not already been addressed in the forum - there is a search tool.
    2. Ensure that the correct syntax has been used. For any function, detailed instructions are available directly in Apollo, e.g. by using ?apollo_mnl for apollo_mnl
    3. Check the frequently asked questions section on the Apollo website, which discusses some common issues/failures. Please see http://www.apollochoicemodelling.com/faq.html
    4. Make sure that R is using the latest official release of Apollo.
  4. If the above steps do not resolve the issue, then users should follow these steps when posting a question:
    1. provide full details on the issue, including the entire code and output, including any error messages
    2. posts will not immediately appear on the forum, but will be checked by a moderator first. This may take a day or two at busy times. There is no need to submit the post multiple times.

estimation in WTP-space

Ask questions about errors you encouunter. Please make sure to include full details about your model specifications, and ideally your model file.
maa033
Posts: 35
Joined: 23 Jul 2020, 14:00

Re: estimation in WTP-space

Post by maa033 »

Hi again
I started the seachStart procedure about 20 hours ago, and, having reached the 2nd step, it is still running! However, it has come up With 100 potential candidates for starting values, and has singled out the optimal one (lowest LL-value). So, would it be OK to stop the procedure now, and use the optimal set of starting values to run the estimation model? Interrupting the command, I Guess I have to insert these starting values into the script manually?
dpalma
Posts: 190
Joined: 24 Apr 2020, 17:54

Re: estimation in WTP-space

Post by dpalma »

Hi,

Yes, you can stop the process and manually input the best candidate so far as the starting values for you model.

It could be that if you let the process run for longer, another candidate may surpasses the one picked as best in the first step. So it is a trade-off between estimation time and the chance of finding the global optima.

Best
David
maa033
Posts: 35
Joined: 23 Jul 2020, 14:00

Re: estimation in WTP-space

Post by maa033 »

Hi again

I am sorry to coming back to this estimation in WTP-Space :-(
Having tried X-number of times, I now keep getting this Message when trying to estimate the model in WTP-Space.

final value 1673.816891
stopped after 200 iterations

Estimated parameters:
[,1]
ascB -3.3220
ascT -4.0208
cost_B_mu 0.1577
cost_B_sig -2.1317
torsk_B_mu 0.0853
torsk_B_sig -0.6740
laks_B_mu 0.2147
laks_B_sig 0.4641
bunn_B_mu 0.9370
bunn_B_sig 0.0557
land_B_mu -0.0156
land_B_sig -0.0167
cost_T_mu -10.1026
cost_T_sig -5.2677
torsk_T_mu -1.1440
torsk_T_sig 25.7946
laks_T_mu 2.1085
laks_T_sig -24.7002
bunn_T_mu -9.2221
bunn_T_sig 21.1261
land_T_mu -0.1026
land_T_sig -13.0907

ERROR: Estimation failed. No covariance matrix to compute.


I used the model specification from above (earlier Message in this post), and have used the searchStart for better starting values. I even have reinstalled R and RStudio, incase there were any issues With the program.

Do you have any suggestion what I could do to make the model run in WTP-Space?
maa033
Posts: 35
Joined: 23 Jul 2020, 14:00

Re: estimation in WTP-space

Post by maa033 »

Hi again
I have sorted out the problem With no covariance Matrix. It was all in the starting-values. I played around With starting values based on output from similar models, which turned out to work.
As the output for the models in WTP-Space seems to be very sensitive to starting-values, is there a code (in Apollo) which initiates a loop that automatically updates starting-values, i.e. when one round of estimation is finished it automatically applies the output estimates as starting-values in a New round of estimation, and so on? Is there a code for this? Or do we need to feed in updated staring-values manually, and start the estimation anew manually?
dpalma
Posts: 190
Joined: 24 Apr 2020, 17:54

Re: estimation in WTP-space

Post by dpalma »

Hi,

You can load parameter estimates of older models as starting values using the apollo_readBeta function. You can type ?apollo_readBeta for details, or look for it in the manual.

Using apollo_searchStart you can provide a range from which random starting values will be drawn. Any that won't work will be discarded automatically.

However (at least for now) there is no automatic method that, given a complex model, estimates a simpler one and uses the estimated parameters of the simpler model as the starting values of the more complex model. It is, however, a common and useful way to find good starting values.

Best
David
maa033
Posts: 35
Joined: 23 Jul 2020, 14:00

Re: estimation in WTP-space

Post by maa033 »

Hi, and thank you for all you assistance on this matter, estimation in WTP-Space, so far.
I have a question regarding the interpretation of the output from the delta-Method code to derive mean coefficients for the log-normally Distributed cost parameter.

Value Robust s.e. Rob t-ratio (0)
Mean for exp(N( cost_B_mu , cost_B_sig ) 12.7987 21.8128 0.59
SD for exp(N( cost_B_mu , cost_B_sig ) 213.7246 762.5900 0.28

Value Robust s.e. Rob t-ratio (0)
Mean for exp(N( cost_T_mu , cost_T_sig ) 18.7284 12.3904 1.51
SD for exp(N( cost_T_mu , cost_T_sig ) 353.2016 457.2003 0.77

Is this output the correct to use when we assess the significance of the cost-parameter coefficient? Which means that the cost-parameter is not significant. Or do we have to take the ln of the mean coefficent and s.e.?
stephanehess
Site Admin
Posts: 974
Joined: 24 Apr 2020, 16:29

Re: estimation in WTP-space

Post by stephanehess »

Hi

could you please share the outputs for the model that this was produced for, including the covariance matrix

Thanks

Stephane
--------------------------------
Stephane Hess
www.stephanehess.me.uk
maa033
Posts: 35
Joined: 23 Jul 2020, 14:00

Re: estimation in WTP-space

Post by maa033 »

Hi

I don't have Access to the previous model just now, but provide you With the results from another mix MNL model With lognormal cost parameter, use of deltaMethod to check for significance of estimated cost parameter.

This is all output from that model, including the vcoc. As you can see, a very significant cost parameter according to the p-value, but strange outputs for estimate and std.error (only 0).

Model run using Apollo for R, version 0.1.0
www.ApolloChoiceModelling.com

Model name : MMNL_data_gruve
Model description : MMNL model with fixed cost attr
Model run at : 2020-09-08 14:23:26
Estimation method : bfgs
Model diagnosis : successful convergence
Number of individuals : 1123
Number of observations : 8546

Number of cores used : 3
Number of inter-person draws : 100 (halton)

LL(start) : -102324
LL(0) : -9388.741
LL(final) : -7268.239
Rho-square (0) : 0.2259
Adj.Rho-square (0) : 0.2249
AIC : 14554.48
BIC : 14617.96
Estimated parameters : 9
Time taken (hh:mm:ss) : 00:06:54.14
Iterations : 64
Min abs eigenvalue of hessian : 0.562111
Some eigenvalues of hessian are positive, indicating potential problems!

Estimates:
Estimate Std.err. t.ratio(0) Rob.std.err. Rob.t.ratio(0)
asc -2.2083 0.0862 -25.60 0.0946 -23.36
seabed_mu 0.1034 0.0083 12.49 0.0117 8.81
seabed_sig 0.1648 0.0098 16.76 0.0163 10.13
salmon_mu -0.0104 0.0092 -1.12 0.0095 -1.10
salmon_sig 0.1539 0.0144 10.69 0.0218 7.07
jobs_mu 0.0075 0.0018 4.21 0.0019 3.89
jobs_sig 0.0219 0.0033 6.67 0.0044 5.03
cost_mu -12.1086 0.0080 -1511.39 0.0108 -1122.73
cost_sig 0.5983 NaN NaN 0.0061 98.75

Overview of choices for model component "MNL"
alt1 alt2 alt3
Times available 8546.00 8546.00 8546.00
Times chosen 2537.00 3169.00 2840.00
Percentage chosen overall 29.69 37.08 33.23
Percentage chosen when available 29.69 37.08 33.23

> apollo_saveOutput(model)
Estimates saved to MMNL_data_gruve_estimates.csv
Classical covariance matrix saved to MMNL_data_gruve_covar.csv
Robust covariance matrix saved to MMNL_data_gruve_robcovar.csv
Classical correlation matrix saved to MMNL_data_gruve_covar.csv
Robust correlation matrix saved to MMNL_data_gruve_robcorr.csv
Model object saved to MMNL_data_gruve.rds
>
> # ################################################################# #
> ##### POST-PROCESSING ####
> # ################################################################# #
>
> ### Print outputs of additional diagnostics to new output file (remember to close file writing when complete)
> sink(paste(model$apollo_control$modelName,"_additional_output.txt",sep=""),split=TRUE)
> deltaMethod_settings <- list(operation="lognormal", parName1="cost_mu", parName2="cost_sig")
> apollo_deltaMethod(model, deltaMethod_settings)

Running Delta method computations
Value Robust s.e. Rob t-ratio (0)
Mean for exp(N( cost_mu , cost_sig ) 0 0 108.41
SD for exp(N( cost_mu , cost_sig ) 0 0 76.67

> vcov(model)
asc seabed_mu seabed_sig salmon_mu
asc 7.438793e-03 -2.157429e-04 -2.001317e-04 -1.926618e-04
seabed_mu -2.157429e-04 6.854278e-05 5.289119e-05 -2.314408e-05
seabed_sig -2.001317e-04 5.289119e-05 9.667019e-05 -3.488203e-05
salmon_mu -1.926618e-04 -2.314408e-05 -3.488203e-05 8.538430e-05
salmon_sig -2.797337e-05 -5.517701e-05 -8.196384e-05 6.516198e-05
jobs_mu -4.120852e-05 -3.767929e-06 -4.629812e-06 -7.073551e-07
jobs_sig -1.826342e-05 -9.308441e-06 -1.497270e-05 3.115459e-06
cost_mu 2.569514e-04 -6.477486e-05 -4.826137e-05 2.385143e-05
cost_sig 5.947195e-05 1.307637e-05 1.960251e-05 -2.408104e-06
salmon_sig jobs_mu jobs_sig cost_mu
asc -2.797337e-05 -4.120852e-05 -1.826342e-05 2.569514e-04
seabed_mu -5.517701e-05 -3.767929e-06 -9.308441e-06 -6.477486e-05
seabed_sig -8.196384e-05 -4.629812e-06 -1.497270e-05 -4.826137e-05
salmon_mu 6.516198e-05 -7.073551e-07 3.115459e-06 2.385143e-05
salmon_sig 2.072066e-04 3.320100e-06 1.014155e-05 5.185691e-05
jobs_mu 3.320100e-06 3.178285e-06 2.853481e-06 5.896439e-07
jobs_sig 1.014155e-05 2.853481e-06 1.082481e-05 6.454960e-06
cost_mu 5.185691e-05 5.896439e-07 6.454960e-06 6.418521e-05
cost_sig -1.346165e-05 -6.031766e-06 -1.367829e-05 1.099505e+00
cost_sig
asc 5.947195e-05
seabed_mu 1.307637e-05
seabed_sig 1.960251e-05
salmon_mu -2.408104e-06
salmon_sig -1.346165e-05
jobs_mu -6.031766e-06
jobs_sig -1.367829e-05
cost_mu 1.099505e+00
cost_sig -1.099492e+00
>
stephanehess
Site Admin
Posts: 974
Joined: 24 Apr 2020, 16:29

Re: estimation in WTP-space

Post by stephanehess »

Hi

this is entirely normal. Your estimates clearly show that the log of the cost coefficient has a mean that is very negative. So the actual cost coefficient is near zero, and not significantly different from zero. The significance test for cost_mu is against zero. It shows that this parameter is very much different from zero, but that doesn't mean that the cost coefficient itself is different from zero, quite the opposite in fact in this case.

Does this make sense to you?

Code: Select all

Estimate Std.err. t.ratio(0) Rob.std.err. Rob.t.ratio(0)
cost_mu -12.1086 0.0080 -1511.39 0.0108 -1122.73
cost_sig 0.5983 NaN NaN 0.0061 98.75
--------------------------------
Stephane Hess
www.stephanehess.me.uk
maa033
Posts: 35
Joined: 23 Jul 2020, 14:00

Re: estimation in WTP-space

Post by maa033 »

This makes sense.
Thank you.
Post Reply