Important: Read this before posting to this forum

  1. This forum is for questions related to the use of Apollo. We will answer some general choice modelling questions too, where appropriate, and time permitting. We cannot answer questions about how to estimate choice models with other software packages.
  2. There is a very detailed manual for Apollo available at http://www.ApolloChoiceModelling.com/manual.html. This contains detailed descriptions of the various Apollo functions, and numerous examples are available at http://www.ApolloChoiceModelling.com/examples.html. In addition, help files are available for all functions, using e.g. ?apollo_mnl
  3. Before asking a question on the forum, users are kindly requested to follow these steps:
    1. Check that the same issue has not already been addressed in the forum - there is a search tool.
    2. Ensure that the correct syntax has been used. For any function, detailed instructions are available directly in Apollo, e.g. by using ?apollo_mnl for apollo_mnl
    3. Check the frequently asked questions section on the Apollo website, which discusses some common issues/failures. Please see http://www.apollochoicemodelling.com/faq.html
    4. Make sure that R is using the latest official release of Apollo.
  4. If the above steps do not resolve the issue, then users should follow these steps when posting a question:
    1. provide full details on the issue, including the entire code and output, including any error messages
    2. posts will not immediately appear on the forum, but will be checked by a moderator first. This may take a day or two at busy times. There is no need to submit the post multiple times.

interpretation of cost estimate derived by the Delta method

Ask questions about the results reported after estimation. If the output includes errors, please include your model code if possible.
Locked
maa033
Posts: 35
Joined: 23 Jul 2020, 14:00

interpretation of cost estimate derived by the Delta method

Post by maa033 »

Hi

Having estimated a MMNL model in preference Space, With non-cost parameters specified as normal and the cost parameter to be log-normal, I use the delta Method to derive estimates for the cost attribute. Like this:

>
> deltaMethod_settings <- list(operation="lognormal", parName1="cost_T_mu", parName2="cost_T_sig")
> apollo_deltaMethod(model, deltaMethod_settings)

Running Delta method computations
Value Robust s.e. Rob t-ratio (0)
Mean for exp(N( cost_T_mu , cost_T_sig ) 91.77 60.04 1.5285
SD for exp(N( cost_T_mu , cost_T_sig ) 4746.52 6551.25 0.7245

I now Wonder how to interpret the output of this delta Method command?
What is the Mean value of 91.77? Do I have to take the ln of this value to get a parameter value that can be compared to the normally Distributed non-cost parameter estimates? How does this value relate to the normally Distributed non-cost parameter estimates?

The estimated value for this log-normally Distributed parameter is:

cost_T_mu 0.57328 0.50500 1.1352 0.56978 1.0061
cost_T_sig -2.80928 0.31262 -8.9863 0.29491 -9.5258
stephanehess
Site Admin
Posts: 974
Joined: 24 Apr 2020, 16:29

Re: interpretation of cost estimate derived by the Delta method

Post by stephanehess »

Hi

your original parameter estimates cost_T_mu and cost_T_sig are the means and standard deviations for the logarithm of the cost coefficient. That logarithm is normally distributed. What the delta method function gives you here are the means and standard deviations for the actual lognormal coefficient. But you clearly have some issues here with very large standard errors

Stephane
--------------------------------
Stephane Hess
www.stephanehess.me.uk
maa033
Posts: 35
Joined: 23 Jul 2020, 14:00

Re: interpretation of cost estimate derived by the Delta method

Post by maa033 »

Thanks for clarifying the interpretations.

Not only standard error of the cost parameter is large, so is also the standard deviation.
Estimating the model I use 1000 sobolOwenFaureTezuka draws. This type of draws was recommended due to the large number of random parameters to be estimated (20). I also tried MLHS draws, but gave a far lower fit.

I also estimate the model in WTP space, as I am more interested in the WTP units. Most (all) WTPs come out as significant, but this may then be due to the high std.error and std.deviation of the cost parameter? Does this mean the estimated WTP parameters are biased or unreliable?
stephanehess
Site Admin
Posts: 974
Joined: 24 Apr 2020, 16:29

Re: interpretation of cost estimate derived by the Delta method

Post by stephanehess »

The fact that MLHS gives lower fit than solo does not mean anything. You do not know what is the correct model fit, and both are approximations.

But estimating a model with 20 random parameters is something that I would only ever recommend if you have huge amounts of very good data
--------------------------------
Stephane Hess
www.stephanehess.me.uk
maa033
Posts: 35
Joined: 23 Jul 2020, 14:00

Re: interpretation of cost estimate derived by the Delta method

Post by maa033 »

Ok.

Here is the results from the model:

Main model WTP-space baseline vs treatment

Model run at : 2020-11-19 22:37:00
Estimation method : bfgs
Model diagnosis : successful convergence
Number of individuals : 293
Number of observations : 2599
Number of cores used : 3
Number of inter-individual draws : 1000 (sobolOwenFaureTezuka)
LL(start) : -1700.566
LL(0) : -2855.293
LL(final) : -1594.237
Rho-square (0) : 0.4417
Adj.Rho-square (0) : 0.434
AIC : 3232.47
BIC : 3361.46
Estimated parameters : 22
Time taken (hh:mm:ss) : 00:51:54.45
Iterations : 143
Min abs eigenvalue of Hessian : 7406.791
Some eigenvalues of Hessian are positive, indicating potential
roblems!

Estimates:
Estimate s.e. t.rat.(0) Rob.s.e. Rob.t.rat.(0)
ascB -5.256855 0.592955 -8.8655 0.599993 -8.7615
ascT -3.410379 0.349505 -9.7578 0.457567 -7.4533
cost_B_mu -0.128596 0.164458 -0.7819 0.138520 -0.9284
cost_B_sig 2.445333 0.249065 9.8181 0.183206 13.3475
torsk_B_mu 0.224047 0.008341 26.8613 0.003170 70.6685
torsk_B_sig 0.443072 0.013636 32.4925 0.005729 77.3358
laks_B_mu 0.004356 NaN NaN 0.003395 1.2829
laks_B_sig -0.644243 0.024355 -26.4517 0.010136 -63.5599
bunn_B_mu 0.490220 0.042642 11.4963 0.020226 24.2376
bunn_B_sig 0.080375 0.018651 4.3094 0.006471 12.4215
land_B_mu -0.018104 8.3887e-04 -21.5816 6.7579e-04 -26.7896
land_B_sig 0.025363 0.001739 14.5857 5.2770e-04 48.0636
cost_T_mu -0.153816 0.246945 -0.6229 0.264393 -0.5818
cost_T_sig -2.449008 0.343699 -7.1254 0.295616 -8.2844
torsk_T_mu 0.089525 0.021731 4.1197 0.011830 7.5679
torsk_T_sig -0.590551 0.038342 -15.4022 0.021270 -27.7640
laks_T_mu 0.148830 0.025861 5.7549 0.013072 11.3857
laks_T_sig -0.032052 0.010519 -3.0470 0.005503 -5.8241
bunn_T_mu 0.353340 0.107609 3.2836 0.069526 5.0821
bunn_T_sig 0.080316 0.059426 1.3515 0.015645 5.1338
land_T_mu 0.004931 0.005298 0.9307 0.002590 1.9034
land_T_sig -0.006983 0.004916 -1.4205 0.002651 -2.6341

And the transformed cost-parameters:
Running Delta method computations
Value Robust s.e. Rob t-ratio (0)
Mean for exp(N( cost_B_mu , cost_B_sig ) 17.48 7.422 2.355
SD for exp(N( cost_B_mu , cost_B_sig ) 347.16 299.573 1.159

Value Robust s.e. Rob t-ratio (0)
Mean for exp(N( cost_T_mu , cost_T_sig ) 17.2 12.14 1.4174
SD for exp(N( cost_T_mu , cost_T_sig ) 344.7 484.84 0.7109


I am testing whether attribute coefficients in the baseline differ from those in the treatment. I could run the model for the baseline and rteatment samples separately, but my idea was that running the model on all observations simultaneosuly would be more efficient. So what I do is to compare torsk_B_mu With torsk_T_mu (and so on for 4 non-cost attributes). I run the model in WTP-Space to avoid issues of different scale across baseline and treatment. The problem is the cost-attribute. First, it is not significant in the treatment. Second, it has a very large standard deviation. Most other coefficients are significant, but my question is whether I can trust results from this model given a non-significant cost-parameter for the treatment cost?
stephanehess
Site Admin
Posts: 974
Joined: 24 Apr 2020, 16:29

Re: interpretation of cost estimate derived by the Delta method

Post by stephanehess »

Hi

as I say in my earlier reply "But estimating a model with 20 random parameters is something that I would only ever recommend if you have huge amounts of very good data"

I think you are just trying to squeeze too much out of your data. You only have 293 people in your data and you're trying to estimate a huge number of random parameters.

Stephane
--------------------------------
Stephane Hess
www.stephanehess.me.uk
maa033
Posts: 35
Joined: 23 Jul 2020, 14:00

Re: interpretation of cost estimate derived by the Delta method

Post by maa033 »

OK. Point taken!

So, With this dataset (150 persons and 1347 observations in baseline and 143 persons and 1252 observations in treatment), is it possible to run a model With only random attribute parameters (alltogehter 5 attributes)? If yes, is it better to run the model separateley for baseline and treatment observations? If no, how many random parameters is it advicable to have in a model run on this dataset?

Sorry for these quite basic questions, but I have not been sufficiently aware of these restrictions when analysing the data. I have run MNL models and MMNL models on the two datasets separately, but for my purpose, which is to compare the non-cost attribute coefficients across baseline and treatment, it has been more convenient to use the model presented above. I would really appreciate some advice regarding the type of Statistical model appropriate to set up for this data set and given the purpose of the estimation.

Thank you in advance.
stephanehess
Site Admin
Posts: 974
Joined: 24 Apr 2020, 16:29

Re: interpretation of cost estimate derived by the Delta method

Post by stephanehess »

Hi

there is no straightforward answer to this question. The number of random parameters that you can estimate will really depend on the richness and quality of your data alongside the amount of data you have. Running separate models for baseline and treatment will not solve your problem as you will be halving the data.

What I would suggest is that you first study the differences between baseline and treatment in a simpler model, as well as determining for which parameters random heterogeneity exists and matters

Stephane
--------------------------------
Stephane Hess
www.stephanehess.me.uk
Locked