Important: Read this before posting to this forum

  1. This forum is for questions related to the use of Apollo. We will answer some general choice modelling questions too, where appropriate, and time permitting. We cannot answer questions about how to estimate choice models with other software packages.
  2. There is a very detailed manual for Apollo available at http://www.ApolloChoiceModelling.com/manual.html. This contains detailed descriptions of the various Apollo functions, and numerous examples are available at http://www.ApolloChoiceModelling.com/examples.html. In addition, help files are available for all functions, using e.g. ?apollo_mnl
  3. Before asking a question on the forum, users are kindly requested to follow these steps:
    1. Check that the same issue has not already been addressed in the forum - there is a search tool.
    2. Ensure that the correct syntax has been used. For any function, detailed instructions are available directly in Apollo, e.g. by using ?apollo_mnl for apollo_mnl
    3. Check the frequently asked questions section on the Apollo website, which discusses some common issues/failures. Please see http://www.apollochoicemodelling.com/faq.html
    4. Make sure that R is using the latest official release of Apollo.
  4. If the above steps do not resolve the issue, then users should follow these steps when posting a question:
    1. provide full details on the issue, including the entire code and output, including any error messages
    2. posts will not immediately appear on the forum, but will be checked by a moderator first. We check the forum at least twice a week. It may thus take a couple of days for your post to appear and before we reply. There is no need to submit the post multiple times.

WTP-space correlation between normals and log-normals

Ask questions about model specifications. Ideally include a mathematical explanation of your proposed model.
Post Reply
DamienJ
Posts: 14
Joined: 08 Nov 2022, 08:57

WTP-space correlation between normals and log-normals

Post by DamienJ »

We are trying to estimate a model in the WTA space. (price is a subsidy). To do that, we are making the hypothesis that the price (subsidy) paramater is distributed as a log-normal, and that the other parameters are normally distributed and correlated (including with the price parameter) and can also be shifted by some socio-economic variables.

Is this an acceptable formulation?

If yest, given this setting, I am not sure whether the usual formula (square root of sum of squares) for calculating the sd of the parameters still apply. In particular, I am not sure about the correlation between a log-normal and a normal distribution?

Here is some code:

Code: Select all

rm(list = ls())
apollo_initialise()

### Set core controls
apollo_control = list(
  modelName       = "MXL_corr_ln_control",
  modelDescr      = "Mixed logit model, correlated, controls ",
  indivID         = "interview__key",  
  mixing          = TRUE,
  nCores          = 6,
  outputDirectory = "output/Correlated"
)

apollo_beta = c(
mu_asc = -3, sigma_asc = 0,               
mu_b_subs = -3.4,
      sigma_s2 = 4,
# Cholesky matrix to represent correlation
# shifts of means by socio-eco variables
mu_w_legu = 22,
    sigma_ls = 0, sigma_l2 = -16.954438, 
      gamma_legu_female = -6,
      gamma_legu_W9 =   -5, 
      gamma_legu_W28 =   0, 
      gamma_legu_risk =   3, 
      gamma_legu_age_med = 0,
      gamma_legu_age_young = -10,
      gamma_legu_ha_person =  10,

mu_w_fodd    =  1,        
    sigma_fs = 0, sigma_fl = 0.064647,  sigma_f2 =  +0.4, 
      gamma_fodd_female = 0.7,
      gamma_fodd_W9     = 0  , 
      gamma_fodd_W28    = 0.2, 
      gamma_fodd_risk   = 0.03,
      gamma_fodd_age_med = 0.2,
      gamma_fodd_age_young = -0.1,
      gamma_fodd_ha_person = -0.3, 
        


#apollo_fixed = c()

### Set parameters for generating draws
apollo_draws = list(
  interDrawsType = "sobolOwenFaureTezuka",
  interNDraws    = 1500,
  interUnifDraws = c(),
  interNormDraws = c("draws_asc", "draws_legu","draws_fodd", "draws_subs"),
  intraDrawsType = "halton",
  intraNDraws    = 0,
  intraUnifDraws = c(),
  intraNormDraws = c())

### Create random parameters
apollo_randCoeff = function(apollo_beta, apollo_inputs){
  randcoeff = list()
  randcoeff[["asc_SQ"]] =   mu_asc + sigma_asc * draws_asc
  randcoeff[["b_subs"]] =   exp(mu_b_subs + sigma_s2 * draws_subs)
  randcoeff[["wta_legu"]] = mu_w_legu + sigma_ls * draws_subs + sigma_l2 * draws_legu + 
  gamma_legu_female*GENDER + gamma_legu_W9*W9 + 
    gamma_legu_W28*W28 + gamma_legu_risk*RISK_SCALE + 
    gamma_legu_age_med*age_med + gamma_legu_age_young* age_young +
    gamma_legu_ha_person*ha_person
  
  randcoeff[["wta_fodd"]] = mu_b_fodd + sigma_fs * draws_subs + sigma_fl * draws_legu + sigma_f2* draws_fodd  +
    gamma_fodd_female*GENDER + gamma_fodd_W9*W9 + 
    gamma_fodd_W28*W28 + gamma_fodd_risk*RISK_SCALE + 
    gamma_fodd_age_med*age_med + gamma_fodd_age_young* age_young +
    gamma_fodd_ha_person*ha_person
  
  return(randcoeff)
}


Then for post estimation of correlations and sd:

Code: Select all

ses <- apollo_deltaMethod(model, deltaMethod_settings = list(expression=c(
  sd.subs = "sigma_s2",
  sd.legu= "sqrt(sigma_ls^2 + sigma_l2^2)",
  sd.fodd="sqrt(sigma_fl^2 + sigma_f2^2)",

#create the cholesky matrix
row1 <- c(model$estimate["sigma_s2"], 0, 0, 0,0)
row2 <- c(model$estimate["sigma_ls"], model$estimate["sigma_l2"], 0, 0,0)
row3 <- c(model$estimate["sigma_fs"], model$estimate["sigma_fl"], model$estimate["sigma_f2"], 0,0)

cholesky <- as.matrix(rbind(row1, row2, row3))
 
# compute covariance matrix
cov_matrix <- cholesky %*% t(cholesky)

# compute the correlation matrix
corr_matrix <- cov2cor(cov_matrix)
corr_matrix
stephanehess
Site Admin
Posts: 1355
Joined: 24 Apr 2020, 16:29

Re: WTP-space correlation between normals and log-normals

Post by stephanehess »

Hi

regarding whether Normals are appropriate, that depends on your attribute. Does it make sense that the WTP can be positive as well as negative.

Regarding the correlation, I suggest you use apollo_unconditionals and then look at the correlations there

Stephane
--------------------------------
Stephane Hess
www.stephanehess.me.uk
DamienJ
Posts: 14
Joined: 08 Nov 2022, 08:57

Re: WTP-space correlation between normals and log-normals

Post by DamienJ »

Thank you for you message and the precisions.

Actually, we wanted the price attribute to be positive (price is a subsidy), so it makes senses to use log-normal.

I ran a model with the previously given code, and I get the following results (see below):

This "looks" satisfactory, but I have two doubts:

1. Should I be concerned about the diagnosis "relative function convergence"?
2. In the WTP space, the price coefficient is lognormal (constrained to be positive). The models finds some significant coefficients, but given these coefficients (mu_b_subs -3.512082, sigma_s2 3.542534), the estimated price coefficient has a very wide range
While 83% of the unconditionals are below 1, there is an extremely long tail
> sum(uncond$b_subs < 1) / sum(uncond$b_subs>0)
[1] 0.839256
> sum(uncond$b_subs > 10) / sum(uncond$b_subs>0)
[1] 0.050356
Should I be concered by long tail for the price coefficient and try-out another distribution that allows me to reduce/control this long tail?


Damien




Model run by djourdain using Apollo 0.3.2 on R 4.2.1 for Windows.
Please acknowledge the use of Apollo by citing Hess & Palma (2019)
DOI 10.1016/j.jocm.2019.100170
www.ApolloChoiceModelling.com

Model name : MXL_corr_ln_control
Model description : Mixed logit model, correlated, subs uncorrelated, controls
Model run at : 2024-06-04 12:22:16
Estimation method : bgw
Model diagnosis : Relative function convergence
Optimisation diagnosis : Maximum found
hessian properties : Negative definite
maximum eigenvalue : -0.022813
reciprocal of condition number : 2.4425e-07
Number of individuals : 250
Number of rows in database : 1500
Number of modelled outcomes : 1500

Number of cores used : 6
Number of inter-individual draws : 2000 (sobolOwenFaureTezuka)

LL(start) : -1253.31
LL at equal shares, LL(0) : -1647.92
LL at observed shares, LL(C) : -1264.45
LL(final) : -1141.38
Rho-squared vs equal shares : 0.3074
Adj.Rho-squared vs equal shares : 0.277
Rho-squared vs observed shares : 0.0973
Adj.Rho-squared vs observed shares : 0.0594
AIC : 2382.76
BIC : 2648.42

Estimated parameters : 50
Time taken (hh:mm:ss) : 01:32:25.43
pre-estimation : 00:09:25.81
estimation : 00:40:59.43
initial estimation : 00:37:54.08
estimation after rescaling : 00:03:5.34
post-estimation : 00:42:0.2
Iterations : 327
initial estimation : 304
estimation after rescaling : 23

Unconstrained optimisation.

Estimates:
Estimate s.e. t.rat.(0) Rob.s.e. Rob.t.rat.(0)
mu_asc -3.108025 0.36989 -8.402642 0.393467 -7.89908
sigma_asc -1.802611 0.30330 -5.943364 0.353287 -5.10240
mu_b_subs -3.512082 0.49726 -7.062885 0.503527 -6.97496
sigma_s2 3.542534 0.67649 5.236652 0.675944 5.24087
mu_b_legu 34.881930 5.71941 6.098873 3.017363 11.56040
sigma_ls -21.493736 2.56178 -8.390151 1.475397 -14.56811
sigma_l2 -16.285837 1.26366 -12.887795 0.616964 -26.39675
gamma_legu_female 4.397324 2.34248 1.877210 0.866837 5.07284
gamma_legu_W9 -11.853573 3.30103 -3.590874 1.487321 -7.96975
gamma_legu_W28 0.013034 2.67318 0.004876 0.974261 0.01338
gamma_legu_risk 2.410866 0.26707 9.027229 0.089046 27.07453
gamma_legu_age_med 4.732648 2.15079 2.200420 0.732129 6.46423
gamma_legu_age_young -6.651651 2.07008 -3.213230 0.681690 -9.75759
gamma_legu_ha_person 19.550295 3.35955 5.819320 1.158536 16.87501
mu_b_fodd -0.096244 0.29707 -0.323974 0.196226 -0.49047
sigma_fs -0.036170 0.12139 -0.297967 0.088171 -0.41022
sigma_fl 0.011754 0.05625 0.208968 0.019560 0.60094
sigma_f2 -0.097031 0.03923 -2.473565 0.019450 -4.98873
gamma_fodd_female 1.075279 0.11762 9.141596 0.050342 21.35949
gamma_fodd_W9 -0.007126 0.11007 -0.064738 0.058462 -0.12188
gamma_fodd_W28 0.838688 0.18273 4.589791 0.073499 11.41087
gamma_fodd_risk -0.063766 0.01669 -3.820473 0.008429 -7.56522
gamma_fodd_age_med 0.800927 0.11606 6.901111 0.056466 14.18421
gamma_fodd_age_young -0.372089 0.16640 -2.236093 0.067742 -5.49273
gamma_fodd_ha_person -0.244728 0.18440 -1.327137 0.127105 -1.92540
mu_b_work -6.888498 1.67908 -4.102551 1.298577 -5.30465
sigma_ws -2.488604 0.75845 -3.281156 0.648289 -3.83873
sigma_wl -0.149911 0.33147 -0.452259 0.077177 -1.94243
sigma_wf 3.927623 0.36342 10.807329 0.119920 32.75194
sigma_w2 -1.776150 0.35873 -4.951179 0.088811 -19.99920
gamma_work_female -4.148609 0.66643 -6.225161 0.294343 -14.09446
gamma_work_W9 9.247183 1.04132 8.880230 0.376109 24.58646
gamma_work_W28 5.942517 0.88046 6.749346 0.339104 17.52416
gamma_work_risk 0.266559 0.10289 2.590779 0.050191 5.31086
gamma_work_age_med -1.349855 0.86988 -1.551772 0.461142 -2.92720
gamma_work_age_young -2.289686 0.73721 -3.105887 0.364862 -6.27549
gamma_work_ha_person 0.910730 0.41518 2.193560 0.157363 5.78744
mu_b_engr -2.170425 1.00019 -2.170004 0.488964 -4.43883
sigma_es 1.340217 0.36673 3.654540 0.233346 5.74348
sigma_el 0.479358 0.20328 2.358145 0.071328 6.72044
sigma_ef -0.585718 0.18351 -3.191775 0.067773 -8.64233
sigma_ew 0.293942 0.17423 1.687044 0.047881 6.13899
sigma_e2 -0.158969 0.16616 -0.956723 0.038664 -4.11154
gamma_engr_female 1.530725 0.35811 4.274436 0.107149 14.28591
gamma_engr_W9 -0.866254 0.50612 -1.711542 0.216846 -3.99479
gamma_engr_W28 -4.133029 0.45400 -9.103528 0.146046 -28.29941
gamma_engr_risk 0.089474 0.06506 1.375152 0.024163 3.70291
gamma_engr_age_med 0.805257 0.56646 1.421561 0.251167 3.20606
gamma_engr_age_young 3.366678 0.50420 6.677313 0.205911 16.35013
gamma_engr_ha_person -0.506666 0.29329 -1.727540 0.117749 -4.30295
stephanehess
Site Admin
Posts: 1355
Joined: 24 Apr 2020, 16:29

Re: WTP-space correlation between normals and log-normals

Post by stephanehess »

Hi

Positive lognormal then makes sense. But does the Normal make sense for the others, i.e. can they be positive and negative?

Relative function convergence is normal convergence with BGW

Long tails are common with lognormals. You could try a log-uniform (exponential of a uniform)

Stephane
--------------------------------
Stephane Hess
www.stephanehess.me.uk
DamienJ
Posts: 14
Joined: 08 Nov 2022, 08:57

Re: WTP-space correlation between normals and log-normals

Post by DamienJ »

Thank you for the advices. We will try the log-uniform;

Damien
Post Reply