Between subject design

[dom]

Hi

First of all thanks for this great package and the very detailed user manual. This manual helped me a lot, but I still have some questions that I now ask here.

Here is what I want to do: I have choice data with one observation per individual from an experiment with two conditions (each subject is in only one of the two conditions). The choice was between 12 alternatives with different attributes (the payoff of the alternative as well as the norm and guilt associated with choosing this alternative). I want to test whether our manipulation lead to significantly different preference weights between the two conditions. Thus were it not for the two conditions, I would have a conditional logit model. But now I want to interact the choice specific attributes with the (individual specific) condition variable.

I have accomplished to run a model with only the interaction, but not the condition main effect. However I read that this should not be done and thus I now try to include the condition main effect. I tried to include the condition main effect by adopting example 3. That is I let the alternative specific constants interact with the condition variable (coded as 1 and 0). Thus the difference between the alternative specific constant with and without the condition interaction would be the group main effect. Do I see this correct? This is how I specified the model:

Code: Select all

 # Utility functions
  V[['v0']]<- asc_value_0 + asc_0_shift_private * condition + payoff * possible_payoff_0 + norm * personal_norm_0 + guilt * guilt_0 + condition_guilt * condition * guilt_0 + condition_norms * condition * personal_norm_0
  V[['v1']]<- asc_value_1 + asc_1_shift_private * condition + payoff * possible_payoff_1 + norm * personal_norm_1 + guilt * guilt_1 + condition_guilt * condition * guilt_1 + condition_norms * condition * personal_norm_1
  V[['v2']]<- asc_value_2 + asc_2_shift_private * condition + payoff * possible_payoff_2 + norm * personal_norm_2 + guilt * guilt_2 + condition_guilt * condition * guilt_2 + condition_norms * condition * personal_norm_2
  V[['v3']]<- asc_value_3 + asc_3_shift_private * condition + payoff * possible_payoff_3 + norm * personal_norm_3 + guilt * guilt_3 + condition_guilt * condition * guilt_3 + condition_norms * condition * personal_norm_3
  V[['v4']]<- asc_value_4 + asc_4_shift_private * condition + payoff * possible_payoff_4 + norm * personal_norm_4 + guilt * guilt_4 + condition_guilt * condition * guilt_4 + condition_norms * condition * personal_norm_4
  V[['v5']]<- asc_value_5 + asc_5_shift_private * condition + payoff * possible_payoff_5 + norm * personal_norm_5 + guilt * guilt_5 + condition_guilt * condition * guilt_5 + condition_norms * condition * personal_norm_5
  V[['v6']]<- asc_value_6 + asc_6_shift_private * condition + payoff * possible_payoff_6 + norm * personal_norm_6 + guilt * guilt_6 + condition_guilt * condition * guilt_6 + condition_norms * condition * personal_norm_6
  V[['v7']]<- asc_value_7 + asc_7_shift_private * condition + payoff * possible_payoff_7 + norm * personal_norm_7 + guilt * guilt_7 + condition_guilt * condition * guilt_7 + condition_norms * condition * personal_norm_7
  V[['v8']]<- asc_value_8 + asc_8_shift_private * condition + payoff * possible_payoff_8 + norm * personal_norm_8 + guilt * guilt_8 + condition_guilt * condition * guilt_8 + condition_norms * condition * personal_norm_8
  V[['v9']]<- asc_value_9 + asc_9_shift_private * condition + payoff * possible_payoff_9 + norm * personal_norm_9 + guilt * guilt_9 + condition_guilt * condition * guilt_9 + condition_norms * condition * personal_norm_9
  V[['v10']] <- asc_value_10 + asc_10_shift_private * condition + payoff * possible_payoff_10 + norm * personal_norm_10 + guilt * guilt_10 + condition_guilt * condition * guilt_10 + condition_norms * condition * personal_norm_10
  V[['vexit']] <- asc_value_exit + asc_exit_shift_private * condition + payoff * pexit + norm * personal_norm_exit + guilt * guilt_exit + condition_guilt * condition * guilt_exit + condition_norms * condition * personal_norm_exit

This is the result:

Code: Select all

Model run using Apollo for R, version 0.1.0 
www.ApolloChoiceModelling.com

Model name                       : beispiel
Model description                : blabla
Model run at                     : 2020-05-13 13:26:56
Estimation method                : bfgs
Model diagnosis                  : successful convergence 
Number of individuals            : 281
Number of observations           : 281

Number of cores used             :  1 
Number of inter-person draws     : 100 (halton)
WARNING: Inter-person draws were used
         without a panel data structure.

LL(start)                        : -698.2588
LL(0)                            : -698.2588
LL(final)                        : -336.9148
Rho-square (0)                   :  0.5175 
Adj.Rho-square (0)               :  0.4717 
AIC                              :  737.83 
BIC                              :  854.26 
Estimated parameters             :  32
Time taken (hh:mm:ss)            :  00:14:8.61 
Iterations                       :  51 
Min abs eigenvalue of hessian    :  0 

Estimates:
                         Estimate  Std.err. t.ratio(0) Rob.std.err. Rob.t.ratio(0)
mu_log_payoff             -0.5118    0.0620      -8.26       0.0580          -8.83
sigma_log_v_payoff_inter  -0.0009    0.0596      -0.02       0.0016          -0.57
mu_log_guilt              -0.0528    0.1159      -0.46       0.1023          -0.52
sigma_log_v_guilt_inter    0.0024    0.0900       0.03       0.0039           0.61
mu_log_norm                0.9489    0.2913       3.26       0.2872           3.30
sigma_log_v_norm_inter     0.9129    0.3582       2.55       0.3685           2.48
condition_guilt           -0.1584    0.1473      -1.08       0.1309          -1.21
condition_norms           -0.2479    0.3822      -0.65       0.3748          -0.66
asc_value_0               11.4341 1685.4522       0.01       0.6595          17.34
asc_value_1                8.3262 1685.4524       0.00       1.0631           7.83
asc_value_2                8.7828 1685.4523       0.01       0.9045           9.71
asc_value_3                6.3819 1685.4526       0.00       1.3624           4.68
asc_value_4                7.2394 1685.4524       0.00       1.0454           6.92
asc_value_5                7.7598 1685.4524       0.00       1.0498           7.39
asc_value_6              -10.9021 1043.2783      -0.01       1.1359          -9.60
asc_value_7              -12.1538 3500.2843       0.00       1.0978         -11.07
asc_value_8              -12.9747 4028.5519       0.00       1.1152         -11.63
asc_value_9              -13.6013 4233.2051       0.00       1.1592         -11.73
asc_value_10             -10.9540 2359.7707       0.00       1.7876          -6.13
asc_value_exit            10.6619 1685.4524       0.01       0.9996          10.67
asc_0_shift_private       -1.9864 2410.8225       0.00       1.6954          -1.17
asc_1_shift_private       -1.8551 2410.8227       0.00       1.8820          -0.99
asc_2_shift_private       -2.3897 2410.8226       0.00       1.8459          -1.29
asc_3_shift_private       -0.9485 2410.8228       0.00       2.1320          -0.44
asc_4_shift_private       -2.4024 2410.8227       0.00       1.9391          -1.24
asc_5_shift_private       -3.6892 2410.8226       0.00       1.9274          -1.91
asc_6_shift_private       12.9082       NaN        NaN       0.7332          17.60
asc_7_shift_private       -1.6972 4206.9374       0.00       1.7801          -0.95
asc_8_shift_private       -2.5073 5177.9694       0.00       1.7553          -1.43
asc_9_shift_private       -3.1862 6154.4091       0.00       1.8266          -1.74
asc_10_shift_private      10.8388 3106.9923       0.00       2.5987           4.17
asc_exit_shift_private    -3.0851 2410.8226       0.00       1.9488          -1.58

Overview of choices for model component "MNL"
                                     v0     v1     v2     v3     v4     v5     v6  v7  v8  v9    v10  vexit
Times available                  281.00 281.00 281.00 281.00 281.00 281.00 281.00 281 281 281 281.00 281.00
Times chosen                      67.00   7.00  15.00   8.00  13.00  35.00   2.00   0   0   0   2.00 132.00
Percentage chosen overall         23.84   2.49   5.34   2.85   4.63  12.46   0.71   0   0   0   0.71  46.98
Percentage chosen when available  23.84   2.49   5.34   2.85   4.63  12.46   0.71   0   0   0   0.71  46.98

WARNING: some alternatives are never chosen in your data!
WARNING: Availability not provided to 'apollo_mnl' (or some elements are NA).
         Full availability assumed.

None of the ASCs are significant, does this mean that there is no main effect for condition?

And then my last question: the paper I build on did not use ASCs in their conditional logit model. To compare my results with theirs, I would like to also not include ASCs. I could do that by fitting a model for each condition and then compare the utility weights between models with a z-test. Is this a sensible thing to do to avoid including ASCs?

Thanks in advance for any answers!

Best,
Dominik

[stephanehess]

Hi Dominik

so what you are doing here is to allow for differencesin the value for the ASCs and the marginal utility coefficients as a function of the condition. That's the right thing to do. When you allow for differences in all terms, then this is equivalent to estimating two separate models, but you will be able to more easily see where the differences are as you are estimating a main effect and a difference.

But please note your model is overspecified as you are estimating ASCs for all alternatives. You need to normalise one to zero.

I don't understand your comment in relation to dropping the ASCs. They are an important part of any choice model as you otherwise make the assumption that the error terms all have the same mean.

Best wishes

Stephane

[dom]

Hi Stephane

Thanks for your answer, highly appreciate it! I set one of the constants to zero, as you suggested.

My question regarding the ASCs is this: I assume the paper I build on dropped the ASCs for a reson (I wrote the authors but haven't heard back yet). My assumption is that they did this because the ASCS are collinear with the payoff predictor (perfectly collinear in their case, strongly collinear in my case) [For alternatives 'v0' to 'v10' the payoff varies linearly, from 'v0' having a payoff of 10 to 'v10' having a payoff of zero and is the same for all subjects. The payoff for 'vexit' in my study represents a subject specific reservation price and thus varies between subjects and is also not linear as is the case with the payoffs of the other options. However, in the study that I build on the payoff for 'vexit' was also 10 (same as for 'v0') and if I fit my model with this configuration the Std.errors are not identifiable (model output not shown here).]

I also have data from a choice situation where the 'vexit' choice was not available (thus only having choices that are perfectly linear in the payoff). In this case the std. errors of the payoff predictor as well as of the ASCs are not identifiable:

Code: Select all

www.ApolloChoiceModelling.com

Model name                       : beispiel
Model description                : blabla
Model run at                     : 2020-05-19 12:35:34
Estimation method                : bfgs
Model diagnosis                  : successful convergence 
Number of individuals            : 281
Number of observations           : 281

Number of cores used             :  1 
Number of inter-person draws     : 100 (halton)
WARNING: Inter-person draws were used
         without a panel data structure.

LL(start)                        : -673.8086
LL(0)                            : -673.8086
LL(final)                        : -368.8546
Rho-square (0)                   :  0.4526 
Adj.Rho-square (0)               :  0.411 
AIC                              :  793.71 
BIC                              :  895.58 
Estimated parameters             :  28
Time taken (hh:mm:ss)            :  00:05:49.26 
Iterations                       :  59 
Min abs eigenvalue of hessian    :  0 
Some eigenvalues of hessian are positive, indicating potential problems!

Estimates:
                         Estimate  Std.err. t.ratio(0) Rob.std.err. Rob.t.ratio(0)
mu_log_payoff              1.4512       NaN        NaN       0.9807           1.48
sigma_log_v_payoff_inter   0.0934    0.1581       0.59       0.0483           1.93
mu_log_guilt              -0.5733    0.2000      -2.87       0.2123          -2.70
sigma_log_v_guilt_inter    0.5027    0.2261       2.22       0.2340           2.15
mu_log_norm                2.7212    0.5823       4.67       0.6184           4.40
sigma_log_v_norm_inter     2.2412    0.7203       3.11       1.0214           2.19
condition_guilt            0.2454    0.2704       0.91       0.2650           0.93
condition_norms            0.0207    0.6025       0.03       0.6434           0.03
asc_value_0                0.0000        NA         NA           NA             NA
asc_value_1               -1.2362       NaN        NaN       1.1165          -1.11
asc_value_2                1.0849       NaN        NaN       1.9657           0.55
asc_value_3                1.6733       NaN        NaN       2.9208           0.57
asc_value_4                2.1570       NaN        NaN       3.8413           0.56
asc_value_5                4.3068       NaN        NaN       4.7563           0.91
asc_value_6                1.7143       NaN        NaN       5.7268           0.30
asc_value_7               -9.9572       NaN        NaN       6.3879          -1.56
asc_value_8                4.7317       NaN        NaN       7.7337           0.61
asc_value_9               -8.4472       NaN        NaN       9.5006          -0.89
asc_value_10               6.0019       NaN        NaN       9.4709           0.63
asc_0_shift_private        0.0000        NA         NA           NA             NA
asc_1_shift_private        0.5878    0.6955       0.85       0.6681           0.88
asc_2_shift_private       -0.2166    0.5997      -0.36       0.5358          -0.40
asc_3_shift_private       -0.4934    0.7801      -0.63       0.6883          -0.72
asc_4_shift_private       -0.5631    0.9566      -0.59       0.9257          -0.61
asc_5_shift_private       -1.5737    1.0541      -1.49       0.9743          -1.62
asc_6_shift_private      -17.4070 3683.8410       0.00       1.4800         -11.76
asc_7_shift_private       12.2410  527.3570       0.02       1.4489           8.45
asc_8_shift_private      -19.4115 9018.5064       0.00       1.3899         -13.97
asc_9_shift_private       13.4219  637.7485       0.02       1.4427           9.30
asc_10_shift_private      -0.8835    2.4981      -0.35       1.7151          -0.52

Overview of choices for model component "MNL"
                                     v0     v1     v2     v3     v4     v5     v6     v7     v8     v9    v10
Times available                  281.00 281.00 281.00 281.00 281.00 281.00 281.00 281.00 281.00 281.00 281.00
Times chosen                      92.00  17.00  32.00  22.00  17.00  94.00   1.00   1.00   2.00   1.00   2.00
Percentage chosen overall         32.74   6.05  11.39   7.83   6.05  33.45   0.36   0.36   0.71   0.36   0.71
Percentage chosen when available  32.74   6.05  11.39   7.83   6.05  33.45   0.36   0.36   0.71   0.36   0.71

WARNING: Availability not provided to 'apollo_mnl' (or some elements are NA).
         Full availability assumed.

Am I right to conclude that this model and the model in the study I build on suffers from perfect multicollinearity and is therefore not identifiable?

Given that the model with ASCs and only options that are linear in the payoff (as the model printed above), the 'vexit' option should be the only option that influences the payoff preference weights (since this is the option that "breaks" the perfect multicollinearity between ASCs and the payoff variable). Right? This would explain why the payoff predictor is negative: subjects are willing to accept a decrease in payoff if they do not have to decide how to split the money (which is in line with our hypotheses). This is the only way a negative preference weight for the payoff makes sense to me, because people should prefer options with higher payoff and this is also what we see in the choice data (e.g., if subjects do not take the 'vexit' option most subjects kept the whole endowment for themselves and subjects almost never give more than 50% of the endowment to the recipient).

Given my interpretation is correct and my model suffers from multicollinearity, would this justify dropping the ASCs? I'm really sorry to bother you with these questions but I just can't wrap my mind around it. One of the reasons I ask is because both studies that did not include ASCs that I build on appeared in top journals and I therefore assume that their methodology should be (more or less) sound.

I paste the table of the conditional logit model from the paper I build on below and also another table from the same author of another study where they also used a conditional logit model without ASCs.

Thanks again in advance for any answers!

Best,
Dominik

Krupka, E. L., & Weber, R. A. (2013). Identifying social norms using coordination games: Why does dictator game sharing vary?. Journal of the European Economic Association, 11(3), 495-524.



Krupka, E. L., Leider, S., & Jiang, M. (2017). A meeting of the minds: informal agreements and social norms. Management Science, 63(6), 1708-1729.

[stephanehess]

Dominik

I think what you need to do first is to analyse the choices in the two subsets. Your results are quite possibly driven by a situation where nobody in the private segment chooses option 6 or 8

Best wishes

Stephane