Difference between standard vs robust t-ratios
Posted: 09 Dec 2025, 12:35
Dear Stephane and David,
I am estimating a fractional multinomial logit model using apollo. I notice that the standard t-ratios for most variables are quite low, while the robust t-ratios suggest that many of them are statistically significant.
I've never seen such a systematic discrepancy between standard and robust tests with other discrete choice models. Therefore I'm not sure whether this difference may reflect a property of the model (or the way it is implemented in apollo), rather than an issue with my data or code. Can I use the robust test to analyze the results in this case? Thank you in advance.
Below is the full output of the model.
Gabriel,
I am estimating a fractional multinomial logit model using apollo. I notice that the standard t-ratios for most variables are quite low, while the robust t-ratios suggest that many of them are statistically significant.
I've never seen such a systematic discrepancy between standard and robust tests with other discrete choice models. Therefore I'm not sure whether this difference may reflect a property of the model (or the way it is implemented in apollo), rather than an issue with my data or code. Can I use the robust test to analyze the results in this case? Thank you in advance.
Below is the full output of the model.
Code: Select all
Model name : Fractional
Model description : Fractional MNL model
Model run at : 2025-12-02 22:59:52.436716
Estimation method : bgw
Estimation diagnosis : Relative function convergence
Optimisation diagnosis : Maximum found
hessian properties : Negative definite
maximum eigenvalue : -0.124928
reciprocal of condition number : 1.01163e-08
Number of individuals : 5180
Number of rows in database : 5180
Number of modelled outcomes : 5180
Number of cores used : 1
Model without mixing
LL(start) : -5680.27
LL at equal shares, LL(0) : -5680.27
LL at observed shares, LL(C) : NA
LL(final) : -4557.28
Rho-squared vs equal shares : 0.1977
Adj.Rho-squared vs equal shares : 0.1921
Rho-squared vs observed shares : Not applicable
Adj.Rho-squared vs observed shares : Not applicable
AIC : 9178.55
BIC : 9388.23
Estimated parameters : 32
Time taken (hh:mm:ss) : 00:00:23.7
pre-estimation : 00:00:1.33
estimation : 00:00:7.77
post-estimation : 00:00:14.6
Iterations : 50
Unconstrained optimisation.
Estimates:
Estimate s.e. t.rat.(0) Rob.s.e. Rob.t.rat.(0)
asc_col 5.826301 2.790860 2.0876 0.348454 16.720
asc_ind 0.000000 NA NA NA NA
asc_atv -2.420021 1.140123 -2.1226 0.117835 -20.537
b_ttape 2.723348 0.425435 6.4013 0.040978 66.458
b_ttcol -7.816364 0.573848 -13.6210 0.102087 -76.565
b_ttind -2.585719 0.306861 -8.4264 0.026677 -96.925
b_pop2_col 0.088968 0.137526 0.6469 0.013708 6.490
b_pop3_col 0.111646 0.216836 0.5149 0.025029 4.461
b_pop4_col 0.047566 0.235148 0.2023 0.028696 1.658
b_pop5_col -0.131018 0.547853 -0.2391 0.063876 -2.051
b_pop2_atv 0.098774 0.089586 1.1026 0.006674 14.799
b_pop3_atv 0.068007 0.153841 0.4421 0.013005 5.229
b_pop4_atv -0.149704 0.187518 -0.7983 0.021359 -7.009
b_pop5_atv -0.554429 0.627334 -0.8838 0.048418 -11.451
b_pib_col 0.000000 NA NA NA NA
b_pib_atv -0.026192 0.056848 -0.4607 0.005422 -4.830
b_fleet_col -0.159229 0.130966 -1.2158 0.014913 -10.677
b_fleet_atv -0.046722 0.088861 -0.5258 0.008119 -5.754
b_gas -0.077313 0.134126 -0.5764 0.011393 -6.786
b_diesel -0.287663 0.388380 -0.7407 0.048337 -5.951
b_plan 0.000000 NA NA NA NA
b_orgao 0.000000 NA NA NA NA
b_conselho_atv 0.023318 0.111749 0.2087 0.010065 2.317
b_ciclovia 0.019607 0.075041 0.2613 0.005947 3.297
b_tarifa 0.053730 0.114811 0.4680 0.013497 3.981
b_app 0.000000 NA NA NA NA
b_planociclo 0.000000 NA NA NA NA
b_bici 0.000000 NA NA NA NA
b_bicicletario 0.000000 NA NA NA NA
b_calcada 0.000000 NA NA NA NA
b_fundo 0.000000 NA NA NA NA
b_concessao 0.000000 NA NA NA NA
b_norte_col -0.059849 0.293560 -0.2039 0.032554 -1.838
b_norte_atv -0.094518 0.169947 -0.5562 0.013721 -6.889
b_nordeste_col -0.230149 0.190529 -1.2079 0.021568 -10.671
b_nordeste_atv -0.062330 0.121295 -0.5139 0.010248 -6.082
b_sul_col 0.123148 0.151525 0.8127 0.016664 7.390
b_sul_atv -0.055750 0.106606 -0.5230 0.008147 -6.843
b_centro_oeste_col 0.125816 0.207938 0.6051 0.023721 5.304
b_centro_oeste_atv -0.063971 0.129182 -0.4952 0.009819 -6.515
b_via_pav 0.000000 NA NA NA NA
b_sex -0.002044 0.003651 -0.5600 7.0089e-04 -2.917
b_elderly 0.000000 NA NA NA NA
b_education 0.000000 NA NA NA NA
b_gini 0.293605 0.548000 0.5358 0.042453 6.916
b_dens_pop_atv 0.000000 NA NA NA NA
b_dens_pop_col 0.043743 0.049148 0.8900 0.006127 7.140
Overview of choices for FMNL model component :
atv col ind
Times available 5180.00 5154.00 5180.00
Observations with non-zero share 5180.00 5154.00 5180.00
Average share overall 0.38 0.11 0.51
Average share when available 0.38 0.11 0.51
20 most extreme outliers in terms of lowest average per choice prediction:
row Avg prob per choice
4100707 0.2921887
4205175 0.2927029
2805000 0.3030603
3300233 0.3065466
4116406 0.3070853
4121257 0.3077599
5217609 0.3118413
4302055 0.3124531
3549508 0.3144094
3160801 0.3154507
2100832 0.3157393
3516853 0.3173567
3106002 0.3177793
2805208 0.3189328
3170750 0.3191493
3300951 0.3194096
3170503 0.3194572
3302502 0.3195912
3118007 0.3197797
3300100 0.3207021
Settings and functions used in model definition:
apollo_control
--------------
Value
modelName "Fractional"
modelDescr "Fractional MNL model on time use data"
indivID "codigo"
outputDirectory "output/"
debug "FALSE"
nCores "1"
workInLogs "FALSE"
seed "13"
mixing "FALSE"
HB "FALSE"
noValidation "FALSE"
noDiagnostics "FALSE"
calculateLLC "TRUE"
analyticHessian "FALSE"
memorySaver "FALSE"
panelData "FALSE"
analyticGrad "TRUE"
analyticGrad_manualSet "FALSE"
overridePanel "FALSE"
preventOverridePanel "FALSE"
noModification "FALSE"
Hessian routines attempted
--------------------------
numerical jacobian of LL analytical gradient
Scaling used in computing Hessian
---------------------------------
Value
asc_col 5.826300950
asc_atv 2.420020668
b_ttape 2.723348348
b_ttcol 7.816364497
b_ttind 2.585718671
b_pop2_col 0.088967707
b_pop3_col 0.111645544
b_pop4_col 0.047565660
b_pop5_col 0.131018482
b_pop2_atv 0.098774327
b_pop3_atv 0.068006608
b_pop4_atv 0.149703784
b_pop5_atv 0.554429174
b_pib_atv 0.026192165
b_fleet_col 0.159228557
b_fleet_atv 0.046721716
b_gas 0.077312939
b_diesel 0.287662999
b_conselho_atv 0.023317903
b_ciclovia 0.019607110
b_tarifa 0.053729756
b_norte_col 0.059848875
b_norte_atv 0.094518409
b_nordeste_col 0.230149159
b_nordeste_atv 0.062330391
b_sul_col 0.123148128
b_sul_atv 0.055749988
b_centro_oeste_col 0.125816396
b_centro_oeste_atv 0.063970689
b_sex 0.002044499
b_gini 0.293604508
b_dens_pop_col 0.043743493
apollo_probabilities
----------------------
function(apollo_beta, apollo_inputs, functionality="estimate"){
### Attach inputs and detach after function exit
apollo_attach(apollo_beta, apollo_inputs)
on.exit(apollo_detach(apollo_beta, apollo_inputs))
### Create list of probabilities P
P = list()
### List of utilities: these must use the same names as in fmnl_settings, order is irrelevant
V = list()
V[["col"]] = asc_col +
b_ttcol*tt_col +
b_pop2_col*pop_2 +
b_pop3_col*pop_3 +
b_pop4_col*pop_4 +
b_pop5_col*pop_5 +
b_pib_col*log(pib_per_capita) +
b_fleet_col*log(rate_vehicle) +
b_diesel*mean_value_diesel +
b_plan*(mtra07=="Sim") +
b_orgao*orgao +
b_tarifa*politica_tarifa +
b_fundo*fundo +
b_concessao*concessao +
b_norte_col*norte +
b_sul_col*sul +
b_nordeste_col*nordeste +
b_centro_oeste_col*centro_oeste +
b_sex*ind_sex +
b_elderly*ind_elderly +
b_education*education +
b_gini*gini +
b_dens_pop_col*log(dens_pop)
V[["ind"]] = asc_ind +
b_ttind*tt_ind +
b_gas*mean_value +
b_app*app +
b_via_pav*total_via_pav
V[["atv"]] = asc_atv +
b_ttape*tt_ape +
b_pop2_atv*pop_2 +
b_pop3_atv*pop_3 +
b_pop4_atv*pop_4 +
b_pop5_atv*pop_5 +
b_pib_atv*log(pib_per_capita) +
b_fleet_atv*log(rate_vehicle) +
b_plan*(mtra07=="Sim") +
b_orgao*orgao +
b_conselho_atv*conselho +
b_ciclovia*ciclovia +
b_planociclo*plano_cicloviario +
b_bici*total_bici +
b_bicicletario*bicicletario +
b_calcada*total_calcada +
b_fundo*fundo +
b_norte_atv*norte +
b_sul_atv*sul +
b_nordeste_atv*nordeste +
b_centro_oeste_atv*centro_oeste +
b_sex*ind_sex +
b_elderly*ind_elderly +
b_education*education +
b_gini*gini +
b_dens_pop_atv*log(dens_pop)
### Define settings for MNL model component
fmnl_settings = list(
alternatives = c("atv","col","ind"),
choiceShares = list(atv = perc_atv,
col = perc_col,
ind = perc_ind),
utilities = V,
avail = list(atv=1,ind=1,col=avail_col)
)
### Compute probabilities using FMNL model
P[["model"]] = apollo_fmnl(fmnl_settings, functionality)
### Prepare and return outputs of function
P = apollo_prepareProb(P, apollo_inputs, functionality)
return(P)
}