Important: Read this before posting to this forum

  1. This forum is for questions related to the use of Apollo. We will answer some general choice modelling questions too, where appropriate, and time permitting. We cannot answer questions about how to estimate choice models with other software packages.
  2. There is a very detailed manual for Apollo available at http://www.ApolloChoiceModelling.com/manual.html. This contains detailed descriptions of the various Apollo functions, and numerous examples are available at http://www.ApolloChoiceModelling.com/examples.html. In addition, help files are available for all functions, using e.g. ?apollo_mnl
  3. Before asking a question on the forum, users are kindly requested to follow these steps:
    1. Check that the same issue has not already been addressed in the forum - there is a search tool.
    2. Ensure that the correct syntax has been used. For any function, detailed instructions are available directly in Apollo, e.g. by using ?apollo_mnl for apollo_mnl
    3. Check the frequently asked questions section on the Apollo website, which discusses some common issues/failures. Please see http://www.apollochoicemodelling.com/faq.html
    4. Make sure that R is using the latest official release of Apollo.
  4. If the above steps do not resolve the issue, then users should follow these steps when posting a question:
    1. provide full details on the issue, including the entire code and output, including any error messages
    2. posts will not immediately appear on the forum, but will be checked by a moderator first. This may take a day or two at busy times. There is no need to submit the post multiple times.

LL final value for RUM mode choices

Ask questions about errors you encouunter. Please make sure to include full details about your model specifications, and ideally your model file.
Post Reply
paulojunio
Posts: 1
Joined: 16 Aug 2022, 17:54

LL final value for RUM mode choices

Post by paulojunio »

Hello, my name is Paulo Júnio.
I have been working in a code about mode choice and my results are not in the normal intervals found in the literature. The final results of LLfinal value and rho² are too high. I don't if the problem is on the code or in the database. Could anyone help me, please? Thanks.

My code and my results are written below and my database is attached on https://we.tl/t-7bYBWMEDBp
Paulo Júnio Moura Rosa

CODE

#setting work directory

setwd("C:\\Users\\Paulo Junior\\Documents\\img")

rm(list = ls())

### Load Apollo library
library(apollo)

### Initialise code
apollo_initialise()

### Set core controls
apollo_control = list(
modelName ="Modelo_espacial_modos_1000_calib_filtro",
modelDescr ="Modelo logit espacial com os atributos dos modos",
indivID ="X"
)


# ################################################################# #
#### LOAD DATA AND APPLY ANY TRANSFORMATIONS ####
# ################################################################# #

database = read.csv("C:\\Users\\Paulo Junior\\Documents\\img\\database_espacial_calib_60_1000.csv",header=TRUE)
# ################################################################# #
#### ANALYSIS OF CHOICES ####
# ################################################################# #

choiceAnalysis_settings <- list(
alternatives = c(tp=0, walk=1),
avail = list(tp=database$av_tp, walk=database$av_walk),
choiceVar = database$choice,
explanators = database[,c("sexo","cr1","cr2","age1","age2","grau_ins1","grau_ins2","no_morad","pico_manha")]
)

apollo_choiceAnalysis(choiceAnalysis_settings, apollo_control, database)

# ################################################################# #
#### DEFINE MODEL PARAMETERS ####
# ################################################################# #

### Vector of parameters, including any that are kept fixed in estimation
apollo_beta=c(asc_tp = 0,
asc_walk = 0,
b_tt_walk = 0,
b_tt_tp = 0,
b_co_tp = 0,
b_d_esto = 0,
b_d_estd = 0)

### Vector with names (in quotes) of parameters to be kept fixed at their starting value in apollo_beta, use apollo_beta_fixed = c() if none
apollo_fixed = c("asc_walk")

# ################################################################# #
#### GROUP AND VALIDATE INPUTS ####
# ################################################################# #

apollo_inputs = apollo_validateInputs()

# ################################################################# #
#### DEFINE MODEL AND LIKELIHOOD FUNCTION ####
# ################################################################# #

apollo_probabilities=function(apollo_beta, apollo_inputs, functionality="estimate"){

### Attach inputs and detach after function exit
apollo_attach(apollo_beta, apollo_inputs)
on.exit(apollo_detach(apollo_beta, apollo_inputs))

### Create list of probabilities P
P = list()

### List of utilities: these must use the same names as in mnl_settings, order is irrelevant
V = list()
V[['tp']] = asc_tp + b_tt_tp * tt_tp + b_co_tp * co_tp
V[['walk']] = asc_walk + b_tt_walk * t_ape_od + b_d_esto * d_esto + b_d_estd * d_estd


### Define settings for MNL model component
mnl_settings = list(
alternatives = c(tp=0, walk=1),
avail = list(tp=av_tp, walk=av_walk),
choiceVar = choice,
V = V
)

### Compute probabilities using MNL model
P[['model']] = apollo_mnl(mnl_settings, functionality)

### Take product across observation for same individual
#P = apollo_panelProd(P, apollo_inputs, functionality)

### Prepare and return outputs of function
P = apollo_prepareProb(P, apollo_inputs, functionality)
return(P)
}

# ################################################################# #
#### MODEL ESTIMATION ####
# ################################################################# #

model = apollo_estimate(apollo_beta, apollo_fixed, apollo_probabilities, apollo_inputs)

# ################################################################# #
#### MODEL OUTPUTS ####
# ################################################################# #

# ----------------------------------------------------------------- #
#---- FORMATTED OUTPUT (TO SCREEN) ----
# ----------------------------------------------------------------- #

apollo_modelOutput(model)

# ----------------------------------------------------------------- #
#---- FORMATTED OUTPUT (TO FILE, using model name) ----
# ----------------------------------------------------------------- #

apollo_saveOutput(model,saveOutput_settings =list(printPVal=2,printT1=1,printDataReport=TRUE))

RESULTS

Model run by smauad using Apollo 0.2.7 on R 4.1.2 for Windows.
www.ApolloChoiceModelling.com

Model name : Modelo_espacial_modos_1000_calib_filtro
Model description : Modelo logit espacial com os atributos dos modos
Model run at : 2022-08-15 22:16:08
Estimation method : bfgs
Model diagnosis : successful convergence
Number of individuals : 2214
Number of rows in database : 2214
Number of modelled outcomes : 2214

Number of cores used : 1
Model without mixing

LL(start) : -1534.63
LL(0) : -1534.63
LL(C) : -1534.57
LL(final) : -350.43
Rho-square (0) : 0.7717
Adj.Rho-square (0) : 0.7677
Rho-square (C) : 0.7716
Adj.Rho-square (C) : 0.7677
AIC : 712.85
BIC : 747.07

Estimated parameters : 6
Time taken (hh:mm:ss) : 00:00:1.81
pre-estimation : 00:00:0.44
estimation : 00:00:1.1
post-estimation : 00:00:0.27
Iterations : 64
Min abs eigenvalue of Hessian : 0.000358

Unconstrained optimisation.

Estimates:
Estimate s.e. t.rat.(0) p(2-sided) t.rat(1) p(2-sided) Rob.s.e. Rob.t.rat.(0) p(2-sided)
asc_tp -70.553606 51.08886 -1.3810 0.1673 -1.401 0.1613 5.971978 -11.8141 0.0000
asc_walk 0.000000 NA NA NA NA NA NA NA NA
b_tt_walk -0.169249 0.01063 -15.9178 0.0000 -109.967 0.0000 0.010426 -16.2333 0.0000
b_tt_tp 0.001082 0.01079 0.1003 0.9201 -92.613 0.0000 0.009750 0.1109 0.9117
b_co_tp 17.920241 13.44559 1.3328 0.1826 1.258 0.2082 1.572265 11.3977 0.0000
b_d_esto 0.001469 2.3933e-04 6.1394 8.283e-10 -4172.191 0.0000 2.0784e-04 7.0697 1.553e-12
b_d_estd 0.001861 2.9052e-04 6.4040 1.513e-10 -3435.684 0.0000 2.6527e-04 7.0137 2.322e-12
Rob.t.rat.(1) p(2-sided)
asc_tp -11.98 0.000
asc_walk NA NA
b_tt_walk -112.15 0.000
b_tt_tp -102.45 0.000
b_co_tp 10.76 0.000
b_d_esto -4804.38 0.000
b_d_estd -3762.73 0.000


Overview of choices for MNL model component :
tp walk
Times available 2214.00 2214.00
Times chosen 1115.00 1099.00
Percentage chosen overall 50.36 49.64
Percentage chosen when available 50.36 49.64


Classical covariance matrix:
asc_tp b_tt_walk b_tt_tp b_co_tp b_d_esto b_d_estd
asc_tp 2610.071650 0.014219 -0.029214 -686.908221 -6.4445e-04 -0.001782
b_tt_walk 0.014219 1.1306e-04 3.618e-05 -0.003411 -3.068e-07 -6.167e-08
b_tt_tp -0.029214 3.618e-05 1.1634e-04 0.007365 1.294e-06 1.831e-06
b_co_tp -686.908221 -0.003411 0.007365 180.783976 1.7037e-04 4.6848e-04
b_d_esto -6.4445e-04 -3.068e-07 1.294e-06 1.7037e-04 5.728e-08 1.843e-08
b_d_estd -0.001782 -6.167e-08 1.831e-06 4.6848e-04 1.843e-08 8.440e-08

Robust covariance matrix:
asc_tp b_tt_walk b_tt_tp b_co_tp b_d_esto b_d_estd
asc_tp 35.664525 0.015289 -0.024434 -9.380575 -4.6177e-04 -0.001468
b_tt_walk 0.015289 1.0870e-04 3.030e-05 -0.003723 -5.088e-07 -9.620e-08
b_tt_tp -0.024434 3.030e-05 9.506e-05 0.006166 1.025e-06 1.530e-06
b_co_tp -9.380575 -0.003723 0.006166 2.472018 1.1969e-04 3.8494e-04
b_d_esto -4.6177e-04 -5.088e-07 1.025e-06 1.1969e-04 4.320e-08 1.305e-08
b_d_estd -0.001468 -9.620e-08 1.530e-06 3.8494e-04 1.305e-08 7.037e-08

Classical correlation matrix:
asc_tp b_tt_walk b_tt_tp b_co_tp b_d_esto b_d_estd
asc_tp 1.00000 0.02618 -0.05302 -0.99998 -0.05271 -0.12007
b_tt_walk 0.02618 1.00000 0.31544 -0.02386 -0.12055 -0.01997
b_tt_tp -0.05302 0.31544 1.00000 0.05079 0.50129 0.58432
b_co_tp -0.99998 -0.02386 0.05079 1.00000 0.05294 0.11993
b_d_esto -0.05271 -0.12055 0.50129 0.05294 1.00000 0.26503
b_d_estd -0.12007 -0.01997 0.58432 0.11993 0.26503 1.00000

Robust correlation matrix:
asc_tp b_tt_walk b_tt_tp b_co_tp b_d_esto b_d_estd
asc_tp 1.0000 0.24554 -0.4196 -0.9990 -0.3720 -0.92652
b_tt_walk 0.2455 1.00000 0.2980 -0.2271 -0.2348 -0.03478
b_tt_tp -0.4196 0.29803 1.0000 0.4022 0.5056 0.59167
b_co_tp -0.9990 -0.22709 0.4022 1.0000 0.3663 0.92295
b_d_esto -0.3720 -0.23482 0.5056 0.3663 1.0000 0.23667
b_d_estd -0.9265 -0.03478 0.5917 0.9230 0.2367 1.00000

20 worst outliers in terms of lowest average per choice prediction:
row Avg prob per choice
1794 0.004446392
534 0.021477366
437 0.024035930
224 0.027617440
1766 0.035189330
608 0.037264231
661 0.037308480
1 0.044653198
1113 0.046397421
1339 0.052472827
1650 0.058746748
2086 0.065114515
282 0.068367327
1436 0.068714080
1258 0.069585206
493 0.071161888
1206 0.075885430
1762 0.075930387
2028 0.081325492
1428 0.082729577

Changes in parameter estimates from starting values:
Initial Estimate Difference
asc_tp 0.000 -70.553606 -70.553606
asc_walk 0.000 0.000000 0.000000
b_tt_walk 0.000 -0.169249 -0.169249
b_tt_tp 0.000 0.001082 0.001082
b_co_tp 0.000 17.920241 17.920241
b_d_esto 0.000 0.001469 0.001469
b_d_estd 0.000 0.001861 0.001861

Settings and functions used in model definition:

apollo_control
--------------
Value
modelName "Modelo_espacial_modos_1000_calib_filtro"
modelDescr "Modelo logit espacial com os atributos dos modos"
indivID "X"
debug "FALSE"
nCores "1"
workInLogs "FALSE"
seed "13"
mixing "FALSE"
HB "FALSE"
noValidation "FALSE"
noDiagnostics "FALSE"
calculateLLC "TRUE"
outputDirectory "D:/Downloads/"
panelData "FALSE"
analyticGrad "TRUE"
analyticGrad_manualSet "FALSE"

Hessian routines attempted
--------------
numerical jacobian of LL analytical gradient

Scaling in estimation
--------------
Value
asc_tp 70.545344978
b_tt_walk 0.169383036
b_tt_tp 0.001081562
b_co_tp 17.917369037
b_d_esto 0.001469822
b_d_estd 0.001860690

Scaling used in computing Hessian
--------------
Value
asc_tp 70.553605836
b_tt_walk 0.169249237
b_tt_tp 0.001081565
b_co_tp 17.920240621
b_d_esto 0.001469345
b_d_estd 0.001860510


apollo_probabilities
--------------------
function(apollo_beta, apollo_inputs, functionality="estimate"){

### Attach inputs and detach after function exit
apollo_attach(apollo_beta, apollo_inputs)
on.exit(apollo_detach(apollo_beta, apollo_inputs))

### Create list of probabilities P
P = list()

### List of utilities: these must use the same names as in mnl_settings, order is irrelevant
V = list()
V[['tp']] = asc_tp + b_tt_tp * tt_tp + b_co_tp * co_tp
V[['walk']] = asc_walk + b_tt_walk * t_ape_od + b_d_esto * d_esto + b_d_estd * d_estd


### Define settings for MNL model component
mnl_settings = list(
alternatives = c(tp=0, walk=1),
avail = list(tp=av_tp, walk=av_walk),
choiceVar = choice,
V = V
)

### Compute probabilities using MNL model
P[['model']] = apollo_mnl(mnl_settings, functionality)

### Take product across observation for same individual
#P = apollo_panelProd(P, apollo_inputs, functionality)

### Prepare and return outputs of function
P = apollo_prepareProb(P, apollo_inputs, functionality)
return(P)
}
<bytecode: 0x0000017e290e41c8>
stephanehess
Site Admin
Posts: 974
Joined: 24 Apr 2020, 16:29

Re: LL final value for RUM mode choices

Post by stephanehess »

Hi

model fit for mode choice models can be very high, so I wouldn't worry about that per se.

But your model clearly has other issues - look at your very negative ASC and the wrongly signed time coefficient.

I suggest you start off by making the coefficients generic, i.e. not mode specific

Stephane
--------------------------------
Stephane Hess
www.stephanehess.me.uk
Post Reply