Important: Read this before posting to this forum

  1. This forum is for questions related to the use of Apollo. We will answer some general choice modelling questions too, where appropriate, and time permitting. We cannot answer questions about how to estimate choice models with other software packages.
  2. There is a very detailed manual for Apollo available at http://www.ApolloChoiceModelling.com/manual.html. This contains detailed descriptions of the various Apollo functions, and numerous examples are available at http://www.ApolloChoiceModelling.com/examples.html. In addition, help files are available for all functions, using e.g. ?apollo_mnl
  3. Before asking a question on the forum, users are kindly requested to follow these steps:
    1. Check that the same issue has not already been addressed in the forum - there is a search tool.
    2. Ensure that the correct syntax has been used. For any function, detailed instructions are available directly in Apollo, e.g. by using ?apollo_mnl for apollo_mnl
    3. Check the frequently asked questions section on the Apollo website, which discusses some common issues/failures. Please see http://www.apollochoicemodelling.com/faq.html
    4. Make sure that R is using the latest official release of Apollo.
  4. If the above steps do not resolve the issue, then users should follow these steps when posting a question:
    1. provide full details on the issue, including the entire code and output, including any error messages
    2. posts will not immediately appear on the forum, but will be checked by a moderator first. This may take a day or two at busy times. There is no need to submit the post multiple times.

Error in LCNL model

Ask questions about the results reported after estimation. If the output includes errors, please include your model code if possible.
Post Reply
Chunyu Song
Posts: 5
Joined: 11 Aug 2022, 09:58

Error in LCNL model

Post by Chunyu Song »

Dear Apollo team,

I'm estimating individuals' mode choice using LCNL (Latent Class Nested Logit) model. The model is made-up of 4 choice sets (bus, railway, highway and airline), each of which contains 4 continuous variables (cost, access time, journey time and frequency) with 3 levels and 1 dummy variable (seat comfort) with 3 levels. However, when estimating the model I run into the following warning message:

(b_seatAm= seat_comfort_average*male
b_seatGm= seat_comfort_good*male)

Warning message:
In sqrt(diag(varcov)) : NaNs produced

I am using the following code to estimate the model:

Code: Select all

### Clear memory
rm(list = ls())

### Load Apollo library
library(apollo)

### Initialise code
apollo_initialise()

apollo_control = list(
  modelName ="Apollo_example_20_LC-NL-SCY",
  modelDescr ="LC model with class allocation model",
  indivID ="ID",
  nCores = 6
)
database = read.csv("D:\\QDU\\zhoudao\\Apollo\\mode choice-Apollo\\Apollo-mode choice-SCY.csv",header = TRUE)
database = subset(database,database$SP==1)
## scaled variables
apollo_beta = c(asc_bus_c1 = 0,
                asc_train_c1 = 0,
                asc_high_c1 = 0,
                asc_air_c1 = 0,
                b_cost_c1 = 0,
                b_acct_c1 = 0,
                b_jourt_c1 = 0,
                b_freq_c1 = 0,
                b_seat_P_c1 = 0,
                b_seat_A_c1 = 0,
                b_seat_G_c1 = 0,
                lambda_A_c1 = 1,
                lambda_B_c1 = 0.5,
                lambda_C_c1 = 1,
                delta_c1 = 0,
                asc_bus_c2 = 0,
                asc_train_c2 = 0,
                asc_high_c2 = 0,
                asc_air_c2 = 0,
                b_cost_c2 = 0,
                b_acct_c2 = 0,
                b_jourt_c2 = 0,
                b_freq_c2 = 0,
                b_seat_P_c2 = 0,
                b_seat_A_c2 = 0,
                b_seat_G_c2 = 0,
                lambda_A_c2 = 1,
                lambda_B_c2 = 0.5,
                lambda_C_c2 = 1,
                delta_c2 = 0,
                b_seatAm_c1 = 0,
                b_seatGm_c1 = 0,
                b_seatAm_c2 = 0,
                b_seatGm_c2 = 0
)
apollo_fixed = c("asc_bus_c1","asc_bus_c2","delta_c2","lambda_A_c1","lambda_A_c2","lambda_C_c1","lambda_C_c2","b_seat_P_c1","b_seat_P_c2")




## Define latent class components
apollo_lcPars = function(apollo_beta, apollo_inputs){
  lcpars = list()
  lcpars[["b_cost"]] = list(b_cost_c1,b_cost_c2)
  lcpars[["b_acct"]] = list(b_acct_c1,b_acct_c2)

  lcpars[["b_jourt"]] = list(b_jourt_c1,b_jourt_c2)
  lcpars[["b_freq"]] = list(b_freq_c1, b_freq_c2)
  lcpars[["b_seat_P"]] = list(b_seat_P_c1,b_seat_P_c2)
  lcpars[["b_seat_A"]] = list(b_seat_A_c1,b_seat_A_c2)
  lcpars[["b_seat_G"]] = list(b_seat_G_c1,b_seat_G_c2)
  lcpars[["asc_train"]] = list(asc_train_c1,asc_train_c2)
  lcpars[["asc_high"]] = list(asc_high_c1,asc_high_c2)
  lcpars[["asc_air"]] = list(asc_air_c1,asc_air_c2)
  lcpars[["b_seatAm"]] = list (b_seatAm_c1,b_seatAm_c2)
  lcpars[["b_seatGm"]] = list(b_seatGm_c1,b_seatGm_c2)

  
  #Utilities of class allocation model
  V = list()
  V[["class_1"]] = delta_c1
  V[["class_2"]] = delta_c2

  
  classAlloc_settings = list(
    classes      = c(class_1=1, class_2=2), 
    utilities    = V
  )
  
  lcpars[["pi_values"]] = apollo_classAlloc(classAlloc_settings)
  
  return(lcpars)
}



### Group and validate inputs
apollo_inputs = apollo_validateInputs()


### Define model and likelihood function

apollo_probabilities = function(apollo_beta, apollo_inputs, functionality = "estimate"){
  ## Attach inputs and detach after function exit
  apollo_attach(apollo_beta, apollo_inputs)
  on.exit(apollo_detach(apollo_beta, apollo_inputs))
  ### Create list of probabilities P
  P = list()

  ### Specify nests for NL model
  nlNests = list()
  nlNests[[1]] = list(root=1, A_c1=lambda_A_c1, B_c1 = lambda_B_c1, C_c1 = lambda_C_c1)
  nlNests[[2]] = list(root=1,A_c2=lambda_A_c2, B_c2 = lambda_B_c2, C_c2 = lambda_C_c2)

  
  nlStructure = list()
  nlStructure[[1]] = list()
  nlStructure[[2]] = list()

  
  nlStructure[[1]][["root"]] = c("A_c1","B_c1","C_c1")
  nlStructure[[2]][["root"]] = c("A_c2","B_c2","C_c2")

  
  
  nlStructure[[1]][["A_c1"]] = c("bus")
  nlStructure[[1]][["B_c1"]] = c("train","high")
  nlStructure[[1]][["C_c1"]] = c("air")
  nlStructure[[2]][["A_c2"]] = c("bus")
  nlStructure[[2]][["B_c2"]] = c("train","high")
  nlStructure[[2]][["C_c2"]] = c("air")

  
  
  nl_settings <- list(
    alternatives = c(bus=1, train=2, high=3, air=4),
    avail = list(bus=av_bus, train=av_train, high=av_high, air=av_air),
    choiceVar = choice,
    nlNests = nlNests,
    nlStructure = nlStructure
  )
  
  

  ### Loop over classes
  s=1
  while(s<=2){
    ### Compute class-specific utilities
    V=list()
    V[['bus']] =                    b_cost[[s]] * cost_bus +   b_acct[[s]] * acct_bus  +   b_jourt[[s]] * jourt_bus +   b_freq[[s]] * freq_bus +   b_seat_P[[s]] * (seat_bus==1) +   b_seat_A[[s]] * (seat_bus==2) +   b_seat_G[[s]] * (seat_bus==3) +  b_seatAm[[s]]*((seat_bus==2)*male) + b_seatGm[[s]]*((seat_bus==3)*male)
    V[['train']] = asc_train[[s]] + b_cost[[s]] * cost_train + b_acct[[s]] * acct_train  + b_jourt[[s]] * jourt_train + b_freq[[s]] * freq_train + b_seat_P[[s]] * (seat_train==1) + b_seat_A[[s]] * (seat_train==2) + b_seat_G[[s]] * (seat_train==3)+ b_seatAm[[s]]*((seat_train==2)*male)+b_seatGm[[s]]*((seat_train==3)*male)
    V[['high']] =  asc_high[[s]] +  b_cost[[s]] * cost_high +  b_acct[[s]] * acct_high  +  b_jourt[[s]] * jourt_high +  b_freq[[s]] * freq_high +  b_seat_P[[s]] * (seat_high==1) +  b_seat_A[[s]] * (seat_high==2) +  b_seat_G[[s]] * (seat_high==3) + b_seatAm[[s]]*((seat_high==2)*male)+ b_seatGm[[s]]*((seat_high==3)*male)
    V[['air']] =   asc_air[[s]] +   b_cost[[s]] * cost_air +   b_acct[[s]] * acct_air +    b_jourt[[s]] * jourt_air +   b_freq[[s]] * freq_air +   b_seat_P[[s]] * (seat_air==1) +   b_seat_A[[s]] * (seat_air==2) +   b_seat_G[[s]] * (seat_air==3) +  b_seatAm[[s]]*((seat_air==2)*male)+  b_seatGm[[s]]*((seat_air==3)*male)
    nl_settings$V = V
    nl_settings$nlNests = nlNests[[s]]
    nl_settings$nlStructure = nlStructure[[s]]
    P[[s]] = apollo_nl(nl_settings, functionality)
    P[[s]] = apollo_panelProd(P[[s]],apollo_inputs,functionality)
    
    s=s+1
  }
  lc_settings = list(inClassProb = P, classProb=pi_values)
  P[["model"]] = apollo_lc(lc_settings,apollo_inputs,functionality)
  
  P = apollo_prepareProb(P,apollo_inputs,functionality)
}

#apollo_control$worklnLogs=TRUE
model = apollo_estimate(apollo_beta, apollo_fixed, apollo_probabilities,apollo_inputs)
apollo_modelOutput(model,modelOutput_settings = list(printPVal=TRUE))
And, the outputs are as follows:

Code: Select all

Warning message:
In sqrt(diag(varcov)) : NaNs produced
> apollo_modelOutput(model,modelOutput_settings = list(printPVal=TRUE))
Model run by lenovo using Apollo 0.2.7 on R 4.2.1 for Windows.
www.ApolloChoiceModelling.com

Model name                       : Apollo_example_20_LC-NL-SCY
Model description                : LC model with class allocation model
Model run at                     : 2023-05-06 16:04:02
Estimation method                : bfgs
Model diagnosis                  : successful convergence 
Number of individuals            : 439
Number of rows in database       : 2634
Number of modelled outcomes      : 7902
                               1 : 2634
                               2 : 2634
                           model : 2634

Number of cores used             :  6 
Model without mixing

LL(start)                        : -3936.22
LL(0, whole model)               : -3651.5
LL(C, whole model)               : -3028.74
LL(final, whole model)           : -2629.62
Rho-square (0)                   :  0.2799 
Adj.Rho-square (0)               :  0.273 
Rho-square (C)                   :  0.1318 
Adj.Rho-square (C)               :  0.1235 
AIC                              :  5309.25 
BIC                              :  5483.62 

LL(0,component_1)                : -3651.5
LL(final,component_1)            : -Inf Likelihood equal to zero for at least
                                        one individual in this component.
LL(0,component_2)                : -3651.5
LL(final,component_2)            : -Inf Likelihood equal to zero for at least
                                        one individual in this component.

Estimated parameters             :  25
Time taken (hh:mm:ss)            :  00:02:9.77 
     pre-estimation              :  00:00:9.32 
     estimation                  :  00:01:23.39 
     post-estimation             :  00:00:37.06 
Iterations                       :  129  
Min abs eigenvalue of Hessian    :  1431139 
Some eigenvalues of Hessian are positive, indicating potential problems!

Unconstrained optimisation.

Estimates:
                Estimate        s.e.   t.rat.(0)  p(1-sided)    Rob.s.e. Rob.t.rat.(0)  p(1-sided)
asc_bus_c1      0.000000          NA          NA          NA          NA            NA          NA
asc_train_c1    3.248857         NaN         NaN         NaN    0.001106    2938.38622     0.00000
asc_high_c1     3.096513         NaN         NaN         NaN    0.001508    2053.82939     0.00000
asc_air_c1      1.846577    0.065261    28.29523    0.000000    0.123262      14.98096     0.00000
b_cost_c1      -0.001809         NaN         NaN         NaN   7.643e-06    -236.62763     0.00000
b_acct_c1      -0.150757         NaN         NaN         NaN    0.026421      -5.70598   5.784e-09
b_jourt_c1     -0.197729         NaN         NaN         NaN  1.5580e-04   -1269.09841     0.00000
b_freq_c1       0.014161         NaN         NaN         NaN  7.9082e-04      17.90612     0.00000
b_seat_P_c1     0.000000          NA          NA          NA          NA            NA          NA
b_seat_A_c1     0.167447    0.008071    20.74793    0.000000    0.002400      69.78094     0.00000
b_seat_G_c1    -0.040720         NaN         NaN         NaN    0.002171     -18.75382     0.00000
lambda_A_c1     1.000000          NA          NA          NA          NA            NA          NA
lambda_B_c1     0.007403         NaN         NaN         NaN    0.001313       5.63643   8.681e-09
lambda_C_c1     1.000000          NA          NA          NA          NA            NA          NA
delta_c1        0.162121    0.099651     1.62689    0.051880    0.104840       1.54638     0.06101
asc_bus_c2      0.000000          NA          NA          NA          NA            NA          NA
asc_train_c2    0.872793    0.131832     6.62049   1.790e-11    0.161702       5.39753   3.378e-08
asc_high_c2     0.946578    0.150436     6.29221   1.565e-10    0.144131       6.56750   2.558e-11
asc_air_c2     -0.013408    0.228424    -0.05870    0.476597    0.247061      -0.05427     0.47836
b_cost_c2      -0.003953  6.1395e-04    -6.43865   6.027e-11  6.7848e-04      -5.82625   2.834e-09
b_acct_c2      -0.255237    0.099688    -2.56035    0.005228    0.079069      -3.22804  6.2322e-04
b_jourt_c2     -0.038368    0.014965    -2.56384    0.005176    0.012041      -3.18652  7.1997e-04
b_freq_c2       0.002302    0.005912     0.38938    0.348499    0.004756       0.48400     0.31419
b_seat_P_c2     0.000000          NA          NA          NA          NA            NA          NA
b_seat_A_c2     0.577226    0.112626     5.12518   1.486e-07    0.115537       4.99603   2.926e-07
b_seat_G_c2     0.767851    0.110701     6.93629   2.013e-12    0.130821       5.86947   2.186e-09
lambda_A_c2     1.000000          NA          NA          NA          NA            NA          NA
lambda_B_c2     0.659898    0.148464     4.44483   4.398e-06    0.134554       4.90435   4.687e-07
lambda_C_c2     1.000000          NA          NA          NA          NA            NA          NA
delta_c2        0.000000          NA          NA          NA          NA            NA          NA
b_seatAm_c1     0.007009    0.004399     1.59340    0.055535    0.004567       1.53484     0.06241
b_seatGm_c1     0.005471    0.003246     1.68586    0.045912    0.004424       1.23663     0.10811
b_seatAm_c2    -0.586488    0.147873    -3.96618   3.652e-05    0.172829      -3.39347  3.4507e-04
b_seatGm_c2    -0.639784    0.159948    -3.99994   3.168e-05    0.188321      -3.39731  3.4026e-04

Nesting structure for NL model component 1:
Nest: root (1)
|-Nest: A_c1 (1)
|  '-Alternative: bus
|-Nest: B_c1 (0.0074)
|  |-Alternative: train
|  '-Alternative: high
'-Nest: C_c1 (1)
   '-Alternative: air

Nesting structure for NL model component 2:
Nest: root (1)
|-Nest: A_c2 (1)
|  '-Alternative: bus
|-Nest: B_c2 (0.6599)
|  |-Alternative: train
|  '-Alternative: high
'-Nest: C_c2 (1)
   '-Alternative: air


Summary of class allocation for LC model component :
         Mean prob.
Class_1      0.7317
Class_2      0.2683

Do you have any idea what is happening and is there any way to resolve this issue?

Best regards, Chunyu Song
stephanehess
Site Admin
Posts: 998
Joined: 24 Apr 2020, 16:29

Re: Error in LCNL model

Post by stephanehess »

Hi

this looks like an overspecification or other identification issue. Did you start with a NL model without the latent class structure? Did that work fine?

Stephane
--------------------------------
Stephane Hess
www.stephanehess.me.uk
Chunyu Song
Posts: 5
Joined: 11 Aug 2022, 09:58

Re: Error in LCNL model

Post by Chunyu Song »

Dear Stephane

Thanks for your reply, I have tested the NL model without the latent class structure, and the results were fine.

Chunyu Song
Last edited by Chunyu Song on 16 May 2023, 08:55, edited 2 times in total.
stephanehess
Site Admin
Posts: 998
Joined: 24 Apr 2020, 16:29

Re: Error in LCNL model

Post by stephanehess »

it looks like the issue could be that your nesting parameter goes almost to zero in one of the classes
--------------------------------
Stephane Hess
www.stephanehess.me.uk
Post Reply