Important: Read this before posting to this forum

  1. This forum is for questions related to the use of Apollo. We will answer some general choice modelling questions too, where appropriate, and time permitting. We cannot answer questions about how to estimate choice models with other software packages.
  2. There is a very detailed manual for Apollo available at http://www.ApolloChoiceModelling.com/manual.html. This contains detailed descriptions of the various Apollo functions, and numerous examples are available at http://www.ApolloChoiceModelling.com/examples.html. In addition, help files are available for all functions, using e.g. ?apollo_mnl
  3. Before asking a question on the forum, users are kindly requested to follow these steps:
    1. Check that the same issue has not already been addressed in the forum - there is a search tool.
    2. Ensure that the correct syntax has been used. For any function, detailed instructions are available directly in Apollo, e.g. by using ?apollo_mnl for apollo_mnl
    3. Check the frequently asked questions section on the Apollo website, which discusses some common issues/failures. Please see http://www.apollochoicemodelling.com/faq.html
    4. Make sure that R is using the latest official release of Apollo.
  4. If the above steps do not resolve the issue, then users should follow these steps when posting a question:
    1. provide full details on the issue, including the entire code and output, including any error messages
    2. posts will not immediately appear on the forum, but will be checked by a moderator first. This may take a day or two at busy times. There is no need to submit the post multiple times.

LC models class probability and confidence interval

Ask questions about the results reported after estimation. If the output includes errors, please include your model code if possible.
Post Reply
simbala
Posts: 2
Joined: 07 Oct 2024, 04:03

LC models class probability and confidence interval

Post by simbala »

Hi Apollo Team,

I have two questions regarding the output, but I couldn't find a solution after reading the manual. Using this LC model as an example http://www.apollochoicemodelling.com/fi ... variates.r


1. Is there a way to save the estimated mean class probability in the output file? For example, if I have 2 classes with the mean probability of 0.7 and 0.3, how can I save those numbers in the final output file?

2. How can I also get the output for the 95% confidence interval of the estimates? I can figure out a way to output the p-values by adding

Code: Select all

saveOutput_settings = list(printPVal=2)
Is there a similar way to get and save 95% CI as well?

Thank you in advance for your help!
stephanehess
Site Admin
Posts: 1210
Joined: 24 Apr 2020, 16:29

Re: LC models class probability and confidence interval

Post by stephanehess »

Hi

1. the mean class probabilites are saved in the output file alraedy, see e.g. at https://apollochoicemodelling.com/files ... output.txt

2. we do not automatically compute CIs. You can compute those easily as the estimate +/- 1.96 * s.e.

Stephane
--------------------------------
Stephane Hess
www.stephanehess.me.uk
simbala
Posts: 2
Joined: 07 Oct 2024, 04:03

Re: LC models class probability and confidence interval

Post by simbala »

Hi Professor Hess,

Thank you! I just checked I used the same code from https://apollochoicemodelling.com/files ... output.txt, but my output file looked different. I am missing many parts of the output, including summary of model component. I couldn't identify where it might have gone wrong. Could you help me with this issue?

What I got is

Code: Select all

Model run by xx using Apollo 0.3.3 on R 4.4.1 for Windows.
Please acknowledge the use of Apollo by citing Hess & Palma (2019)
  DOI 10.1016/j.jocm.2019.100170
  www.ApolloChoiceModelling.com

Model name                                  : LC_with_covariates
Model description                           : LC model with covariates in class allocation model on Swiss route choice data
Model run at                                : 2024-10-30 14:48:30.092399
Estimation method                           : bgw
Model diagnosis                             : Relative function convergence
Optimisation diagnosis                      : Maximum found
     hessian properties                     : Negative definite
     maximum eigenvalue                     : -8.872086
     reciprocal of condition number         : 5.37477e-05
Number of individuals                       : 388
Number of rows in database                  : 3492
Number of modelled outcomes                 : 3492

Number of cores used                        :  2 
Model without mixing

LL(start)                                   : -1755.5
LL (whole model) at equal shares, LL(0)     : -2420.47
LL (whole model) at observed shares, LL(C)  : -2420.39
LL(final, whole model)                      : -1559.01
Rho-squared vs equal shares                  :  0.3559 
Adj.Rho-squared vs equal shares              :  0.3509 
Rho-squared vs observed shares               :  0.3559 
Adj.Rho-squared vs observed shares           :  0.3518 
AIC                                         :  3142.03 
BIC                                         :  3215.93 

LL(0,Class_1)                    : -2420.47
LL(final,Class_1)                : -1776.33
LL(0,Class_2)                    : -2420.47
LL(final,Class_2)                : -2467.09

Estimated parameters                        : 12
Time taken (hh:mm:ss)                       :  00:00:3.93 
     pre-estimation                         :  00:00:2.74 
     estimation                             :  00:00:0.41 
     post-estimation                        :  00:00:0.77 
Iterations                                  :  23  

Unconstrained optimisation.

Estimates:
 
Summary of class allocation for model component :


Overview of choices for MNL model component Class_1:
                                    alt1    alt2
Times available                  3492.00 3492.00
Times chosen                     1734.00 1758.00
Percentage chosen overall          49.66   50.34
Percentage chosen when available   49.66   50.34



Overview of choices for MNL model component Class_2:
                                    alt1    alt2
Times available                  3492.00 3492.00
Times chosen                     1734.00 1758.00
Percentage chosen overall          49.66   50.34
Percentage chosen when available   49.66   50.34

Classical covariance matrix:

Robust covariance matrix:

Classical correlation matrix:
  
Robust correlation matrix:

 20 most extreme outliers in terms of lowest average per choice prediction:
  
Changes in parameter estimates from starting values:

Settings and functions used in model definition:

apollo_control
--------------
                       Value                                                                          
modelName              "example_LC_with_covariates"                                                           
modelDescr             "LC model with covariates in class allocation model on Swiss route choice data"
indivID                "ID"                                                                           
nCores                 "2"                                                                            
outputDirectory        "results/"                                                                     
debug                  "FALSE"                                                                        
workInLogs             "FALSE"                                                                        
seed                   "13"                                                                           
mixing                 "FALSE"                                                                        
HB                     "FALSE"                                                                        
noValidation           "FALSE"                                                                        
noDiagnostics          "FALSE"                                                                        
calculateLLC           "TRUE"                                                                         
analyticHessian        "FALSE"                                                                        
memorySaver            "FALSE"                                                                        
panelData              "TRUE"                                                                         
analyticGrad           "TRUE"                                                                         
analyticGrad_manualSet "FALSE"                                                                        
overridePanel          "FALSE"                                                                        
preventOverridePanel   "FALSE"                                                                        
noModification         "FALSE"                                                                        

Hessian routines attempted
--------------------------
numerical jacobian of LL analytical gradient

Scaling used in computing Hessian
---------------------------------
                     Value
asc_1           0.03462696
beta_tt_a       0.03724285
beta_tt_b       0.20952989
beta_tc_a       0.04989347
beta_tc_b       0.77576327
beta_hw_a       0.03426097
beta_hw_b       0.05466599
beta_ch_a       0.60949847
beta_ch_b       2.67045554
delta_a         0.20944353
gamma_commute_a 0.43370076
gamma_car_av_a  0.58813204


apollo_lcPars
---------------
function(apollo_beta, apollo_inputs){
  lcpars = list()
  lcpars[["beta_tt"]] = list(beta_tt_a, beta_tt_b)
  lcpars[["beta_tc"]] = list(beta_tc_a, beta_tc_b)
  lcpars[["beta_hw"]] = list(beta_hw_a, beta_hw_b)
  lcpars[["beta_ch"]] = list(beta_ch_a, beta_ch_b)
  
  ### Utilities of class allocation model
  V=list()
  V[["class_a"]] = delta_a + gamma_commute_a*commute + gamma_car_av_a*car_availability
  V[["class_b"]] = delta_b + gamma_commute_b*commute + gamma_car_av_b*car_availability
  
  ### Settings for class allocation models
  classAlloc_settings = list(
    classes      = c(class_a=1, class_b=2), 
    utilities    = V  
  )
  
  lcpars[["pi_values"]] = apollo_classAlloc(classAlloc_settings)
  
  return(lcpars)
}
<bytecode:0x0000024490a01060>


apollo_probabilities
----------------------
function(apollo_beta, apollo_inputs, functionality="estimate"){
  
  ### Attach inputs and detach after function exit
  apollo_attach(apollo_beta, apollo_inputs)
  on.exit(apollo_detach(apollo_beta, apollo_inputs))
  
  ### Create list of probabilities P
  P = list()
  
  ### Define settings for MNL model component that are generic across classes
  mnl_settings = list(
    alternatives = c(alt1=1, alt2=2),
    avail        = list(alt1=1, alt2=1),
    choiceVar    = choice
  )
  
  ### Loop over classes
  for(s in 1:2){
    
    ### Compute class-specific utilities
    V=list()
    V[["alt1"]]  = asc_1 + beta_tc[[s]]*tc1 + beta_tt[[s]]*tt1 + beta_hw[[s]]*hw1 + beta_ch[[s]]*ch1
    V[["alt2"]]  = asc_2 + beta_tc[[s]]*tc2 + beta_tt[[s]]*tt2 + beta_hw[[s]]*hw2 + beta_ch[[s]]*ch2
    
    mnl_settings$utilities     = V
    mnl_settings$componentName = paste0("Class_",s)
    
    ### Compute within-class choice probabilities using MNL model
    P[[paste0("Class_",s)]] = apollo_mnl(mnl_settings, functionality)
    
    ### Take product across observation for same individual
    P[[paste0("Class_",s)]] = apollo_panelProd(P[[paste0("Class_",s)]], apollo_inputs ,functionality)
    
  }
  
  ### Compute latent class model probabilities
  lc_settings  = list(inClassProb = P, classProb=pi_values)
  P[["model"]] = apollo_lc(lc_settings, apollo_inputs, functionality)
  
  ### Prepare and return outputs of function
  P = apollo_prepareProb(P, apollo_inputs, functionality)
  return(P)
}
Thank you!
stephanehess
Site Admin
Posts: 1210
Joined: 24 Apr 2020, 16:29

Re: LC models class probability and confidence interval

Post by stephanehess »

Hi

do you see the estimates on screen if you use apollo_modelOutput?

We've never seen this issue before and it might be specific to your machine

Stephane
--------------------------------
Stephane Hess
www.stephanehess.me.uk
Post Reply