Page 1 of 1

Bug multi-core with modular code

Posted: 10 May 2022, 08:51
by alvarogutyerrez
Dear Stephane and David,

I am using apollo to run a Mixed Logit (MIXL) model, but I use the starting values from a simple Multinomial Logit (MNL) model. My program (see below) runs perfectly when I use one core. However, it fails when using more than one displaying the following error:

Code: Select all

Pre-processing likelihood function...
Preparing workers for multithreading...

Error in apollo_makeLogLike(apollo_beta, apollo_fixed, apollo_probabilities,  : 
  object 'apollo_beta' not found
Below you might see my Rsession:

Code: Select all

> sessionInfo(package = NULL)
R version 4.2.0 (2022-04-22 ucrt)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 10 x64 (build 19044)

Matrix products: default

locale:
[1] LC_COLLATE=Spanish_Latin America.utf8 
[2] LC_CTYPE=Spanish_Latin America.utf8   
[3] LC_MONETARY=Spanish_Latin America.utf8
[4] LC_NUMERIC=C                          
[5] LC_TIME=Spanish_Latin America.utf8    

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
 [1] forcats_0.5.1   stringr_1.4.0   dplyr_1.0.8     purrr_0.3.4    
 [5] tidyr_1.2.0     tibble_3.1.6    ggplot2_3.3.5   tidyverse_1.3.1
 [9] apollo_0.2.7    readr_2.1.2    
 
Here is the main program I mention at the very beginning:

Code: Select all

library(readr)
library(apollo)
library(tidyverse)

df_raw <- as_tibble(read.table("http://transp-or.epfl.ch/data/swissmetro.dat",header = T))

## data procesing
database = df_raw %>%
  dplyr::rename_all(.funs = tolower) %>%
  dplyr::filter( !((purpose != 1) &  (purpose != 3)) ) %>%
  dplyr::filter( (choice != 0) ) %>%
  dplyr::group_by(id) %>%
  mutate(across(ends_with('_co'), ~ . / 100)) %>%
  mutate(across(ends_with('_tt'), ~ . / 100)) %>%
  mutate(sm_co = sm_co * (ga == 0),
         train_co = train_co * (ga == 0)
  ) %>%
  as.data.frame()

### ################################ ###
#### Simple MNL for starting values ####
### ################################ ###

# Initialize code
# Set core controls
apollo_control_mnl = list(
  modelName       = "MIXL",
  modelDescr      = "Mixed logit model on swissmetro",
  indivID         = "id",
  mixing          = FALSE,
  nCores          = 1,
  outputDirectory = NULL
)
# Vector of parameters, including any that are kept fixed in estimation
apollo_beta_mnl = c(
  ASC_car=0,
  ASC_sm = 0,

  b_time_mu    = 0,
  b_cost_mu    = 0,

  b_he = 0,

  b_age  =0 ,
  b_ga = 0,
  b_seats = 0,
  b_luggage = 0
)
# Fixed params
apollo_fixed_mnl = c()
# Validate inputs
apollo_inputs_mnl = apollo_validateInputs(apollo_beta  = apollo_beta_mnl,
                                          apollo_fixed = apollo_fixed_mnl,
                                          apollo_control = apollo_control_mnl)
# LL
apollo_probabilities_mnl=function(apollo_beta, apollo_inputs, functionality="estimate"){

  ### Function initialisation: do not change the following three commands
  ### Attach inputs and detach after function exit
  apollo_attach(apollo_beta, apollo_inputs)
  on.exit(apollo_detach(apollo_beta, apollo_inputs))

  ### Create list of probabilities P
  P = list()

  ### List of utilities: these must use the same names as in mnl_settings, order is irrelevant
  V = list()
  V[["train"]] =           b_time_mu * train_tt + b_cost_mu * train_co + b_he * train_he + b_age * age

  V[["sm"]]    = ASC_sm  + b_time_mu * sm_tt    + b_cost_mu * sm_co + b_ga * ga +  b_seats * sm_seats + b_he * sm_he

  V[["car"]]   = ASC_car + b_time_mu * car_tt   + b_cost_mu * car_co  + b_luggage * luggage


  ### Define settings for MNL model component
  mnl_settings = list(
    alternatives  = c(train = 1,
                      sm    = 2,
                      car   = 3),
    avail      = list(train = train_av ,
                      sm    = sm_av    ,
                      car   = car_av ) ,
    choiceVar     = choice,
    utilities     = V
  )

  ### Compute probabilities using MNL model
  P[["model"]] = apollo_mnl(mnl_settings, functionality)

  ### Take product across observation for same individual
  P = apollo_panelProd(P, apollo_inputs, functionality)

  ### Prepare and return outputs of function
  P = apollo_prepareProb(P, apollo_inputs, functionality)
  return(P)
}


model_mnl = apollo_estimate(apollo_beta = apollo_beta_mnl,
                            apollo_fixed = apollo_fixed_mnl,
                            apollo_probabilities = apollo_probabilities_mnl,
                            apollo_inputs = apollo_inputs_mnl )



mnl_estimates <- model_mnl$estimate
apollo_modelOutput(model_mnl)



### ################################ ###
####  MIXL:      ####
### ################################ ###



# Initialize code
# Set core controls
apollo_control_MIXL_H0 = list(
  modelName       = "MIXL",
  modelDescr      = "Mixed logit model on swissmetro",
  indivID         = "id",
  mixing          = TRUE,
  nCores          = 1,
  outputDirectory = NULL
)
# Vector of parameters, including any that are kept fixed in estimation
apollo_beta_MIXL_H0 = c(
  ASC_car = 0,
  ASC_sm = 0,

  b_time_mu    = 0,
  b_time_sd1    = 0,

  b_cost_mu    = 0,
  b_cost_sd1    = 0,

  b_he = 0,

  b_age  =0 ,
  b_ga = 0,
  b_seats = 0,
  b_luggage = 0
)
# Vector with names (in quotes)
apollo_fixed_MIXL_H0 = c()

# Using MNL estimates as starting values for the mixed logit.
apollo_beta_MIXL_H0[intersect(names(apollo_beta_MIXL_H0), names(mnl_estimates))] <- mnl_estimates




### Set parameters for generating draws
apollo_draws_MIXL_H0 = list(
  interDrawsType = "sobol",
  interNDraws    = 20,
  interUnifDraws = c(),
  interNormDraws = c("draws_cost","draws_time"),
  intraDrawsType = "halton",
  intraNDraws    = 0,
  intraUnifDraws = c(),
  intraNormDraws = c()
)

### Create random parameters
apollo_randCoeff_MIXL_H0 = function(apollo_beta, apollo_inputs){
  randcoeff = list()

  randcoeff[["b_time"]] = -exp( b_time_mu + b_time_sd1 * draws_time )
  randcoeff[["b_cost"]] = -exp( b_cost_mu + b_cost_sd1 * draws_cost )

  return(randcoeff)
}


apollo_inputs_MIXL_H0 = apollo_validateInputs(
  apollo_beta      = apollo_beta_MIXL_H0,
  apollo_fixed     = apollo_fixed_MIXL_H0,
  apollo_control   = apollo_control_MIXL_H0,
  apollo_draws     = apollo_draws_MIXL_H0 ,
  apollo_randCoeff = apollo_randCoeff_MIXL_H0

)

apollo_probabilities_MIXL_H0=function(apollo_beta, apollo_inputs, functionality="estimate"){

  ### Function initialisation: do not change the following three commands
  ### Attach inputs and detach after function exit
  apollo_attach(apollo_beta, apollo_inputs)
  on.exit(apollo_detach(apollo_beta, apollo_inputs))

  ### Create list of probabilities P
  P = list()


  ### List of utilities: these must use the same names as in mnl_settings, order is irrelevant
  V = list()
  V[["train"]] =           b_time * train_tt + b_cost * train_co + b_he * train_he + b_age * age

  V[["sm"]]    = ASC_sm  + b_time * sm_tt    + b_cost * sm_co + b_ga * ga +  b_seats * sm_seats + b_he * sm_he

  V[["car"]]   = ASC_car + b_time * car_tt   + b_cost * car_co  + b_luggage * luggage


  ### Define settings for MNL model component
  mnl_settings = list(
    alternatives  = c(train = 1,
                      sm    = 2,
                      car   = 3),
    avail      = list(train = train_av ,
                      sm    = sm_av    ,
                      car   = car_av ) ,
    choiceVar     = choice,
    utilities     = V
  )

  ### Compute probabilities using MNL model
  P[["model"]] = apollo_mnl(mnl_settings, functionality)

  ### Take product across observation for same individual
  P = apollo_panelProd(P, apollo_inputs, functionality)


  ### Average across inter-individual draws
  P = apollo_avgInterDraws(P, apollo_inputs, functionality)

  ### Prepare and return outputs of function
  P = apollo_prepareProb(P, apollo_inputs, functionality)
  return(P)
}

model_MIXL_H0 = apollo_estimate(
  apollo_beta          = apollo_beta_MIXL_H0,
  apollo_fixed         = apollo_fixed_MIXL_H0,
  apollo_probabilities = apollo_probabilities_MIXL_H0,
  apollo_inputs        = apollo_inputs_MIXL_H0 )

apollo_modelOutput(model_MIXL_H0)


model_MIXL_H0_conditionals = apollo_conditionals(model = model_MIXL_H0,
                                                 apollo_probabilities = apollo_probabilities_MIXL_H0,
                                                 apollo_inputs = apollo_inputs_MIXL_H0)


I hope you could help me with that. I assume there is a problem since I am not calling the apollo objects as usual. For example, I have multiple "apollo_beta" one for the MNL model and another for the MIXL model, so I assume this might be the source of the problem...

Thank you in advance for your time.

Best regards,

Álvaro

Re: Bug multi-core with modular code

Posted: 10 May 2022, 11:43
by stephanehess
Alvaro

to help us isolate the issue, could you try running it for a case where you actually call the vector apollo_beta in the main environment, rather than having the separate names?

Thanks

Stephane

Re: Bug multi-core with modular code

Posted: 10 May 2022, 15:28
by alvarogutyerrez
Hi Stephane,

Thank you for your quick reply.

Yes, indeed, this is exactly how I solved the problem. The program runs if the atomic vector "apollo_beta" is present in the Global Environment. However, this solution forces me to create several "apollo_beta" objects along the way, which, unfortunately, can mask bugs as well. This is why I suspect that `apollo_makeLogLike()` is searching for "apollo_beta" in the global environment instead of recognizing the function's argument. However, I haven't had time to scan the source code of the said function properly, so I am not very sure.

Best regards,

Álvaro

Re: Bug multi-core with modular code

Posted: 12 May 2022, 16:20
by dpalma
Hi Álvaro,

Thanks for pointing this out. Indeed it is a bug. apollo_estimate was using the apollo_beta in the global environment, as opposed to the one given to it as an argument. This will be fixed in v0.3.0 of Apollo. In the meantime, your manual fix should take care of the problem.

Note that in version v0.2.7 a similar issue is present for apollo_fixed. It will be solved in v0.3.0 as well.

Cheers
David