Important: Read this before posting to this forum

  1. This forum is for questions related to the use of Apollo. We will answer some general choice modelling questions too, where appropriate, and time permitting. We cannot answer questions about how to estimate choice models with other software packages.
  2. There is a very detailed manual for Apollo available at http://www.ApolloChoiceModelling.com/manual.html. This contains detailed descriptions of the various Apollo functions, and numerous examples are available at http://www.ApolloChoiceModelling.com/examples.html. In addition, help files are available for all functions, using e.g. ?apollo_mnl
  3. Before asking a question on the forum, users are kindly requested to follow these steps:
    1. Check that the same issue has not already been addressed in the forum - there is a search tool.
    2. Ensure that the correct syntax has been used. For any function, detailed instructions are available directly in Apollo, e.g. by using ?apollo_mnl for apollo_mnl
    3. Check the frequently asked questions section on the Apollo website, which discusses some common issues/failures. Please see http://www.apollochoicemodelling.com/faq.html
    4. Make sure that R is using the latest official release of Apollo.
  4. If the above steps do not resolve the issue, then users should follow these steps when posting a question:
    1. provide full details on the issue, including the entire code and output, including any error messages
    2. posts will not immediately appear on the forum, but will be checked by a moderator first. This may take a day or two at busy times. There is no need to submit the post multiple times.

Unable to compute analytical gradients

Ask questions about errors you encouunter. Please make sure to include full details about your model specifications, and ideally your model file.
Post Reply
gregorymacfarlane
Posts: 11
Joined: 29 Oct 2020, 23:11
Contact:

Unable to compute analytical gradients

Post by gregorymacfarlane »

I am estimating a multinomial logit model where observations have eight alternatives, and all are available. The data is available on GitHub https://github.com/urban-stack/kids-ind ... 17-all.rds, and the code to estimate the models is available on GitHub here: https://github.com/urban-stack/kids-ind ... nd3_cov1.R

When estimating this model, I encountered the warning

Code: Select all

Apollo was not able to compute analytical gradients for your model. This could be because you are using model components for which analytical gradients are not yet
  implemented, or because you coded your own model functions. If however you only used apollo_mnl, apollo_fmnl, apollo_normalDensity, apollo_ol or apollo_op then
  there could be another issue. You might want to ask for help in the Apollo forum (http://www.apollochoicemodelling.com/forum) on how to solve this issue. If you
  do, please post your code and data (if not confidential)
The resulting model then failed to calculate the Hessian, and returned no standard errors. It seems as though i am using a somewhat standard MNL, so I don't know what the difficulty could be. In fact, I estimated an MNL with almost exactly the same utility functions on the same dataset just weeks ago.
stephanehess
Site Admin
Posts: 998
Joined: 24 Apr 2020, 16:29

Re: Unable to compute analytical gradients

Post by stephanehess »

Greg

in relation to the analytic gradients, I suspect this is due to your use of lapply. Apollo needs to parse the utilities for taking derivatives, and that works when the user writes them out fully or if using a loop (see http://apollochoicemodelling.com/files/ ... e_coding.r) but likely not when using lapply

Regarding the Hessian, I ran your model using the development version and it converged fine and I got the Hessian too. Can you try with that (http://apollochoicemodelling.com/files/ ... e_coding.r)

Stephane
--------------------------------
Stephane Hess
www.stephanehess.me.uk
gregorymacfarlane
Posts: 11
Joined: 29 Oct 2020, 23:11
Contact:

Re: Unable to compute analytical gradients

Post by gregorymacfarlane »

Thanks for your response. Interesting about the utility functions in lapply vs for. I just did a little test with the same model, and see that if I do

Code: Select all

 V = list()
  ### compute class-specific utilities
  V[["walk_alone"  ]] = ...
Then the hessian is singular and no standard errors are returned.

Code: Select all

Estimates:
                    Estimate        s.e.   t.rat.(0)    Rob.s.e. Rob.t.rat.(0)
asc_dp               0.23280          NA          NA          NA            NA
asc_do              -0.52663          NA          NA          NA            NA
asc_wa               0.00000          NA          NA          NA            NA 
...
But if I use a single bracket for the utility functions

Code: Select all

V["walk_alone"] = ...
then I get the warning about not being able to calculate an analytical gradient. BUT, the model returns the same coefficients but WITH some standard errors.

Code: Select all

   
                     Estimate        s.e.   t.rat.(0)    Rob.s.e. Rob.t.rat.(0)
asc_dp               0.23280       45.37    0.005131         NaN           NaN
asc_do              -0.52664       80.72   -0.006525    0.026598      -19.7997
asc_wa               0.00000          NA          NA          NA            NA
asc_wp               0.06979       62.82    0.001111         NaN           NaN
I'm aware of the difference between [[ and [ in R, so I can understand why things would be different, but I guess I don't know that I should trust these standard errors. It's an interesting problem.

As to your direct suggestion about using the development version, I am unable to compile the development version from source, I believe it has something do to with my fortran compiler (see below). Do you have binaries of the development version for Mac?

Code: Select all

In file included from apolloDFT.cpp:5:
In file included from /Library/Frameworks/R.framework/Versions/4.2-arm64/Resources/library/RcppEigen/include/RcppEigen.h:25:
In file included from /Library/Frameworks/R.framework/Versions/4.2-arm64/Resources/library/RcppEigen/include/RcppEigenForward.h:40:
In file included from /Library/Frameworks/R.framework/Versions/4.2-arm64/Resources/library/RcppEigen/include/unsupported/Eigen/SparseExtra:51:
/Library/Frameworks/R.framework/Versions/4.2-arm64/Resources/library/RcppEigen/include/Eigen/src/Core/util/ReenableStupidWarnings.h:14:30: warning: pragma diagnostic pop could not pop, no matching push [-Wunknown-pragmas]
    #pragma clang diagnostic pop
                             ^
23 warnings generated.
/opt/homebrew/opt/llvm/bin/clang++ -dynamiclib -Wl,-headerpad_max_install_names -undefined dynamic_lookup -single_module -multiply_defined suppress -L/Library/Frameworks/R.framework/Resources/lib -L/opt/homebrew/opt/gettext/lib -L/opt/homebrew/opt/llvm/lib -Wl,-rpath,/opt/homebrew/opt/llvm/lib -o apollo.so RcppExports.o apolloDFT.o -L/Library/Frameworks/R.framework/Resources/lib -lRlapack -L/Library/Frameworks/R.framework/Resources/lib -lRblas -L/opt/R/arm64/gfortran/lib/gcc/aarch64-apple-darwin20.6.0/12.0.1 -L/opt/R/arm64/gfortran/lib -lgfortran -lemutls_w -lquadmath -F/Library/Frameworks/R.framework/.. -framework R -Wl,-framework -Wl,CoreFoundation
ld: warning: directory not found for option '-L/opt/R/arm64/gfortran/lib/gcc/aarch64-apple-darwin20.6.0/12.0.1'
ld: warning: directory not found for option '-L/opt/R/arm64/gfortran/lib'
ld: library not found for -lgfortran
clang-16: error: linker command failed with exit code 1 (use -v to see invocation)
make: *** [apollo.so] Error 1
ERROR: compilation failed for package ‘apollo’
* removing ‘/Library/Frameworks/R.framework/Versions/4.2-arm64/Resources/library/apollo’
* restoring previous ‘/Library/Frameworks/R.framework/Versions/4.2-arm64/Resources/library/apollo’
Warning in install.packages :
  installation of package ‘/Users/gregmacfarlane/Downloads/apollo_0.3.0.tar.gz’ had non-zero exit status
stephanehess
Site Admin
Posts: 998
Joined: 24 Apr 2020, 16:29

Re: Unable to compute analytical gradients

Post by stephanehess »

Greg

I'm happy to look into this further for you. can you send me the updated model files too, please?

Regarding a binary, are you on an Intel chip or an Apple Silicon chip?

Stephane
--------------------------------
Stephane Hess
www.stephanehess.me.uk
gregorymacfarlane
Posts: 11
Joined: 29 Oct 2020, 23:11
Contact:

Re: Unable to compute analytical gradients

Post by gregorymacfarlane »

I have pushed updates to the code and model output files at the GitHub link in the OP.
stephanehess
Site Admin
Posts: 998
Joined: 24 Apr 2020, 16:29

Re: Unable to compute analytical gradients

Post by stephanehess »

Hi

I had a look at the code.

First, there was a simple mistake in the model. The parameters were not multiplying any attributes, e.g. beta_age_wa should be multiplying age, etc. This was the cause for the identification issue.

Code: Select all

  V[["walk_alone"  ]] = asc_wa + beta_age_wa+ beta_female_wa + beta_distance_wa + beta_density_wa + beta_school_wa + beta_income_wa
Second, I optimised the code so that you can still produce the utilities (and also the betas) iteratively, and it all runs fine, including with analytical gradients. Please see below:

Code: Select all

rm(list = ls())
library(apollo)
library(readr)
library(here)

# Initialisation ===========================
# 
# 
### Initialise code
apollo_initialise()
### Set core controls
apollo_control = list(
  modelName = "mnl_ind3_01",
  modelDescr = "MNL on kid trip data - simplified independence with first covariates",
  indivID = "id"
)

## class membership

# Data =====================================
# Read in data and do basic manipulations for coherence
# 
data <- readr::read_rds(("usa-2017-all.rds"))
database <- data |> 
  dplyr::transmute(
    id = 1:dplyr::n(),
    # set up alternatives as named vector 
    choice = dplyr::case_when(
      mode_ind_3 == 720 ~ "drive_parent",
      mode_ind_3 == 730 ~ "drive_others",
      mode_ind_3 == 810 ~ "walk_alone",
      mode_ind_3 == 820 ~ "walk_parent",
      mode_ind_3 == 830 ~ "walk_others",
      mode_ind_3 == 910 ~ "bike_alone",
      mode_ind_3 == 920 ~ "bike_parent",
      mode_ind_3 == 930 ~ "bike_others",
      TRUE ~ as.character(NA)
    ),
    income = income_k, veh_per_driver, n_adults, non_work_mom, non_work_dad,
    age, female, distance, school, density
  ) |> 
  dplyr::filter(!is.na(choice))


# Name settings

J = 8
alts = c("walk_alone","walk_others","walk_parent","bike_alone","bike_others","bike_parent",
         "drive_others","drive_parent")
alt_short = c("wa","wo","wp","ba","bo","bp","do","dp")
attributes = c("age","female","distance","density","school","income")


# Parameters ===============================
# Create a list of all the parameters we need to estimate
#  
apollo_beta=setNames(rep(0,J*(length(attributes)+1)),
                     c(paste0(rep("asc_",J),alt_short),paste0(rep(paste0("beta_",attributes),each=J),"_",rep(alt_short,length(attributes)))))


apollo_fixed = names(apollo_beta)[grepl("wa", names(apollo_beta))]

apollo_fixed=unique(apollo_fixed)

# Model Definition =========================

# verify all the parameters and data are correct and expected
apollo_inputs = apollo_validateInputs()
# include additional elements in apollo_inputs
apollo_inputs$J = 8
apollo_inputs$alts = alts
apollo_inputs$alt_short = alt_short
apollo_inputs$attributes = attributes

# Define model probabilities and likelihood
apollo_probabilities = function(apollo_beta, apollo_inputs, functionality = "estimate") {
  
  ### Attach inputs and detach after function exit
  apollo_attach(apollo_beta, apollo_inputs)
  on.exit(apollo_detach(apollo_beta, apollo_inputs))
  ### Define settings for MNL
  mnl_settings = list(
    alternatives = setNames(alts,alts),
    choiceVar    = choice
  )
  
  
  ### Create list of probabilities P
  P = list()
  
  V = list()
  ### compute class-specific utilities
  for(j in 1:apollo_inputs$J){
    V[[apollo_inputs$alts[j]]] = get(paste0("asc_",apollo_inputs$alt_short[[j]]))
    for(a in apollo_inputs$attributes) V[[apollo_inputs$alts[j]]]=V[[apollo_inputs$alts[j]]]+ get(paste0("beta_",a,"_",apollo_inputs$alt_short[[j]])) * get(a)
  }
  
    
  mnl_settings$utilities <- V
    ### Compute within-class choice probabilities using MNL model
  P[["model"]] = apollo_mnl(mnl_settings, functionality)
  
  ### Prepare and return outputs of function
  P = apollo_prepareProb(P, apollo_inputs, functionality)
  return(P)
}



# ################################################################# #
#### MODEL ESTIMATION                                            ####
# ################################################################# #
### Estimate model
model = apollo_estimate(apollo_beta, apollo_fixed, 
                        apollo_probabilities, apollo_inputs)

### Show output in screen
apollo_modelOutput(model)

### Save output to file(s)
apollo_saveOutput(model)
--------------------------------
Stephane Hess
www.stephanehess.me.uk
gregorymacfarlane
Posts: 11
Joined: 29 Oct 2020, 23:11
Contact:

Re: Unable to compute analytical gradients

Post by gregorymacfarlane »

Ah, well that was silly. I don't think I had that issue with my earlier code. But it works now. Thanks for being the proofreader here.
shivamyadav0568
Posts: 1
Joined: 14 Dec 2023, 18:21

Re: Unable to compute analytical gradients

Post by shivamyadav0568 »

I am facing error that, Apollo was not able to compute analytical gradients for your model. This could be because you are using model components for which analytical gradients are not yet implemented, or because you coded your own model functions. If however you only used apollo_mnl, apollo_fmnl, apollo_normalDensity, apollo_ol or apollo_op, then there could be another issue. In output it provides me the coefficients, BHHH standard error and final LL but i want standard error and also other LL value. Note: I have used for loop. Could you please help me in this. Here is the link for code.
https://github.com/shivam0568/MNL-Airli ... n/FInal2.R
Here is the link for data,
https://github.com/shivam0568/MNL-Airli ... 20data.rar
dpalma
Posts: 190
Joined: 24 Apr 2020, 17:54

Re: Unable to compute analytical gradients

Post by dpalma »

Hi,

Sadly, Apollo cannot handle the use of the parse function and use analytical gradient. So if you use "parse", then you are forced to use numerical gradients.

Looking at your code, you could avoid using "parse" by writing the utility function as follows:

Code: Select all

V[[alt]] = get(paste0("asc_", alt)) + B_fAirline*get(paste0("fAirline_",alt)) + ...
You can look at example MNL_iterative_coding for more detail.

Best
David
Post Reply