Important: Read this before posting to this forum

  1. This forum is for questions related to the use of Apollo. We will answer some general choice modelling questions too, where appropriate, and time permitting. We cannot answer questions about how to estimate choice models with other software packages.
  2. There is a very detailed manual for Apollo available at http://www.ApolloChoiceModelling.com/manual.html. This contains detailed descriptions of the various Apollo functions, and numerous examples are available at http://www.ApolloChoiceModelling.com/examples.html. In addition, help files are available for all functions, using e.g. ?apollo_mnl
  3. Before asking a question on the forum, users are kindly requested to follow these steps:
    1. Check that the same issue has not already been addressed in the forum - there is a search tool.
    2. Ensure that the correct syntax has been used. For any function, detailed instructions are available directly in Apollo, e.g. by using ?apollo_mnl for apollo_mnl
    3. Check the frequently asked questions section on the Apollo website, which discusses some common issues/failures. Please see http://www.apollochoicemodelling.com/faq.html
    4. Make sure that R is using the latest official release of Apollo.
  4. If the above steps do not resolve the issue, then users should follow these steps when posting a question:
    1. provide full details on the issue, including the entire code and output, including any error messages
    2. posts will not immediately appear on the forum, but will be checked by a moderator first. This may take a day or two at busy times. There is no need to submit the post multiple times.

output changed after optimisation

Ask questions about errors you encouunter. Please make sure to include full details about your model specifications, and ideally your model file.
Post Reply
Padma
Posts: 3
Joined: 01 Mar 2023, 06:46

output changed after optimisation

Post by Padma »

hi
i have got the following output,


The output of 'apollo_probabilities' has changed after the optimisation of the code carried out by Apollo. This indicates a problem, and
the unoptimised and unscaled version will be used instead. Please contact the developers for assistance!

kindly help on the notification

regards
Padma
stephanehess
Site Admin
Posts: 974
Joined: 24 Apr 2020, 16:29

Re: output changed after optimisation

Post by stephanehess »

Hi

could you please share your code with us to have a look?

Thanks

Stephane
--------------------------------
Stephane Hess
www.stephanehess.me.uk
Qinnnnn
Posts: 1
Joined: 03 Apr 2023, 14:43

Re: output changed after optimisation

Post by Qinnnnn »

Hello, I got the same output message.
Below is the skeleton of my code. (I omitted some parts to show it simply). Please let me know if you need the source script.

Thanks in advance ;) .


### Load Apollo library
library(apollo)
library(dplyr)
library(tidyverse)

### Initialise code
apollo_initialise()

### Set core controls
apollo_control = list(
modelName = scenario,
modelDescr = "MNL all mode choice",
panelData = FALSE,
indivID = "t.ID",
nCores = 16
)

apollo_fixed = c(names(apollo_beta)[c(as.numeric(beta_gv_matrix) == 1,unique(gsub("^.*?\\.","",beta_av))==FALSE)])

# ################################################################# #
#### GROUP AND VALIDATE INPUTS ####
# ################################################################# #
apollo_inputs = apollo_validateInputs()

# ################################################################# #
#### DEFINE MODEL AND LIKELIHOOD FUNCTION ####
# ################################################################# #

apollo_probabilities=function(apollo_beta, apollo_inputs, functionality="estimate"){

### Attach inputs and detach after function exit
apollo_attach(apollo_beta, apollo_inputs)
on.exit(apollo_detach(apollo_beta, apollo_inputs))

### Create list of probabilities P
P = list()

### List of utilities: these must use the same names as in mnl_settings, order is irrelevant
V = list()

#####Utility functions are omitted here#####

nlNests = list(root=1)

### Specify tree structure for NL model
nlStructure = list()
nlStructure[["root"]] = c("carD","carP","pt","walk","cycle","taxi")


### Alternatives
alternatives = names(V)
names(alternatives) = alternatives

### Availability
avail = list(carD=av_carD, carP=av_carP,pt=av_pt, cycle=av_cycle,walk=av_walk,taxi=av_taxi)


### Define settings for MNL model component
nl_settings = list(
alternatives = alternatives,
avail = avail,
choiceVar = t.mode_MITO_grouped,
V = V,
nlNests = nlNests,
nlStructure = nlStructure
)

### Compute probabilities using MNL model
P[['model']] = apollo_nl(nl_settings, functionality)

### Prepare and return outputs of function
P = apollo_prepareProb(P, apollo_inputs, functionality)

#P = apollo_weighting(P, apollo_inputs, functionality)
return(P)
}

# ################################################################# #
#### MODEL ESTIMATION ####
# ################################################################# #

model = apollo_estimate(apollo_beta, apollo_fixed, apollo_probabilities, apollo_inputs,
estimate_settings=list(writeIter=FALSE))
Padma
Posts: 3
Joined: 01 Mar 2023, 06:46

Re: output changed after optimisation

Post by Padma »

Hi

Kindly find attached the code

# ####################################################### #
#### 1. Definition of core settings
# ####################################################### #

### Clear memory
rm(list = ls())

### Load libraries
library(apollo)
#> Apollo 0.2.8
#> http://www.ApolloChoiceModelling.com
#> See url for a detailed manual, examples and a user forum.
#> Sign up to the user forum to receive updates on new releases.

### Initialise code
apollo_initialise()

### Set core controls
apollo_control = list(
modelName ="MNL6_final_PAPER",
modelDescr ="Simple MNL model on mode choice SP data",
indivID ="ID",
panelData=FALSE,
outputDirectory = "output"
)

# ####################################################### #
#### 2. Data loading ####
# ####################################################### #


database = read.csv("apollo5.csv",header=TRUE)
#database$EDU = as.factor(database$EDU)
#database$age_1 = as.factor(database$age_1)

choiceAnalysis_settings <- list(
alternatives = c(car=1, tw=2, h=3, bus=4, ipt=5, nmt=6, van=7, TM=8),
avail = list(car=database$CAR_AV, tw=database$TW_AV, h=database$H_AV, bus=database$BUS_AV, ipt=database$IPT_AV, nmt=database$NMT_AV, van=database$VAN_AV, TM=database$TM_AV),
choiceVar = database$choice,
explanators = database[,c("VEH","age_1")]

)

### Run function to analyse choice data
apollo_choiceAnalysis(choiceAnalysis_settings, apollo_control, database)


# ####################################################### #
#### 3. Parameter definition ####
# ####################################################### #

### Vector of parameters, including any that are kept fixed
### during estimation
apollo_beta=c(asc_car = 0,
asc_tw=0,
asc_h=0,
asc_van=0,
asc_bus = 0,
asc_ipt = 0,
asc_tm=0,
asc_nmt = 0,
b_t=0,
age1=0,
edu1=0,
cc=0,
b_cost1=0
)




### Vector with names war(in quotes) of parameters to be
### kept fixed at their starting value in apollo_beta.
### Use apollo_beta_fixed = c() for no fixed parameters.
apollo_fixed = c("asc_nmt")

# ####################################################### #
#### 4. Input validation ####
# ####################################################### #

apollo_inputs = apollo_validateInputs()


# ####################################################### #
#### 5. Likelihood definition ####
# ####################################################### #

apollo_probabilities=function(apollo_beta, apollo_inputs,
functionality="output"){

### Attach inputs and detach after function exit
apollo_attach(apollo_beta, apollo_inputs)
on.exit(apollo_detach(apollo_beta, apollo_inputs))

### Create list of probabilities P
P = list()

### List of utilities: these must use the same names as ### in mnl_settings, order is irrelevant.
V = list()


V[["car"]] = asc_car + b_t *(TTC) + b_cost1*TCC+cc*(TCC*TTC)+age1*age_1+edu1*VEH
V[["tw"]] = asc_tw + b_t *(TTTW) + b_cost1*TCTW+age1*age_1+edu1*VEH+cc*(TCTW*TTTW)
V[["h"]] = asc_h + b_t *(TTH) + b_cost1*TCH+age1*age_1+edu1*VEH+cc*(TCH*TTH)
V[["bus"]] = asc_bus + b_t *(TTB) + b_cost1*TCB+age1*age_1+edu1*VEH+cc*(TCB*TTB)
V[["ipt"]] = asc_ipt + b_t *(TTIPT) + b_cost1*TCIPT+age1*age_1+edu1*VEH+cc*(TCIPT*TTIPT)
V[["van"]] = asc_van + b_t *(TTVAN) + b_cost1*TCVAN+age1*age_1+edu1*VEH+cc*(TCVAN*TTVAN)
V[["nmt"]]= asc_nmt+ b_t*(TTNMT)
V[["TM"]]= asc_tm+ b_t*(TTTM)+age1*age_1+edu1*VEH+ b_cost1*TCTM+cc*(TCTM*TTTM)



#V[['car']] = asc_car
#V[['tw']] = asc_tw
#V[['h']] = asc_h
#V[['bus']] = asc_bus
#V[['ipt']] = asc_ipt
#V[['nmt']]= asc_nmt
#V[['van']]= asc_van
#V[['TM']]= asc_tm




### Define settings for MNL model component
mnl_settings = list(
alternatives = c(car=1, tw=2, h=3, bus=4, ipt=5, nmt=6, van=7, TM=8),
avail = list(car=CAR_AV, tw=TW_AV, h=H_AV, bus=BUS_AV, ipt=IPT_AV, nmt=NMT_AV, van=VAN_AV, TM=TM_AV),
choiceVar = choice,
utilities = V
)


P[["model"]] = apollo_mnl(mnl_settings, functionality)

### Take product across observation for same individual
## P = apollo_panelProd(P, apollo_inputs, functionality)

### Prepare and return outputs of function
P = apollo_prepareProb(P, apollo_inputs, functionality)
return(P)
}

# ####################################################### #
#### 6. Model estimation and reporting ####
# ####################################################### #

model = apollo_estimate(apollo_beta, apollo_fixed,
apollo_probabilities,
apollo_inputs,
list(writeIter=FALSE))


modelOutput_Liste = list(printChange=TRUE, printClassical=TRUE, printCorr=TRUE,
printCovar=TRUE, printDataReport=TRUE, printFixed=TRUE,
printFunctions=TRUE, printModelStructure=TRUE,
printOutliers=TRUE, printPVal=2, printT1=TRUE)

apollo_modelOutput(model,modelOutput_settings=modelOutput_Liste)
apollo_saveOutput(model,modelOutput_Liste)




### Use the estimated model to make predictions
predictions_base = apollo_prediction(model,
apollo_probabilities,
apollo_inputs)


database$TCC= 1.01*database$TCC
database$LNTCC=log(database$TCC)
apollo_inputs = apollo_validateInputs()
#> Several observations per individual detected based on the value of ID.
#> Setting panelData in apollo_control set to TRUE.
#> All checks on apollo_control completed.
#> All checks on database completed.
predictions_new = apollo_prediction(model,
apollo_probabilities,
apollo_inputs)


### Compare predictions
change=(predictions_new-predictions_base)/predictions_base
### Not interested in chosen alternative now,
### so drop last column
change=change[,-ncol(change)]
### Summary of changes (possible presence of NAs due to
### unavailable alternatives)
summary(change)
### Compute own elasticity for ipt:
log(sum(predictions_new[,3])/sum(predictions_base[,3]))/log(1.01)

# ----------------------------------------------------------------- #
#---- RECOVERY OF SHARES FOR ALTERNATIVES IN DATABASE ----
# ----------------------------------------------------------------- #

sharesTest_settings = list()
sharesTest_settings[["alternatives"]] = c(car=1, tw=2, h=3, bus=4, ipt=5, nmt=6, van=7, TM=8)
sharesTest_settings[["choiceVar"]] = database$choice
#sharesTest_settings[["subsamples"]] = list(business=(database$business==1),
#leisure=(database$business==0))

apollo_sharesTest(model, apollo_probabilities, apollo_inputs, sharesTest_settings)



deltaMethod_settings=list(expression=c(VTT_car_min1="(b_t/b_cost1)",
VTT_ipt_min1="(b_t/b_cost1)"
))
apollo_deltaMethod(model, deltaMethod_settings)
stephanehess
Site Admin
Posts: 974
Joined: 24 Apr 2020, 16:29

Re: output changed after optimisation

Post by stephanehess »

Hi

apologies for the slow response, the forum was offline due to a technical issue.

It's not clear immediately what's wrong. Would you be able to share your code and data with me via e-mail and I'll have a look

Stephane
--------------------------------
Stephane Hess
www.stephanehess.me.uk
stephanehess
Site Admin
Posts: 974
Joined: 24 Apr 2020, 16:29

Re: output changed after optimisation

Post by stephanehess »

Hi

apologies for the slow response, the forum was offline due to a technical issue.

It's not clear immediately what's wrong. Would you be able to share your code and data with me via e-mail and I'll have a look

Stephane
Qinnnnn wrote: 03 Apr 2023, 15:25 Hello, I got the same output message.
Below is the skeleton of my code. (I omitted some parts to show it simply). Please let me know if you need the source script.

Thanks in advance ;) .


### Load Apollo library
library(apollo)
library(dplyr)
library(tidyverse)

### Initialise code
apollo_initialise()

### Set core controls
apollo_control = list(
modelName = scenario,
modelDescr = "MNL all mode choice",
panelData = FALSE,
indivID = "t.ID",
nCores = 16
)

apollo_fixed = c(names(apollo_beta)[c(as.numeric(beta_gv_matrix) == 1,unique(gsub("^.*?\\.","",beta_av))==FALSE)])

# ################################################################# #
#### GROUP AND VALIDATE INPUTS ####
# ################################################################# #
apollo_inputs = apollo_validateInputs()

# ################################################################# #
#### DEFINE MODEL AND LIKELIHOOD FUNCTION ####
# ################################################################# #

apollo_probabilities=function(apollo_beta, apollo_inputs, functionality="estimate"){

### Attach inputs and detach after function exit
apollo_attach(apollo_beta, apollo_inputs)
on.exit(apollo_detach(apollo_beta, apollo_inputs))

### Create list of probabilities P
P = list()

### List of utilities: these must use the same names as in mnl_settings, order is irrelevant
V = list()

#####Utility functions are omitted here#####

nlNests = list(root=1)

### Specify tree structure for NL model
nlStructure = list()
nlStructure[["root"]] = c("carD","carP","pt","walk","cycle","taxi")


### Alternatives
alternatives = names(V)
names(alternatives) = alternatives

### Availability
avail = list(carD=av_carD, carP=av_carP,pt=av_pt, cycle=av_cycle,walk=av_walk,taxi=av_taxi)


### Define settings for MNL model component
nl_settings = list(
alternatives = alternatives,
avail = avail,
choiceVar = t.mode_MITO_grouped,
V = V,
nlNests = nlNests,
nlStructure = nlStructure
)

### Compute probabilities using MNL model
P[['model']] = apollo_nl(nl_settings, functionality)

### Prepare and return outputs of function
P = apollo_prepareProb(P, apollo_inputs, functionality)

#P = apollo_weighting(P, apollo_inputs, functionality)
return(P)
}

# ################################################################# #
#### MODEL ESTIMATION ####
# ################################################################# #

model = apollo_estimate(apollo_beta, apollo_fixed, apollo_probabilities, apollo_inputs,
estimate_settings=list(writeIter=FALSE))
--------------------------------
Stephane Hess
www.stephanehess.me.uk
BTHopkins
Posts: 4
Joined: 25 Jul 2023, 21:03

Re: output changed after optimisation

Post by BTHopkins »

Hi,

I am having this error as well with version 0.2.8. Any advice would be appreciated:

Code: Select all

# ################################################################# #
#### LOAD LIBRARY AND DEFINE CORE SETTINGS                       ####
# ################################################################# #

### Clear memory
rm(list = ls())

### Set the working directory
setwd("<...>")

### Load Apollo & Haven library
library(apollo)
library(tidyverse)
library(haven)

### Initialise code
apollo_initialise()

### Set core controls
apollo_control = list(
  modelName       = "<...>",
  modelDescr      = "<...>",
  indivID         = "ID", 
  outputDirectory = "demand_estimates",
  nCores          = 7,
  calculateLLC    = FALSE
)

# ################################################################# #
#### LOAD DATA AND APPLY ANY TRANSFORMATIONS                     ####
# ################################################################# #

## Load the SAS dataset and convert it to a dataframe
sas_data <- 
  read_sas("<...>") |> 
  filter(area == "<...>" & year == <...>) |> 
  select(
    ID, choice, 
    starts_with("net_premium_pmpm_"), # Net premiums per member per month
    starts_with("av_"),               # Actuarial value
    starts_with("avail_")             # Availability
  ) |> 
  mutate(
    across(starts_with("avail"), ~ as.logical(.x))
  )

sas_data[] <- lapply(sas_data, function(x) {attributes(x) <- NULL;x})
database <- as.data.frame(sas_data)
datas
## Identify the maximum number of alternatives in a year & rating area
J <- sas_data |> 
  select(starts_with("net_premium_pmpm")) |> 
  ls() |> 
  length()

# ################################################################# #
#### DEFINE MODEL PARAMETERS                                     ####
# ################################################################# #

### Vector of parameters, including any that are kept fixed in estimation
apollo_beta = c(
  b_net_premium_pmpm = 0,
  b_av = 0
)

### Vector with names (in quotes) of parameters to be kept fixed at their 
  # starting value in apollo_beta, use apollo_beta_fixed = c() if none
apollo_fixed = c()

# ################################################################# #
#### GROUP AND VALIDATE INPUTS                                   ####
# ################################################################# #

apollo_inputs   = apollo_validateInputs()
apollo_inputs$J = J # need to retain J (number of alternatives) for use
                    # inside apollo_probabilities

# ################################################################# #
#### DEFINE MODEL AND LIKELIHOOD FUNCTION                        ####
# ################################################################# #

apollo_probabilities = function(apollo_beta, apollo_inputs, 
                               functionality="estimate"){

  ### Attach inputs and detach after function exit
  apollo_attach(apollo_beta, apollo_inputs)
  on.exit(apollo_detach(apollo_beta, apollo_inputs))
  
  ### Create list of probabilities P
  P = list()
  
  ### List of utilities: these must use the same names as in mnl_settings, order is irrelevant
  V = list()
  for(j_short in 1:apollo_inputs$J){
    j <- formatC(j_short, width = 4, flag = "0", format = "d")
    V[[paste0("alt_",j)]] = 0
    V[[paste0("alt_",j)]] = V[[paste0("alt_",j)]] +
      b_net_premium_pmpm * get(paste0("net_premium_pmpm_", j)) + 
      b_av * get(paste0("av_", j))
  }
  
  ### Define settings for MNL model component
  formatted_alternatives <- formatC(1:apollo_inputs$J, width = 4, flag = "0", format = "d")
  mnl_settings = list(
    alternatives  = setNames(formatted_alternatives, names(V)), 
    avail         = setNames(apollo_inputs$database[,paste0("avail_",formatted_alternatives)], names(V)), 
    choiceVar     = choice,
    utilities     = V
  )
  
  ### Compute probabilities using MNL model
  P[["model"]] = apollo_mnl(mnl_settings, functionality)
  
  ### Prepare and return outputs of function
  P = apollo_prepareProb(P, apollo_inputs, functionality)
  return(P)
}

# ################################################################# #
#### MODEL ESTIMATION                                            ####
# ################################################################# #

model = apollo_estimate(apollo_beta, apollo_fixed, apollo_probabilities, apollo_inputs)

# ################################################################# #
#### MODEL OUTPUTS                                               ####
# ################################################################# #

# ----------------------------------------------------------------- #
#---- FORMATTED OUTPUT (TO SCREEN)                               ----
# ----------------------------------------------------------------- #

apollo_modelOutput(model)

# ----------------------------------------------------------------- #
#---- FORMATTED OUTPUT (TO FILE, using model name)               ----
# ----------------------------------------------------------------- #

apollo_saveOutput(model)
stephanehess
Site Admin
Posts: 974
Joined: 24 Apr 2020, 16:29

Re: output changed after optimisation

Post by stephanehess »

Hi

the newest version is 0.2.9.

If it fails with that too, would you be able to share your data and code with us outside the forum and we can investigate?

Thanks

Stephane
--------------------------------
Stephane Hess
www.stephanehess.me.uk
BTHopkins
Posts: 4
Joined: 25 Jul 2023, 21:03

Re: output changed after optimisation

Post by BTHopkins »

Thank you Stephanie. I have requested the package be updated and will follow up if I still have this error after.

By the way, I noticed that the installation instructions in the latest version of the manual refer to 0.2.8.
dpalma
Posts: 190
Joined: 24 Apr 2020, 17:54

Re: output changed after optimisation

Post by dpalma »

Hi,

As of today, the current version is v0.3.0. Y can update it using the following line of code:

Code: Select all

install.packages("apollo")
The manual needs to be updated to the latest version, but it will be soon. Anyhow, the instructions to install (or update) Apollo remain the same, regardless what the latest version is.

Cheers
David
Post Reply