Page 1 of 1

Error in model predictions

Posted: 15 Nov 2022, 12:49
by Shan
Dear Sir,

I hope that you are doing well. I tried to generate model predictions from a simple MNL model. However, an error was reported indicating a non-numeric argument to the binary operator for the code: change=(predictions_new-predictions_base)/predictions_base. I compared my codes for model predictions with those in the Apollo manual but couldn't figure out what was wrong. Here are the codes for my model predictions:
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------
### Use the estimated model to make predictions
predictions_base = apollo_prediction(model,apollo_probabilities,apollo_inputs,prediction_settings=list(runs=30))

### Now imagine the fuel cost for car increases by 1%
database$fuelcost_car=1.01*database$fuelcost_car

### Rerun prediction with the new data
apollo_inputs=apollo_validateInputs()
predictions_new= predictions_new = apollo_prediction(model, apollo_probabilities, apollo_inputs)

### Return to original data
database$fuelcost_car = 1/1.01*database$fuelcost_car
apollo_inputs = apollo_validateInputs()

### Work with predictions at estimates
predictions_base=predictions_base[["at_estimates"]]

### Compute the change in probabilities
change=(predictions_new-predictions_base)/predictions_base

### Not interested in chosen alternative now, so drop the last column
change=change[,-ncol(change)]

### First two columns (change in ID and task) are also not needed
change=change[,-c(1,2)]

### Look at the first individual
change[1:9,]
### And for person 9
change[74:82,]

### Summary of changes (possible presence of NAs for unavailable alternatives)
summary(change)

### Look at mean changes for subsets of the data, ignoring NAs
colMeans(change,na.rm=TRUE)
colMeans(subset(change,database$female==1),na.rm=TRUE)
colMeans(subset(change,database$female==0),na.rm=TRUE)
colMeans(subset(change,(database$income<quantile(database$income,0.25))),na.rm=TRUE)
colMeans(subset(change,(database$income>=quantile(database$income,0.25))|(database$income<=quantile(database$income,0.75))),na.rm=TRUE)
colMeans(subset(change,(database$income>quantile(database$income,0.75))),na.rm=TRUE)

### Computing elasticities
### Own elasticity for car(3:alternatives=1;4:alternatives=2;5:alternatives=3):
log(sum(predictions_new[,3])/sum(predictions_base[,3]))/log(1.1)
### Cross-elasticities for other modes
log(sum(predictions_new[,4])/sum(predictions_base[,4]))/log(1.1)
log(sum(predictions_new[,5])/sum(predictions_base[,5]))/log(1.1)
----------------------------------------------------------------------------------------------------------------------------------------------------------------------
Kind regards,
Shan

Re: Error in model predictions

Posted: 25 Nov 2022, 13:23
by stephanehess
Hi, can you show us the whole output, please?

Re: Error in model predictions

Posted: 25 Nov 2022, 13:48
by Shan
Hi Stephane,
Many thanks for your reply. Here are the outputs I got:

Aggregated prediction
at MLE Sampled mean Sampled std.dev. Quantile 0.025 Quantile 0.972
car 2425 2439 110.09 2267 2651
bus 1363 1361 73.23 1252 1483
metro 2341 2329 85.23 2164 1483
Average prediction
at MLE Sampled mean Sampled std.dev. Quantile 0.025 Quantile 0.972
car 0.3957 0.3979 0.01796 0.3699 0.4325
bus 0.2224 0.2220 0.01195 0.2043 0.2420
metro 0.3820 0.3801 0.01391 0.3531 0.3965
The output from apollo_prediction is a list with two elements: a data.frame containing the predictions at the estimated values, and an array with predictions for different values of the parameters drawn from their asymptotic distribution.

Prediction at model estimates
car bus metro
Aggregate 2418.29 1365.31 2345.40
Average 0.39 0.22 0.38
The output from apollo_prediction is a matrix containing the predictions at the estimated values.

Regards,
Shan

Re: Error in model predictions

Posted: 25 Nov 2022, 13:49
by stephanehess
Shan

we need to see the actual error...

Stephane

Re: Error in model predictions

Posted: 25 Nov 2022, 13:58
by Shan
I am so sorry. Do you mean the error reported in R? It is an Error in FUN (Left, right): non-numeric argument to binary operator. I guess it is noted for the code: change=(predictions_new-predictions_base)/predictions_base.

Re: Error in model predictions

Posted: 25 Nov 2022, 17:10
by dpalma
Hi,

Please try adding these two lines before creating "change":

Code: Select all

predictions_base = predictions_base[,-(1:2)]
predictions_new  = predictions_new[,-(1:2)]
That should remove the first two columns of the prediction, which are the individual ID and the observation number. I am guessing that your individual ID's are non-numeric (maybe names or strings with letters and numbers), which cause an error when trying to divide them.

Your code should look something like the following:

Code: Select all

### Use the estimated model to make predictions
predictions_base = apollo_prediction(model,apollo_probabilities,apollo_inputs,prediction_settings=list(runs=30))

### Now imagine the fuel cost for car increases by 1%
database$fuelcost_car=1.01*database$fuelcost_car

### Rerun prediction with the new data
apollo_inputs=apollo_validateInputs()
predictions_new= predictions_new = apollo_prediction(model, apollo_probabilities, apollo_inputs)

### Return to original data
database$fuelcost_car = 1/1.01*database$fuelcost_car
apollo_inputs = apollo_validateInputs()

### Work with predictions at estimates
predictions_base=predictions_base[["at_estimates"]]

### Remove ID and Observation columns from predictions
predictions_base = predictions_base[,-(1:2)]
predictions_new  = predictions_new[,-(1:2)]

### Compute the change in probabilities
change=(predictions_new-predictions_base)/predictions_base
Cheers
David

Re: Error in model predictions

Posted: 29 Nov 2022, 17:36
by Shan
Hi David,

Thank you for your reply. Yes, my IDs are non-numeric. I will add these two lines before creating "change''. If I add so, does it mean that I have to delete the line which deletes the first two columns after estimating "change"? That is, am I supposed to delete the second line after computing the change in probabilities?

### Compute the change in probabilities
change=(predictions_new-predictions_base)/predictions_base

### Not interested in chosen alternative now, so drop the last column
change=change[,-ncol(change)]

### First two columns (change in ID and task) are also not needed
change=change[,-c(1,2)]

Thanks!
Shan

Re: Error in model predictions

Posted: 30 Nov 2022, 15:22
by stephanehess
Hi Shan

yes, you can drop the line

change=change[,-c(1,2)]

Stephane