ApolloChoiceModelling forum

Dear Stephane,

I´ve conducted the Out of Sample test function in Apollo resulting in the following outputs:

LL per obs in estimation sample LL per obs in validation sample % difference
1 -1.0348 -1.0144 1.97
2 -1.0338 -1.0256 0.79
3 -1.0236 -1.1150 -8.94
4 -1.0296 -1.0592 -2.88
5 -1.0289 -1.0652 -3.53
6 -1.0386 -0.9809 5.56
7 -1.0367 -0.9970 3.83
8 -1.0339 -1.0222 1.13
9 -1.0378 -0.9923 4.38
10 -1.0312 -1.0448 -1.32
Average -1.0329 -1.0317 0.10

Unfortunately, I did not find any specific quantifiable interpretation guideline for the out of sample outputs in the manual.
Can you tell how the results can be interpreted? And is it okay to stay with the default of 10% validation sample for a total sample size of N=330?

Thanks for your support in advance.
Best,
J.

Hi

there is no specific hard rule in terms of what size of difference is acceptable, but your differences look pretty small, suggesting no specific risk of overfitting

Stephane