apollo_outOfSample discrepancy in description of estimation and validation sample
Posted: 20 Aug 2024, 20:07
Hello,
There seems to be a discrepancy in the description of the split between the estimation sample and validation sample when using apollo_outOfSample. I specified 25% of the sample to be used for validation. In the output, it says that 25% of the individuals will be used for estimation, but then when I look at the number of individuals specified for estimation and for forecasting, the number of individuals for validation (forecasting) corresponds to 25% of the sample (182/728). This is the code I used for the out of sample test:
This is the output I received when I ran the code:
Based on the Apollo manual, it seems that there is a typo in the description of percentage "25% of individuals for estimation" and the number of individuals shown is the correct number used for the actual estimation.
In the Apollo manual this is the example code on page 145:
Thank you
There seems to be a discrepancy in the description of the split between the estimation sample and validation sample when using apollo_outOfSample. I specified 25% of the sample to be used for validation. In the output, it says that 25% of the individuals will be used for estimation, but then when I look at the number of individuals specified for estimation and for forecasting, the number of individuals for validation (forecasting) corresponds to 25% of the sample (182/728). This is the code I used for the out of sample test:
Code: Select all
apollo_outOfSample(apollo_beta, apollo_fixed,
apollo_probabilities, apollo_inputs,
outOfSample_settings = list(nRep=10,
validationSize= 0.25))I am assuming that it is a typo in the description and that the correct validation(for forecasting) sample size is used, 25% of the sample for validation, as specified in the code (and not 25% for estimation). But could you please confirm this?Testing likelihood function.
10 separate runs will be conducted, each using a random subset of 25% of individuals for estimation, and the remainder for
validation.
Number of individuals
- for estimation : 546
- for forecasting : 182
- in sample (total): 728
Based on the Apollo manual, it seems that there is a typo in the description of percentage "25% of individuals for estimation" and the number of individuals shown is the correct number used for the actual estimation.
In the Apollo manual this is the example code on page 145:
I am using V.3.2 of Apollo. Perhaps this has been fixed in V.3.3. However, I am currently running my models in V.3.2apollo_outOfSample(apollo_beta, apollo_fixed, apollo_probabilities, apollo_inputs)
2 10 separate runs will be conducted, each using a random subset of 90% for estimation and the
,! remainder
3 for validation.
4 Number of individuals
5 - for estimation : 349
6 - for forecasting : 39
7 - in sample (total): 388
Thank you