Convergence with different types of draws
Posted: 28 Aug 2020, 12:01
Hello, everyone,
i would like to know from you if there is a rule of thumb, when to use which kind of draws (Halton, Sobol, MLHS, ...)?
During my research, severel papers recommended not to use Halton draws due to correlation with a high number of parameters. For example, the paper...
Czajkowski, Mikołaj; Budziński, Wiktor (2019): Simulation error in maximum likelihood estimation of discrete choice models. In: Journal of Choice Modelling 31, S. 73–85. DOI: 10.1016/j.jocm.2019.04.003.
... concludes that Sobol draws are usually "better" (see below) than Halton draws:
"We compare the performance of pseudo-random draws with three quasi Monte Carlo methods (Halton, Sobol and modified Latin hypercube sampling) under 27 experimental conditions that differ with respect to experimental design, number of individuals and number of choice tasks per individual. Based on a Monte Carlo simulation using 100 to 1,000,000 draws, we can compare the relative efficiency of different types of draws. We consistently find that a scrambled Sobol sequence performs the best in terms of the lowest simulation error, while being matched by scrambled Halton draws in the case of 10 attributes."
Now, in some model estimations (e.g. MIXL or ICLV), I have found that a very high number of draws increases the probability that the model will not converge or that I do get NAs for the standard errors, regardless of which draws are used. Furthermore, I have found that Sobol draws lead to NAs in the standard errors already with fewer draws than Halton draws. For example, the estimation of an ICLV with 500 Halton draws still works, whereas 500 Sobol draws result in NAs.
Do you have any advice/experience?
I look forward to your answers!
i would like to know from you if there is a rule of thumb, when to use which kind of draws (Halton, Sobol, MLHS, ...)?
During my research, severel papers recommended not to use Halton draws due to correlation with a high number of parameters. For example, the paper...
Czajkowski, Mikołaj; Budziński, Wiktor (2019): Simulation error in maximum likelihood estimation of discrete choice models. In: Journal of Choice Modelling 31, S. 73–85. DOI: 10.1016/j.jocm.2019.04.003.
... concludes that Sobol draws are usually "better" (see below) than Halton draws:
"We compare the performance of pseudo-random draws with three quasi Monte Carlo methods (Halton, Sobol and modified Latin hypercube sampling) under 27 experimental conditions that differ with respect to experimental design, number of individuals and number of choice tasks per individual. Based on a Monte Carlo simulation using 100 to 1,000,000 draws, we can compare the relative efficiency of different types of draws. We consistently find that a scrambled Sobol sequence performs the best in terms of the lowest simulation error, while being matched by scrambled Halton draws in the case of 10 attributes."
Now, in some model estimations (e.g. MIXL or ICLV), I have found that a very high number of draws increases the probability that the model will not converge or that I do get NAs for the standard errors, regardless of which draws are used. Furthermore, I have found that Sobol draws lead to NAs in the standard errors already with fewer draws than Halton draws. For example, the estimation of an ICLV with 500 Halton draws still works, whereas 500 Sobol draws result in NAs.
Do you have any advice/experience?
I look forward to your answers!