Hi
I am running a MMNL model with 5 random parameters (all negative lognormal) on a panel dataset with 500,000 observations and 3200 individuals. I am running models on a 40 core machine, but the estimation (BHHH) crashes when I specify more than 150 draws (MLHS draws). I have set gradient comparison, Hessian calculation, and the scaling parameters to FALSE.
My question is whether 150 draws are enough for 5 random parameters and whether I should be using different draw types and/or estimation routines. Also, would bootstrapping the estimation on fewer draws be advisable or bad practice?
Thanks for your help!
Important: Read this before posting to this forum
- This forum is for questions related to the use of Apollo. We will answer some general choice modelling questions too, where appropriate, and time permitting. We cannot answer questions about how to estimate choice models with other software packages.
- There is a very detailed manual for Apollo available at http://www.ApolloChoiceModelling.com/manual.html. This contains detailed descriptions of the various Apollo functions, and numerous examples are available at http://www.ApolloChoiceModelling.com/examples.html. In addition, help files are available for all functions, using e.g. ?apollo_mnl
- Before asking a question on the forum, users are kindly requested to follow these steps:
- Check that the same issue has not already been addressed in the forum - there is a search tool.
- Ensure that the correct syntax has been used. For any function, detailed instructions are available directly in Apollo, e.g. by using ?apollo_mnl for apollo_mnl
- Check the frequently asked questions section on the Apollo website, which discusses some common issues/failures. Please see http://www.apollochoicemodelling.com/faq.html
- Make sure that R is using the latest official release of Apollo.
- Users can check which version they are running by entering packageVersion("apollo").
- Then check what is the latest full release (not development version) at http://www.ApolloChoiceModelling.com/code.html.
- To update to the latest official version, just enter install.packages("apollo"). To update to a development version, download the appropriate binary file from http://www.ApolloChoiceModelling.com/code.html, and install the package from file
- If the above steps do not resolve the issue, then users should follow these steps when posting a question:
- provide full details on the issue, including the entire code and output, including any error messages
- posts will not immediately appear on the forum, but will be checked by a moderator first. This may take a day or two at busy times. There is no need to submit the post multiple times.
Estimation fails with increasing number of draws
-
stephanehess
- Site Admin
- Posts: 998
- Joined: 24 Apr 2020, 16:29
Re: Estimation fails with increasing number of draws
Apologies for the slow reply.
This is an extremely large dataset and you are likely to run out of memory. 150 draws is a very small number of draws, and I would consider using sampling of decision-makers with a larger number of draws
This is an extremely large dataset and you are likely to run out of memory. 150 draws is a very small number of draws, and I would consider using sampling of decision-makers with a larger number of draws