Dear Apollo Team,
I am currently working with Apollo in the context of hybrid choice modelling, and I have encountered computational limitations when estimating a relatively complex model.
The model is based on a best-worst scaling / maxdiff choice experiment with approximately 6,400 respondents. The specification I would like to estimate includes a latent class structure, two latent variables in the class allocation component, and more than 150 parameters to be estimated. When running the model with parallel processing, I receive the following error message:
Error in checkForRemoteErrors(lapply(cl, recvResult)) :
6 nodes produced errors; first error: cannot allocate vector of size 28.5 Mb
I understand that this is most likely related to memory limitations, especially because the model objects and data may be copied across workers when using multiple cores.
We are now considering purchasing a new computer specifically for estimating such complex Apollo models. Could you please advise what kind of hardware specifications you would recommend, especially in terms of:
- RAM capacity,
- processor type / number of cores,
- whether fewer but more powerful cores or more cores are preferable,
- and whether you would recommend a workstation/server-type machine for models of this complexity?
More generally, I would be grateful for any practical guidance regarding the hardware requirements for estimating large-scale hybrid choice models with latent variables and latent classes in Apollo.
Thank you very much in advance for your help.
Kind regards,
Peter
Important: Read this before posting to this forum
- This forum is for questions related to the use of Apollo. We will answer some general choice modelling questions too, where appropriate, and time permitting. We cannot answer questions about how to estimate choice models with other software packages.
- There is a very detailed manual for Apollo available at http://www.ApolloChoiceModelling.com/manual.html. This contains detailed descriptions of the various Apollo functions, and numerous examples are available at http://www.ApolloChoiceModelling.com/examples.html. In addition, help files are available for all functions, using e.g. ?apollo_mnl
- Before asking a question on the forum, users are kindly requested to follow these steps:
- Check that the same issue has not already been addressed in the forum - there is a search tool.
- Ensure that the correct syntax has been used. For any function, detailed instructions are available directly in Apollo, e.g. by using ?apollo_mnl for apollo_mnl
- Check the frequently asked questions section on the Apollo website, which discusses some common issues/failures. Please see http://www.apollochoicemodelling.com/faq.html
- Make sure that R is using the latest official release of Apollo.
- Users can check which version they are running by entering packageVersion("apollo").
- Then check what is the latest full release (not development version) at http://www.ApolloChoiceModelling.com/code.html.
- To update to the latest official version, just enter install.packages("apollo"). To update to a development version, download the appropriate binary file from http://www.ApolloChoiceModelling.com/code.html, and install the package from file
- If the above steps do not resolve the issue, then users should follow these steps when posting a question:
- provide full details on the issue, including the entire code and output, including any error messages
- posts will not immediately appear on the forum, but will be checked by a moderator first. We check the forum at least twice a week. It may thus take a couple of days for your post to appear and before we reply. There is no need to submit the post multiple times.
Hardware requirements for complex hybrid latent class models in Apollo
-
stephanehess
- Site Admin
- Posts: 1365
- Joined: 24 Apr 2020, 16:29
Re: Hardware requirements for complex hybrid latent class models in Apollo
Peter
how many draws are you using? And what were the runtimes like for simpler specifications of the model, e.g. just latent class? That would tell us whether there are some issues specific to your implementation, as your model per se is not beyond models we have successfully run on standard laptops
Stephane
how many draws are you using? And what were the runtimes like for simpler specifications of the model, e.g. just latent class? That would tell us whether there are some issues specific to your implementation, as your model per se is not beyond models we have successfully run on standard laptops
Stephane