Advice on Tuning Community Edition

3 posts / 0 new
Last post
sidmanaog
sidmanaog's picture
Offline
Last seen: 1 year 6 months ago
Joined: Nov 6 2012
Junior Boarder

Posts: 6

Sidmund Manaog
Advice on Tuning Community Edition

Hi Calpont Dev,

I'm currently using a community edition of InfiniDB. And the target concurrent users is 1000+. Some huge tables contains 200million up to 1billion records of data and is still increasing. And with the lack of expertise on administrative skills on InfiniDB, I'm currently experiencing crash on the InfiniDB or zero(0) memory on the server. With this scenario, I'm really needing your expert advice and suggestions/recommendations on how should I optimized or what should be the setup of our Calpont.xml.

Below are the details of the InfiniDB Server:

- 2x 4c 2.4GHz
- 128GB RAM
- 5TB HDD

And here are some of the current setup that I modified in the Calpont.xml:
50

128
128K
128
32G
128G
100

16K
10
40
80

Thanks so much!
Sid

bdempsey
bdempsey's picture
Offline
Last seen: 1 month 1 week ago
Joined: Oct 27 2009
Platinum Boarder

Posts: 194

Robert Dempsey
Re: Advice on Tuning Community Edition

A couple of tweaks:

PmMaxMemorySmallSide = 512M
TotalUmMemory = 64G
MaxOutstandingRequests = 8

You didn't say how much swap you have, but I would configure 32-64G.

NumBlocksPct and TotalUmMemory should not exceed the total RAM on the machine. PmMaxMemorySmallSide controls when queries switch from being a PM join to a UM join. On a single-server install, it doesn't make that much difference. Unless you have actual performance observations that say MaxOutstandingRequests works best in your situation at 80, I'd leave it at no more than the number of cores (since you can't do more work simultaneously than the number of cores).

You'll also almost certainly have to tune the mysql front-end to support 1000+ concurrent connections. There are numerous articles on the web for doing this. The InfiniDB mysql configuration file is in /usr/local/Calpont/mysql.

sidmanaog
sidmanaog's picture
Offline
Last seen: 1 year 6 months ago
Joined: Nov 6 2012
Junior Boarder

Posts: 6

Sidmund Manaog
Re: Advice on Tuning Community Edition

Thank you for your quick reply! Will implement this and try to tune the mysql configuration. Thanks for pointing that out!