Problem with join of huge tables

3 posts / 0 new
Last post
borodah
borodah's picture
Offline
Last seen: 2 years 2 months ago
Joined: Jan 11 2012
Junior Boarder

Posts: 7

Boris Antonov
Problem with join of huge tables

Hi there!
Could you help me about a problem? The problem occurs when I try to execute query, which contains join of two big tables (60.000.000 and 180.000.000 rows)
I run the InfiniDB EE on Windows Server 2008 R2 Enterprise with 72 Gb RAM.

There is two scenarios of query fail:

1. If I set [b]TotalUmMemory[/b] less than 18G, I get error:
[code]IDB-2001 join or subselect exceeds memory limit[/code]

2. If I set TotalUmMemory more than 18G, I get [code]An unexpected condition within the query caused an internal processing error within InfiniDB. Please check the log files for more details. Additional Information: was an unknown internal error.[/code]
in console

and
[code]Fri Mar 30 17:45:52 2012 52.364182 |0|0|0| C 28 CAL0000: bad allocation

Fri Mar 30 17:45:52 2012 52.404186 |0|0|0| C 28 CAL0000: bad allocation

Fri Mar 30 17:45:52 2012 52.407186 |0|0|0| C 28 CAL0000: bad allocation

Fri Mar 30 17:45:52 2012 52.482193 |0|0|0| C 28 CAL0000: bad allocation

Fri Mar 30 17:45:52 2012 52.500195 |0|0|0| C 28 CAL0000: bad allocation

Fri Mar 30 17:45:52 2012 52.521197 |0|0|0| C 28 CAL0000: bad allocation

Fri Mar 30 17:45:52 2012 52.528198 |0|0|0| C 28 CAL0000: bad allocation

Fri Mar 30 17:45:52 2012 52.528198 |0|0|0| C 28 CAL0000: bad allocation

Fri Mar 30 17:45:52 2012 52.546200 |0|0|0| C 28 CAL0000: bad allocation

Fri Mar 30 17:45:52 2012 52.552200 |0|0|0| C 28 CAL0000: bad allocation

Fri Mar 30 17:45:52 2012 52.586204 |0|0|0| C 28 CAL0000: bad allocation

Fri Mar 30 17:45:52 2012 52.604206 |0|0|0| C 28 CAL0000: bad allocation

Fri Mar 30 17:45:52 2012 52.606206 |0|0|0| C 28 CAL0000: bad allocation

Fri Mar 30 17:45:52 2012 52.614207 |0|0|0| C 28 CAL0000: bad allocation

Fri Mar 30 17:45:52 2012 52.615207 |0|0|0| C 28 CAL0000: bad allocation

Fri Mar 30 17:45:52 2012 52.642209 |0|0|0| C 28 CAL0000: bad allocation

Fri Mar 30 17:45:52 2012 52.651210 |0|0|0| C 28 CAL0000: bad allocation

Fri Mar 30 17:45:52 2012 52.656211 |0|0|0| C 28 CAL0000: bad allocation

Fri Mar 30 17:45:52 2012 52.663212 |0|0|0| C 28 CAL0000: bad allocation

Fri Mar 30 17:45:52 2012 52.678213 |0|0|0| C 28 CAL0000: bad allocation

Fri Mar 30 17:45:52 2012 52.709216 |0|0|0| C 28 CAL0000: bad allocation

radams
radams's picture
Offline
Last seen: 21 hours 53 min ago
Joined: Jan 3 2011
Administrator

Posts: 487

Robert Adams
Re: Problem with join of huge tables

Hi,

If the join exceeds memory limit, you should reduce the size of the join, splitting the query to use ranges.

Thanks,

Robert

borodah
borodah's picture
Offline
Last seen: 2 years 2 months ago
Joined: Jan 11 2012
Junior Boarder

Posts: 7

Boris Antonov
Re: Problem with join of huge tables

Thank you!