INSERT speed

7 posts / 0 new
Last post
maitri98
maitri98's picture
Offline
Last seen: 4 years 1 day ago
Joined: Aug 31 2010
Junior Boarder

Posts: 6

kevn cramer
INSERT speed

We have not yet tested doing standard INSERTS into InfiniDB and benchmarking it against InnoDB...our current engine of choice. The presumption is InfiniDB will be much slower for obvious reasons.

From what I have read you have an excellent batch loader utility for populating the DB. Have you considered exposing this through an API for use? Most true applications cannot just do a batch load of data from a file. There is auditing that needs done, loads may come from 100's of client machines connecting to the DB through a middle tier server, there is rollback support needed in the case of something bombing out, etc. So being able to do these loads fast yet supporting a true production environment would be useful.

FYI...i am aware that your standard batch loader does not provide rollback support, but if a variant existed that does support this, yet is faster than a standard SQL insert, would be useful.

Kevin

davestokes
davestokes's picture
Offline
Last seen: 3 years 3 months ago
Joined: Apr 24 2010
Gold Boarder

Posts: 116

David Stokes
Re:INSERT speed

Kevin,

I will pass your request to expose the API to the team.

--Dave

maitri98
maitri98's picture
Offline
Last seen: 4 years 1 day ago
Joined: Aug 31 2010
Junior Boarder

Posts: 6

kevn cramer
Re:INSERT speed

Thanks Dave for this and the other replies...

ddumitri
ddumitri's picture
Offline
Last seen: 3 years 3 months ago
Joined: Feb 22 2011
Junior Boarder

Posts: 4

Dan Dumitriu
Re:INSERT speed

Hi,

Currently I am using JDBC and load data infile to do the loading but it's too slow (compared to mysql).

Any updates on the plans to provide an API for the bulk loader?

Thanks
Dan

radams
radams's picture
Offline
Last seen: 2 days 16 hours ago
Joined: Jan 3 2011
Administrator

Posts: 492

Robert Adams
Re:INSERT speed

Hi Dan,

We don't have an API for other apps to use, but cpimport is still the fastest way to load data into Infinidb.

An additional update for the original post, cpimport does have rollback support.

Thanks,

Robert
Calpont

ddumitri
ddumitri's picture
Offline
Last seen: 3 years 3 months ago
Joined: Feb 22 2011
Junior Boarder

Posts: 4

Dan Dumitriu
Re:INSERT speed

I agree it is the fastest :)

As a thing on the wish list : currently I cannot specify the column order in the file (it must be exactly same column order as in the DB). Can we get such a feature?

Also it would be useful to be able to leave out some columns if you do not necessarily want to load them. Currently I am forced to give a file matching exactly the table.

thanks

radams
radams's picture
Offline
Last seen: 2 days 16 hours ago
Joined: Jan 3 2011
Administrator

Posts: 492

Robert Adams
Re:INSERT speed

Thanks for the input.

We have had similar requests for enhanced functionality in this area. Please watch for our next feature release, which will be coming out in the near future, to add flexibility when using cpimport.

Thanks again,

Robert
Calpont