Running multiple instances of cpimport on a Single (Server), Combined (Module) Install against different tables

2 posts / 0 new
Last post
venkat_jdsu
venkat_jdsu's picture
Offline
Last seen: 3 months 4 weeks ago
Joined: Mar 20 2014
Junior Boarder

Posts: 1

venkat.babu@jdsu.com Babu
Running multiple instances of cpimport on a Single (Server), Combined (Module) Install against different tables

Hi,

   I  installed InfiniDB with single UM and PM in one box,  created three tables and tried to load data to each table ( 500 K rows, each row has ~50 columns ).   When I run cpimport it took ~12 seconds to load data to a single table,  since the machine has 8 cores I started running the 3 instances of cpimport with each cpimport  loading data to different table.   I see the time it took  to complete each job is ~36 seconds.   Is there a sequential access happening while loading data to different tables ?  apart from changing/adding hardware  is there any other way to increase performance through software ? 

 

Options with which I am running cpimport are

 

cpimport -r8 -w12 -b13  -s, test table1 table1data.csv

cpimport -r8 -w12 -b13  -s, test table2 table2data.csv

 

cpimport -r8 -w12 -b13  -s, test table3 table3data.csv

 

 

Thanks for your help.

Venkat

 

radams
radams's picture
Offline
Last seen: 3 days 19 hours ago
Joined: Jan 3 2011
Administrator

Posts: 483

Robert Adams
Running multiple instances of cpimport on a Single (Server)

 

 

Running cpimport on separate tables would not be sequential. A cpimport job will only lock the table that it is updating. 

It could be that the system resources (memory or cpu) are slowing down the imports.

 

Thanks,

 

Robert