CPImport only (appears) to report cpu time, not real time

3 posts / 0 new
Last post
allan.wax
allan.wax's picture
Offline
Last seen: 1 month 1 week ago
Joined: Jun 14 2014
Junior Boarder

Posts: 12

Allan Wax
CPImport only (appears) to report cpu time, not real time

I'm using cpimport to get a very large table from MySql to infinidb.  After several attempts and massaging the data a little, the load succeeded.  However the time reported does not correspond to the wall clock time.  It would be nice if both were reported.

 

Below: wall clock is 6 hours but the reported time appears to be 4 hours (14189 sec.).

 

[root@test2 tmp]# /usr/local/Calpont/bin/cpimport f2_stats TrafficProviderDailyRollUp_201405 /data/tmp/tpdru-hour1.csv ...
...
Using table OID 3000 as the default JOB ID
Input file(s) will be read from : /data/tmp
Job description file : /usr/local/Calpont/data/bulk/tmpjob/f2_stats_TrafficProviderDailyRollUp_201405_D20140630_T101528_Job_3000.xml
Log file for this job: /usr/local/Calpont/data/bulk/log/Job_3000.log
2014-06-30 10:15:28 (7942) INFO : successfully loaded job file /usr/local/Calpont/data/bulk/tmpjob/f2_stats_TrafficProviderDailyRollUp_201405_D20140630_T101528_Job_3000.xml
2014-06-30 10:15:28 (7942) INFO : Job file loaded, run time for this step : 0.402345 seconds
2014-06-30 10:15:28 (7942) INFO : PreProcessing check starts
2014-06-30 10:15:28 (7942) INFO : input data file /data/tmp/tpdru-hour1.csv
2014-06-30 10:15:32 (7942) INFO : PreProcessing check completed
2014-06-30 10:15:32 (7942) INFO : preProcess completed, run time for this step : 3.80741 seconds
2014-06-30 10:15:32 (7942) INFO : No of Read Threads Spawned = 1
2014-06-30 10:15:32 (7942) INFO : No of Parse Threads Spawned = 3
2014-06-30 14:11:56 (7942) INFO : For table f2_stats.TrafficProviderDailyRollUp_201405: 2634273816 rows processed and 2634273816 rows inserted.
2014-06-30 14:11:57 (7942) INFO : Bulk load completed, total run time : 14189.2 seconds

 

radams
radams's picture
Offline
Last seen: 4 days 16 hours ago
Joined: Jan 3 2011
Administrator

Posts: 492

Robert Adams
CPImport only (appears) to report cpu time, not real time

 

 

Hi Allan,

 

Looking at your example, 14-10 = 4.

 

Thanks,

 

Robert

 

allan.wax
allan.wax's picture
Offline
Last seen: 1 month 1 week ago
Joined: Jun 14 2014
Junior Boarder

Posts: 12

Allan Wax
6 hours

I knew I should have listened to my elementary school techer blush