Error running cpimport (no extent entries)

7 posts / 0 new
Last post
kensuda
kensuda's picture
Offline
Last seen: 10 months 4 days ago
Joined: Jul 16 2010
Junior Boarder

Posts: 14

Ken Suda
Error running cpimport (no extent entries)

I am attempting to import from a pipe delimited file and am getting the following error:
[code]
job description file : Job_2001.xml
2010-07-15 16:34:24 (10348) INFO : successfully load job file /usr/local/Calpont/data/bulk/job/Job_2001.xml
2010-07-15 16:34:24 (10348) INFO : PreProcessing check starts
ExtentMap::getLastLocalHWM(): There are no extent entries for OID 3017

2010-07-15 16:34:24 (10348) ERR : Error getting last HWM for column file 3017; a BRM get hwm error. [BRM error status: FAILED] [1460]
2010-07-15 16:34:24 (10348) CRIT : Error in pre-processing the job file for table 0 : testdata.data [1460]

Error in loading job data
[/code]

I was able to insert a row using an insert command and delete it. I have dropped and recreated both the database and table with the same results.

"dbrmctl status" returns "OK."

I am using: Server version: 5.1.39 MySQL Community / InfiniDB Community Edition 1.5.2-2 Final (GPL)

Thanks,
Ken

davestokes
davestokes's picture
Offline
Last seen: 3 years 2 months ago
Joined: Apr 24 2010
Gold Boarder

Posts: 116

David Stokes
Re:Error running cpimport (no extent entries)

Hi Ken,

Thank you for downloading and using InfiniDB.

Did you re-run colxml before running cpimport?? Dropping and recreating the table would create new OIDs that would not be updated in the XML file that colxml generates and cpimport uses.

Please keep us advised on your situation.

kensuda
kensuda's picture
Offline
Last seen: 10 months 4 days ago
Joined: Jul 16 2010
Junior Boarder

Posts: 14

Ken Suda
Re:Error running cpimport (no extent entries)

I just re-ran colxml to be sure, and got the same errors.

Thanks,
Ken

[code]
$ /usr/local/Calpont/bin/colxml testdata -j 2001 -t data -u root
Running colxml with the following parameters:
2010-07-16 12:01:55 (30344) INFO :
Schema: testdata
Tables: data
Load Files:
-b 0
-c 1048576
-d |
-e 10
-f CSV
-j 2001
-n
-p /usr/local/Calpont/data/bulk/job/
-r 5
-s
-u root
-w 10485760
-x tbl

File completed for tables:
testdata.data

Normal exit.
$ /usr/local/Calpont/bin/cpimport -j 2001

Bulkload root directory : /usr/local/Calpont/data/bulk
job description file : Job_2001.xml
2010-07-16 12:02:10 (30441) INFO : successfully load job file /usr/local/Calpont/data/bulk/job/Job_2001.xml
2010-07-16 12:02:10 (30441) INFO : PreProcessing check starts
2010-07-16 12:02:10 (30441) ERR : Error opening column file for OID-3033; DBRoot-1; partition-0; segment-2; filename-/usr/local/Calpont/data1/000.dir/000.dir/011.dir/217.dir/000.dir/FILE002.cdf; Error in reading from a database file. [1022]
2010-07-16 12:02:10 (30441) CRIT : Error in pre-processing the job file for table 0 : testdata.data [1022]

Error in loading job data
[/code]

Correction: The actual error is now different.

kensuda
kensuda's picture
Offline
Last seen: 10 months 4 days ago
Joined: Jul 16 2010
Junior Boarder

Posts: 14

Ken Suda
Re:Error running cpimport (no extent entries)

It appears that now we are having a permissions issue. When I run the import command as me, I am getting the failure to access. The actual server processes are running as root. All of the directories appear to only have user write permissions and are owned by root. When I run the command as root, then it appears to work fine. Is there a setting somewhere that the permissions/ownership of server created files are set? For now, we have changed the ownership of the directories and permissions to see if this lets us do the full import.

davestokes
davestokes's picture
Offline
Last seen: 3 years 2 months ago
Joined: Apr 24 2010
Gold Boarder

Posts: 116

David Stokes
Re:Error running cpimport (no extent entries)

We recommend running [i]colxml[/i] and [i]cpimport[/i] as root. This is due to the number of files like the XML file [i]cpimport[/i] uses and roll back files under /usr/local/Calpont/data.

Could you use [i]sudo[/i] to allow non root accounts access?

kensuda
kensuda's picture
Offline
Last seen: 10 months 4 days ago
Joined: Jul 16 2010
Junior Boarder

Posts: 14

Ken Suda
Re:Error running cpimport (no extent entries)

In a production environment we would probably change to running the database as a user with an appropriate umask to avoid running as root. Running as sudo is the equivalent to running as root. What I did not realize at the time was that the cpimport does not run through the server processes, so uses the runners permissions instead of the servers. There should probably be a note in the getting started import page to warn about running the import as the same user as the server or one that has similar permissions. It would have also been better if the initial error message was clearer about being a permission denied issue on the whole database and not something that made it sound like something was corrupt in a column.

davestokes
davestokes's picture
Offline
Last seen: 3 years 2 months ago
Joined: Apr 24 2010
Gold Boarder

Posts: 116

David Stokes
Re:Error running cpimport (no extent entries)

I will forward your comments on to the engineering and documentation folks.