Importing data from hadoop into infinidb

1 post / 0 new
ntp009
ntp009's picture
Offline
Last seen: 1 year 2 weeks ago
Joined: Jun 24 2013
Junior Boarder

Posts: 1

Nirav Thakar
Importing data from hadoop into infinidb

Hi Guys,

I am trying to import data from hadoop hdfs into infinidb.

I have downloaded the connector provided by the Calpont.
I am able to import the data through the infinidb_hadoop_bulkload.sh, but when I try running the job to import the same using the [b]InfiniDBOutputDriver as main class [/b]and by using the following command :
[code] hadoop jar Infinidb_hadoop.jar [/code]
My Driver class is as follows :

[code]public class InfiniDBOutputDriver extends Configured implements Tool
{
public int run (String[] args) throws Exception
{
Configuration conf = new Configuration();
JobConf jobconf = new JobConf(conf, InfiniDoopDriver.class);
DBConfiguration.configureDB(jobconf,
"com.mysql.jdbc.Driver",
"jdbc:mysql://localhost/hadooptest","root", "");
String [] outFields = {"hadoopsample1"};
jobconf.setInputFormat(IDBFileInputFormat.class);
jobconf.setOutputFormat(InfiniDBOutputFormat.class);
jobconf.setOutputKeyClass(NullWritable.class);
jobconf.setOutputValueClass(Text.class);
InfiniDBOutputFormat.setOutput(jobconf, "hadooptest", outFields);
InfiniDBConfiguration idbconf = new InfiniDBConfiguration(jobconf);
idbconf.setInputPath("/var/hadoopsample1.tbl");
idbconf.setOutputPath("/var1/");
idbconf.setInfiniDBHome("/usr/local/Calpont");

jobconf.setMapperClass(InfiniDoopMapper.class);
jobconf.setNumMapTasks(1);
jobconf.setNumReduceTasks(2);
JobClient client = new JobClient();
client.setConf(jobconf);
try {
JobClient.runJob(jobconf);
} catch (Exception e) {
e.printStackTrace();
}

return 0;
}

public static void main(String [] args) throws Exception
{
int ret = ToolRunner.run(new InfiniDBOutputDriver(), args);
System.exit(ret);
}

}
[/code]
but when I try to execute the same, I am getting the following error. :(

[code]13/06/24 19:27:55 INFO mapred.FileInputFormat: Total input paths to process : 9
13/06/24 19:27:55 INFO mapred.JobClient: Cleaning up the staging area hdfs://localhost:8020/var/lib/hadoop-hdfs/cache/mapred/mapred/staging/niravthakar/.staging/job_201306231211_0020
13/06/24 19:27:55 ERROR security.UserGroupInformation: PriviledgedActionException as:niravthakar (auth:SIMPLE) cause:java.io.IOException: Not a file: hdfs://localhost:8020/var/lib
java.io.IOException: Not a file: hdfs://localhost:8020/var/lib
at org.apache.hadoop.mapred.FileInputFormat.getSplits(FileInputFormat.java:212)
at org.apache.hadoop.m