HomeЛюди и блогиRelated VideosMore From: Online Guwahati

Oracle HDFS DataTransfer using Sqoop

2 ratings | 116 views
This video explains how we can transfer the structured data that persisted on Oracle database into HDFS. To process huge amount of data in multi node cluster irrespective of any format (structured, unstructured, semi-structured), we can use sqoop for data ingestion . Ideally, in Data lake we can ingeat any format of data to preccess and subsequeny analysis.
Html code for embedding videos on your blog
Text Comments (2)
Arun Subramanian (5 months ago)
Nice Explanation! In this video u clearly showed the connection between oracle & sqoop. what about HDFS & sqoop? After installing sqoop ,how can we know whether HDFS & sqoop is communicating each other? do we need to setup any configuration after sqoop installation?
Gautam Goswami (3 months ago)
Thanks for writing. SQOOP is the tool which initiates the communication bridge for data transfer from any RDBMS like Oracle, MySql, DB2 etc to HDFS for parallel processing. RDBMS database vendor specific driver class or lib file is mandatory that SQOOP utilizes to establish the connection with HDFS. For Oracle, we need to have ojdbc6_g.jar and similarly, for MySQL, this driver file will be different. Here are answers to your questions what about HDFS & sqoop? SQOOP is a utility or a part of Hadoop ECO System which is responsible to transfer data from any data source to HDFS and vice versa. It can't be leveraged other than HDFS. The HDFS should be data source or destination. how can we know whether HDFS & sqoop is communicating with each other? Probably you might have seen in the video that we should always first start the Hadoop Daemons like NodeManage, ResourceManager, DataNode etc and then need to verify through the browser that Hadoop cluster is up and running. SQOOP will through error/exception once you execute the SQOOP command without running the Hadoop cluster (HDFS). do we need to setup any configuration after sqoop installation? That depends on the Hadoop cluster. For multi-node cluster or cluster already set up on the cloud like AWS or Microsoft Azure.

Would you like to comment?

Join YouTube for a free account, or sign in if you are already a member.