Sqoop is a tool designed to transfer data between Hadoop and relational databases. You can use Sqoop to import data from a relational database management system (RDBMS) such as MySQL or Oracle into the Hadoop Distributed File System (HDFS), transform the data in Hadoop MapReduce, and then export the data back into an RDBMS.
sqoop-import --connect jdbc:oracle:thin:@db.test.com:PORT:INSTANCE_NAME --table DW_DATAMART.HCM_EMPLOYEE_D --fields-terminated-by '\t' --lines-terminated-by '\n' --username RAHUL -P
sqoop-import --connect jdbc:oracle:thin:@db.test.com:PORT:INSTANCE_NAME --table DW_DATAMART.HCM_EMPLOYEE_D --fields-terminated-by '\t' --lines-terminated-by '\n' --username RAHUL -P
The sqoop-eval tool allows users to quickly run simple SQL queries against a database; results are printed to the console. This allows users to preview their import queries to ensure they import the data they expect.
sqoop-eval --connect jdbc:oracle:thin:@db.test.com:PORT:INSTANCE_NAME --select * from DW_DATAMART.HCM_COMPANY_D
sqoop-import --connect jdbc:oracle:thin:@db.test.com:PORT:INSTANCE_NAME --username RAHUL --P --table DW_DATAMART.PAY_PAY_CHK_OPTION_D --hbase-table DW_DATAMART.PAY_PAY_CHK_OPTION_D --column-family cf1 --hbase-create-table
If no primary key defined in the Oracle table
sqoop-import --connect jdbc:oracle:thin:@db.test.com:1725:hrlites --username RAHUL --P --table PSMERCHANTID --hbase-table PSMERCHANTID --column-family cf --hbase-row-key MERCHANTID --hbase-create-table --split-by MERCHANTID
sqoop-import --connect jdbc:oracle:thin:@db.test.com:PORT:INSTANCE_NAME --username RAHUL --P --table DW_DATAMART.PAY_PAYGROUP_D --hbase-table DW_DATAMART.PAY_PAYGROUP_D --column-family cf1 --hbase-create-table
sqoop-import --connect jdbc:oracle:thin:@db.test.com:1725:hrlites --username RAHUL --P --table PSMERCHANTID --hbase-table PSMERCHANTID --column-family cf --hbase-create-table --split-by MERCHANTID
JDBC ORACLE: Examples for Import:
sqoop-import --connect jdbc:oracle:thin:@db.test.com:PORT:INSTANCE_NAME --table DW_DATAMART.HCM_EMPLOYEE_D --fields-terminated-by '\t' --lines-terminated-by '\n' --username RAHUL -P
sqoop-import --connect jdbc:oracle:thin:@db.test.com:PORT:INSTANCE_NAME --table DW_DATAMART.HCM_EMPLOYEE_D --fields-terminated-by '\t' --lines-terminated-by '\n' --username RAHUL -P
JDBC ORACLE: Example for Select:
The sqoop-eval tool allows users to quickly run simple SQL queries against a database; results are printed to the console. This allows users to preview their import queries to ensure they import the data they expect.
sqoop-eval --connect jdbc:oracle:thin:@db.test.com:PORT:INSTANCE_NAME --select * from DW_DATAMART.HCM_COMPANY_D
Sqoop Import to HBASE table:
sqoop-import --connect jdbc:oracle:thin:@db.test.com:PORT:INSTANCE_NAME --username RAHUL --P --table DW_DATAMART.PAY_PAY_CHK_OPTION_D --hbase-table DW_DATAMART.PAY_PAY_CHK_OPTION_D --column-family cf1 --hbase-create-table
If no primary key defined in the Oracle table
sqoop-import --connect jdbc:oracle:thin:@db.test.com:1725:hrlites --username RAHUL --P --table PSMERCHANTID --hbase-table PSMERCHANTID --column-family cf --hbase-row-key MERCHANTID --hbase-create-table --split-by MERCHANTID
sqoop-import --connect jdbc:oracle:thin:@db.test.com:PORT:INSTANCE_NAME --username RAHUL --P --table DW_DATAMART.PAY_PAYGROUP_D --hbase-table DW_DATAMART.PAY_PAYGROUP_D --column-family cf1 --hbase-create-table
sqoop-import --connect jdbc:oracle:thin:@db.test.com:1725:hrlites --username RAHUL --P --table PSMERCHANTID --hbase-table PSMERCHANTID --column-family cf --hbase-create-table --split-by MERCHANTID
t is very excellent blog and useful article thank you for sharing with us , keep posting learn more about Big Data Hadoop
ReplyDeleteimportant information thank you providing this important information on Big Data Hadoop Online Training
Thank You :)
DeleteReally it's a very great article.
ReplyDeletekeep sharing more posts with us.
hadoop admin online course