1.sqoop使用

export SQOOP=/home/hadoop/sqoop-1.4.5.bin__hadoop-2.0.4-alpha/bin/sqoop
function sqoop_exc()
{
${SQOOP} import \
--connection-manager org.apache.sqoop.manager.DDBManager \
--driver com.**.backend.db.DBDriver \
--connect jdbc:mysql://[ip]/[DBName] \
--username  \
--password  \
--fields-terminated-by '\001' \
--split-by id \
--query "${sqlStr}" \
--target-dir ${hadoopDestDir} \
-m 1 \
--delete-target-dir \
--fetch-size 1000 \
--null-string '' \
--null-non-string '' 

}

2.常见问题

2.1 遇到 Java Heap space

查询的表太大400Mb吧 解决办法

//添加命令
sqoop import -Dmapreduce.map.memory.mb=8192 -Dmapreduce.map.java.opts=-Xmx7200m -Dmapreduce.task.io.sort.mb=2400 

2.2 19/05/27 13:36:42 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.

--append and --delete-target-dir can not be used together.

try time:0
Warning: /home/hadoop/soft/sqoop/../hbase does not exist! HBase imports will fail.
Please set $HBASE_HOME to the root of your HBase installation.
Warning: /home/hadoop/soft/sqoop/../hcatalog does not exist! HCatalog jobs will fail.
Please set $HCAT_HOME to the root of your HCatalog installation.
Warning: /home/hadoop/soft/sqoop/../accumulo does not exist! Accumulo imports will fail.
Please set $ACCUMULO_HOME to the root of your Accumulo installation.
Warning: /home/hadoop/soft/sqoop/../zookeeper does not exist! Accumulo imports will fail.
Please set $ZOOKEEPER_HOME to the root of your Zookeeper installation.
19/05/27 13:36:42 INFO sqoop.Sqoop: Running Sqoop version: 1.4.5
19/05/27 13:36:42 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
--append and --delete-target-dir can not be used together.
state=1

遇见此问题 发现

--append and --delete-target-dir  不能同时使用