sqoop从hive导出到mysql报错 ERROR mapreduce.ExportJobBase: Export job failed!

最新推荐文章于 2024-01-04 13:27:31 发布

总分全班倒数第一

最新推荐文章于 2024-01-04 13:27:31 发布

阅读量1.1w

点赞数 7

CC 4.0 BY-SA版权

分类专栏： sqoop 文章标签： sqoop 大数据 mysql hadoop

本文链接：https://blog.csdn.net/weixin_41668549/article/details/90302929

sqoop 专栏收录该内容

3 篇文章

订阅专栏

1，错误日志关键如下

19/05/17 11:48:15 INFO mapreduce.Job: Running job: job_1558105459102_0007
19/05/17 11:48:42 INFO mapreduce.Job: Job job_1558105459102_0007 running in uber mode : false
19/05/17 11:48:42 INFO mapreduce.Job:  map 0% reduce 0%
19/05/17 11:49:25 INFO mapreduce.Job:  map 80% reduce 0%
19/05/17 11:49:27 INFO mapreduce.Job:  map 100% reduce 0%
19/05/17 11:49:28 INFO mapreduce.Job: Job job_1558105459102_0007 failed with state FAILED due to: Task failed task_1558105459102_0007_m_000003
Job failed as tasks failed. failedMaps:1 failedReduces:0

19/05/17 11:49:28 INFO mapreduce.Job: Counters: 12
        Job Counters
                Failed map tasks=4
                Killed map tasks=1
                Launched map tasks=5
                Rack-local map tasks=5
                Total time spent by all maps in occupied slots (ms)=201601
                Total time spent by all reduces in occupied slots (ms)=0
                Total time spent by all map tasks (ms)=201601
                Total vcore-seconds taken by all map tasks=201601
                Total megabyte-seconds taken by all map tasks=206439424
        Map-Reduce Framework
                CPU time spent (ms)=0
                Physical memory (bytes) snapshot=0
                Virtual memory (bytes) snapshot=0
19/05/17 11:49:28 WARN mapreduce.Counters: Group FileSystemCounters is deprecated. Use org.apache.hadoop.mapreduce.FileSystemCounter instead
19/05/17 11:49:28 INFO mapreduce.ExportJobBase: Transferred 0 bytes in 79.8861 seconds (0 bytes/sec)
19/05/17 11:49:28 INFO mapreduce.ExportJobBase: Exported 0 records.
19/05/17 11:49:28 ERROR mapreduce.ExportJobBase: Export job failed!
19/05/17 11:49:28 ERROR tool.ExportTool: Error during export:
Export job failed!
        at org.apache.sqoop.mapreduce.ExportJobBase.runExport(ExportJobBase.java:445)
        at org.apache.sqoop.manager.SqlManager.exportTable(SqlManager.java:931)
        at org.apache.sqoop.tool.ExportTool.exportTable(ExportTool.java:80)
        at org.apache.sqoop.tool.ExportTool.run(ExportTool.java:99)
        at org.apache.sqoop.Sqoop.run(Sqoop.java:147)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
        at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183)
        at org.apache.sqoop.Sqoop.runTool(Sqoop.java:234)
        at org.apache.sqoop.Sqoop.runTool(Sqoop.java:243)
        at org.apache.sqoop.Sqoop.main(Sqoop.java:252)

2，发现map都跑完了，然而最后却一直报错，却又一直找不到错误的关键信息，排查了好久发现是mysql创建对应表的字段的长度不够。
hive的表中有一个职业名称的字段没想到长度达到了47，
3，解决方案，修改mysql表的相应字段的长度，最后完美导入。