本文共 2781 字，大约阅读时间需要 9 分钟。
1.背景:
在迁移ETL至ALiYun EMR环境，使用我们定制的Hive-1.2.1-emr版本过程中，业务脚本需要使用json-serde-1.1.9.9-SNAPSHOT-jar-with-dependencies.jar
2.抛错如下:
Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. Cannot validate serde: org.openx.data.jsonserde.JsonSerDe.
3.解决版本:
升级json-serde jar为最新版
1.Configure mvn
[root@hadoopnn-01 hadoop]# export MAVEN_HOME=/hadoop/maven[root@hadoopnn-01 hadoop]# export PATH=$MAVEN_HOME/bin:$PATH
2.Download json-serde source code
[root@hadoopnn-01 hadoop]# mkdir test[root@hadoopnn-01 hadoop]# cd test[root@hadoopnn-01 test]# git clone https://github.com/rcongiu/Hive-JSON-Serde.git
3.Compile json-serde
[root@hadoopnn-01 test]# cd Hive-JSON-Serde/[root@hadoopnn-01 Hive-JSON-Serde]# mvn -Pcdh5 clean package..................................[INFO] Building jar: /hadoop/test/Hive-JSON-Serde/json-udf/target/json-udf-1.3.8-SNAPSHOT-tests.jar[INFO] ------------------------------------------------------------------------[INFO] Reactor Summary:[INFO] [INFO] openx-json-serde ................................... SUCCESS [ 1.627 s][INFO] json ............................................... SUCCESS [ 4.308 s][INFO] json-cdh4-shim ..................................... SUCCESS [ 3.250 s][INFO] json-cdh5-shim ..................................... SUCCESS [ 2.800 s][INFO] json-serde-main .................................... SUCCESS [ 6.131 s][INFO] json-udf ........................................... SUCCESS [ 1.701 s][INFO] ------------------------------------------------------------------------[INFO] BUILD SUCCESS[INFO] ------------------------------------------------------------------------[INFO] Total time: 20.445 s[INFO] Finished at: 2016-10-04T11:43:33+08:00[INFO] Final Memory: 76M/335M[INFO] ------------------------------------------------------------------------
4.sz json-serde-1.3.8-SNAPSHOT-jar-with-dependencies.jar
[root@hadoopnn-01 Hive-JSON-Serde]# cd json-serde/target/[root@hadoopnn-01 target]# lltotal 180drwxr-xr-x 2 root root 4096 Oct 4 11:43 archive-tmpdrwxr-xr-x 3 root root 4096 Oct 4 11:43 classes-rw-r--r-- 1 root root 37935 Oct 4 11:43 json-serde-1.3.8-SNAPSHOT.jar-rw-r--r-- 1 root root 83787 Oct 4 11:43 json-serde-1.3.8-SNAPSHOT-jar-with-dependencies.jar-rw-r--r-- 1 root root 29997 Oct 4 11:43 json-serde-1.3.8-SNAPSHOT-tests.jardrwxr-xr-x 2 root root 4096 Oct 4 11:43 maven-archiverdrwxr-xr-x 2 root root 4096 Oct 4 11:43 surefiredrwxr-xr-x 2 root root 4096 Oct 4 11:43 surefire-reportsdrwxr-xr-x 4 root root 4096 Oct 4 11:43 test-classes[root@hadoopnn-01 target]# sz json-serde-1.3.8-SNAPSHOT-jar-with-dependencies.jarrzStarting zmodem transfer. Press Ctrl+C to cancel.Transferring json-serde-1.3.8-SNAPSHOT-jar-with-dependencies.jar... 100% 81 KB 81 KB/sec 00:00:01 0 Errors
转载地址：http://vxrlo.baihongyu.com/