本文共 3575 字,大约阅读时间需要 11 分钟。
1.Hive是基于Hadoop构建的一套数据仓库分析系统,它提供了丰富的SQL查询方式来分析存储在Hadoop 分布式文件系统中的数据。其在Hadoop的架构体系中承担了一个SQL解析的过程,它提供了对外的入口来获取用户的指令然后对指令进行分析,解析出一个MapReduce程序组成可执行计划,并按照该计划生成对应的MapReduce任务提交给Hadoop集群处理,获取最终的结果。元数据——如表模式——存储在名为metastore的数据库中。 2.下载http://mirror.bjtu.edu.cn/apache/hive/ 解压到/usr/local/hadoop/hive 3.拷贝mysql-connector-java-5.1.6-bin.jar 到hive 的lib下面 4.将hive下的新版本jline的JAR包拷贝到hadoop下: cp /usr/local/hadoop/hive /lib/jline-2.12.jar /usr/local/hadoop /share/hadoop/yarn/lib 并且移除$HADOOP_HOME/share/hadoop/yarn/lib/下的jline-0.9.94.jar文件 5.gedit ~/.bashrc #HADOOP VARIABLES START export JAVA_HOME=/jdk/jdk1.8.0_91 export HADOOP_INSTALL=/usr/local/hadoop export PATH=$PATH:$HADOOP_INSTALL/bin export PATH=$PATH:$JAVA_HOME/bin export PATH=$PATH:$HADOOP_INSTALL/sbin export HADOOP_MAPRED_HOME=$HADOOP_INSTALL export HADOOP_COMMON_HOME=$HADOOP_INSTALL export HADOOP_HDFS_HOME=$HADOOP_INSTALL export YARN_HOME=$HADOOP_INSTALL exportHADOOP_COMMON_LIB_NATIVE_DIR=$HADOOP_INSTALL/lib/native exportHADOOP_OPTS="-Djava.library.path=$HADOOP_INSTALL/lib" #HADOOP VARIABLES END #set Hive environment export HIVE_HOME=/usr/local/hadoop/hive export PATH=$PATH:$HIVE_HOME/bin export CLASSPATH=$CLASSPATH:$HIVE_HOME/bin 6.gedit /etc/profile #set java environment export JAVA_HOME=/jdk/jdk1.8.0_91 export JRE_HOME=$JAVA_HOME/jre exportCLASSPATH=$CLASSPATH:$JAVA_HOME/lib:$JRE_HOME/lib export PATH=$PATH:$JAVA_HOME/bin:$JRE_HOME/bin #set hive export HIVE_HOME=/usr/local/hadoop/hive export PATH=$HIVE_HOME/bin:$PATH 7.重命名配置文件 对conf 目录下的配置文件重命名,执行如下命令: mv hive-env.sh.template hive-env.sh mv hive-default.xml.template hive-site.xml 8.在hive-env.sh中 HADOOP_HOME=/usr/local/hadoop export HIVE_CONF_DIR=/usr/local/hadoop/hive/conf 9.修改hive-site.xml <property> <name>hive.exec.scratchdir</name> <value>/usr/local/hadoop/hive/tmp</value> <description>HDFS rootscratch dir for Hive jobs which gets created with write all (733) permission.For each connecting user, an HDFS scratch dir:${hive.exec.scratchdir}/<username> is created, with${hive.scratch.dir.permission}.</description> </property> <property> <name>hive.exec.local.scratchdir</name> <value>/usr/local/hadoop/hive/tmp</value> <description>Local scratchspace for Hive jobs</description> </property> <property> <name>hive.downloaded.resources.dir</name> <value>/usr/local/hadoop/hive/tmp</value> Maximum number of bytes ascript is allowed to emit to standard error (per map-reduce task). This prevents runaway scriptsfrom filling logs partitions to capacity </description> </property> <property> <name>hive.metastore.warehouse.dir</name> <value>/usr/local/hadoop/hive/warehouse</value> <description>location ofdefault database for the warehouse</description> </property> <property> <name>hive.metastore.uris</name> <value>thrift://127.0.0.1:9083</value> <description>Thrift URIfor the remote metastore. Used by metastore client to connect to remotemetastore.</description> </property> <property> <name>hive.metastore.local</name> <value>true</value> <description></description> </property> <property> <name>javax.jdo.option.ConnectionPassword</name> <value>myhive</value> <description>password touse against metastore database</description> </property> <property> <name>javax.jdo.option.ConnectionURL</name> <value>jdbc:mysql://localhost:3306/hive?createDatabaseIfNotExsit=true;characterEncoding=UTF-8</value> <description>JDBC connectstring for a JDBC metastore</description> </property> <property> <name>javax.jdo.option.ConnectionUserName</name> <value>myhive</value> <description>Username touse against metastore database</description> </property> 转载地址:http://yomvb.baihongyu.com/