Hive搭建

发表于 | 分类于

在hadoop的基础安装hive，我在node-1上面安装hive

准备工作

  • hadoop环境
  • Mysql服务

安装

1
2
3
4
5
6
7
8
9
tar xvf apache-hive-3.1.2-bin.tar.gz
mv apache-hive-3.1.2 /usr/local/hive

cat >> /etc/profile << "EOF"
export HIVE_HOME=/usr/local/hive
export PATH=$PATH:$HIVE_HOME/bin
EOF

source /etc/profile

配置

hive的主要配置目录在: /usr/local/hive/conf

hive-env.sh

1
cp hive-env.sh.template hive-env.sh

修改配置

1
2
HADOOP_HOME=/usr/local/hadoop
export HIVE_CONF_DIR=/usr/local/hive/conf

hive-site.xml

1
2
cp hive-default.xml.template hive-site.xml
mkdir /usr/local/hive/tmp -p

修改hive-site.xml

  • 删除 ‘&#8’
  • 把 ${system:java.io.tmpdir} 修改成 /usr/local/apache-hive-3.1.2-bin/tmp；修改的操作命令如下（采用的vim全局替换命令）：%s/${system:java.io.tmpdir}//usr/local/apache-hive-3.1.2-bin/tmp/g；
  • 把 {system:user.name} 修改成 {user.name}；修改的操作命令如下（采用的vim全局替换命令）：%s/{system:user.name}/{user.name}/g；

修改数据库连接配置

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
<property>
    <name>javax.jdo.option.ConnectionURL</name>
    <value>jdbc:mysql://localhost:3306/hive?characterEncoding=UTF-8&amp;createDatabaseIfNotExist=true</value>
    <description></description>
</property>
<property>
    <name>javax.jdo.option.ConnectionDriverName</name>
    <value>com.mysql.jdbc.Driver</value>
    <description></description>
</property>
<property>
    <name>javax.jdo.option.ConnectionUserName</name>
    <value>root</value>
</property>
<property>
    <name>javax.jdo.option.ConnectionPassword</name>
    <value>******</value>
</property>

修改hive-jdbc连接配置

1
2
3
4
5
6
7
8
<property>
    <name>hive.server2.thrift.port</name>
    <value>10000</value>
</property>
<property>
    <name>hive.server2.thrift.bind.host</name>
    <value>0.0.0.0</value>
</property>

修改客户端连接认证方式

1
2
3
4
<property>
    <name>hive.server2.authentication</name>
    <value>NOSASL</value>
</property>

添加Mysql连接驱动

1
2
rpm -ivh mysql-connector-java-8.0.18-1.el7.noarch.rpm
cp /usr/share/java/mysql-connector-java.jar /usr/local/hive/lib/

更新guava.jar包

1
2
cp /usr/local/hadoop/share/hadoop/hdfs/lib/guava-27.0-jre.jar  /usr/local/hive/lib/
rm -f /usr/local/hive/lib/guava-19.0.jar

初始化元数据

1
schematool -dbType mysql -initSchema --verbose

启动hive

1
2
3
cd /usr/local/hive
./bin/hive --service metastore &
./bin/hive --service hiveserver2 &

测试

创建表

1
2
3
4
5
6
7
8
9
10
11
create external table if not exists testExtNew(
name string,
addr string
)
row format delimited fields terminated by '\t' 
lines terminated by '\n' 
stored as textfile
location '/wisp/test/';


select * from testExtNew;

上传数据

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
[root@node-1 data]# cat test.txt 
wisp	aaaaaaaaaa
lhp	bbbbbbb
test	cccccccccc
tttt	asdfasf
tttt	asdfasf
tsadfasdf	asddddddd1fasf

[root@node-1 data]# hadoop fs -put /data/test.txt /wisp/test/
[root@node-1 data]# hive
hive> select * from testExtNew;
OK
tttt	asdfasf
tttt	asdfasf
tsadfasdf	asddddddd1fasf
wisp	aaaaaaaaaa
lhp	bbbbbbb
test	cccccccccc

备注