• 欢迎光临~

Hadoop集群常见脚本

开发技术 开发技术 2022-07-18 次浏览

集群启动/停止方法

1.各个模块分开启动/停止(配置ssh是前提)

(1)整体启动/停止HDFS

start-dfs.sh/stop-dfs.sh

(2)整体启动/停止YARN

start-yarn.sh/stop-yarn.sh

2.各个服务组件逐一启动/停止

1)分别启动/停止HDFS 组件

hdfs --daemon start/stop namenode/datanode/secondarynamenode

2)启动/停止YARN

yarn --daemon start/stop  resourcemanager/nodemanager

Hadoop集群常用脚本

Hadoop集群启停脚本(包含HDFS,Yarn,Historyserver): myhadoop.sh

[jinci@hadoop102 ~]$ cd /home/jinci/bin
[jinci@hadoop102 bin]$ vim myhadoop.sh

输入如下内容

#!/bin/bash

if [ $# -lt 1 ]
then
    echo "No Args Input..."
    exit ;
fi

case $1 in
"start")
        echo " =================== 启动 hadoop集群 ==================="

        echo " --------------- 启动 hdfs ---------------"
        ssh hadoop102 "/opt/module/hadoop-3.1.3/sbin/start-dfs.sh"
        echo " --------------- 启动 yarn ---------------"
        ssh hadoop103 "/opt/module/hadoop-3.1.3/sbin/start-yarn.sh"
        echo " --------------- 启动 historyserver ---------------"
        ssh hadoop102 "/opt/module/hadoop-3.1.3/bin/mapred --daemon start historyserver"
;;
"stop")
        echo " =================== 关闭 hadoop集群 ==================="

        echo " --------------- 关闭 historyserver ---------------"
        ssh hadoop102 "/opt/module/hadoop-3.1.3/bin/mapred --daemon stop historyserver"
        echo " --------------- 关闭 yarn ---------------"
        ssh hadoop103 "/opt/module/hadoop-3.1.3/sbin/stop-yarn.sh"
        echo " --------------- 关闭 hdfs ---------------"
        ssh hadoop102 "/opt/module/hadoop-3.1.3/sbin/stop-dfs.sh"
;;
*)
    echo "Input Args Error..."
;;
esac

保存后退出,然后赋予脚本执行权限

[jinci@hadoop102 bin]$ chmod 777 myhadoop.sh

启动/停止脚本

[jinci@hadoop102 ~]$ myhadoop.sh start/stop

查看三台服务器Java进程脚本:jpsall

[jinci@hadoop102 ~]$ cd /home/jinci/bin
[jinci@hadoop102 bin]$ vim jpsall

输入以下内容:

#!/bin/bash

for host in hadoop102 hadoop103 hadoop104
do
        echo =============== $host ===============
        ssh $host jps 
done

保存后退出,然后赋予脚本执行权限

[jinci@hadoop102 bin]$ chmod 777 jpsall

分发/home/jinci/bin 目录, 保证自定义脚本在三台机器上都可以使用

[jinci@hadoop102 ~]$ xsync /home/atguigu/bin/

运行效果:

myhadoop

Hadoop集群常见脚本

jpsall

Hadoop集群常见脚本

程序员灯塔
转载请注明原文链接:Hadoop集群常见脚本
喜欢 (0)