type
status
date
slug
summary
tags
category
icon
password
这里写文章的前言:
一个简单的开头,简述这篇文章讨论的问题、目标、人物、背景是什么?并简述你给出的答案。
可以说说你的故事:阻碍、努力、结果成果,意外与转折。
📝 下载包准备
hadoop下载
选择我们对应的hadoop版本;比如我这里是是3.1.1就选择3.1.1
![notion image](https://www.notion.so/image/https%3A%2F%2Fprod-files-secure.s3.us-west-2.amazonaws.com%2Fb6f5e03a-51ea-46ef-946d-ada5d539c8a9%2F399af403-69be-46d4-a498-f3706b3e173e%2F1111.png?table=block&id=d1e769f7-98fc-4329-8b78-57ad9e408604&t=d1e769f7-98fc-4329-8b78-57ad9e408604)
win10 需要的脚本下载
选择我们对应的hadoop的版本
![notion image](https://www.notion.so/image/https%3A%2F%2Fprod-files-secure.s3.us-west-2.amazonaws.com%2Fb6f5e03a-51ea-46ef-946d-ada5d539c8a9%2Fffdc1001-657e-4779-b40e-d6263e0bbd5f%2F2222.png?table=block&id=61cd2086-a3f2-497c-8233-07f668574cab&t=61cd2086-a3f2-497c-8233-07f668574cab)
对应版本的脚本拷贝
操作一
将 hadoop.ddl 和 winutils.exe 拷贝到 下载的hadoop的版本的bin中
![notion image](https://www.notion.so/image/https%3A%2F%2Fprod-files-secure.s3.us-west-2.amazonaws.com%2Fb6f5e03a-51ea-46ef-946d-ada5d539c8a9%2Fedd207b5-82f4-435c-a407-cc3ca0402578%2F333.png?table=block&id=d710359d-fa98-4939-a939-c556f37e2998&t=d710359d-fa98-4939-a939-c556f37e2998)
操作二
将 hadoop.ddl 文件同时拷贝一份到 C:\Windows\System32 目录
![notion image](https://www.notion.so/image/https%3A%2F%2Fprod-files-secure.s3.us-west-2.amazonaws.com%2Fb6f5e03a-51ea-46ef-946d-ada5d539c8a9%2F94a5fe19-7ed9-4906-bd90-94f48098aafe%2F4444.png?table=block&id=5dbf9113-5e20-4819-8f4e-792da1b379a1&t=5dbf9113-5e20-4819-8f4e-792da1b379a1)
操作三
配置hadoop中的etc\hadoop的 hadoop-env.cmd 文件的java地址;切换为自己的java路径;
如果是C盘的话,Program Files需要替换为 PROGRA~1 ; 如下
![notion image](https://www.notion.so/image/https%3A%2F%2Fprod-files-secure.s3.us-west-2.amazonaws.com%2Fb6f5e03a-51ea-46ef-946d-ada5d539c8a9%2Fd19b0047-a2de-4329-8b46-df9ef8febb4d%2F5555.png?table=block&id=0e023fe8-2c92-4ea3-96a8-85ed55427196&t=0e023fe8-2c92-4ea3-96a8-85ed55427196)
操作四: 配置环境变量
HADOO_HOME配置
![notion image](https://www.notion.so/image/https%3A%2F%2Fprod-files-secure.s3.us-west-2.amazonaws.com%2Fb6f5e03a-51ea-46ef-946d-ada5d539c8a9%2F422a9c13-e0e6-4993-9886-debc469ca788%2F6666.png?table=block&id=65de5d1d-defd-40e6-b618-a59bf5c6f710&t=65de5d1d-defd-40e6-b618-a59bf5c6f710)
PATH中配置%HADOOP_HOME%\bin
![notion image](https://www.notion.so/image/https%3A%2F%2Fprod-files-secure.s3.us-west-2.amazonaws.com%2Fb6f5e03a-51ea-46ef-946d-ada5d539c8a9%2Fd6cf738e-e9f5-4942-bb4d-cd63f410b01b%2F7777.png?table=block&id=2e160ec4-d5a2-4ad1-af64-debb12e7cf54&t=2e160ec4-d5a2-4ad1-af64-debb12e7cf54)
操作五:测试是否成功
cmd中输入hadoop version;如果可以看到版本,就说明配置是成功的
![notion image](https://www.notion.so/image/https%3A%2F%2Fprod-files-secure.s3.us-west-2.amazonaws.com%2Fb6f5e03a-51ea-46ef-946d-ada5d539c8a9%2Fb70f403d-8f66-45fd-bad8-ad4d3cc9378d%2F8888.png?table=block&id=a67ea586-8ddb-4fd2-a43e-3178a5e7e848&t=a67ea586-8ddb-4fd2-a43e-3178a5e7e848)
操作hdfs以及Iceberg
注意点:
- 通过iceberg和hive去读取hdfs的时候,会读取集群的名字;所以这里需要配置根据集群的名字能映射到NameNode(状态为Active的最保险)
- 比如这里的usdp215对应的active namenode是 172.21.129.218 节点的话;则在host里进行配置即可
- 到底就可以通过iceberg或者hdfs等api在win10上读取hadoop的数据等
![notion image](https://www.notion.so/image/https%3A%2F%2Fprod-files-secure.s3.us-west-2.amazonaws.com%2Fb6f5e03a-51ea-46ef-946d-ada5d539c8a9%2Fc08ce122-00d5-471d-a642-08825b8dcaba%2F9999.png?table=block&id=78e9e4de-b34a-40ba-8bdd-077fa785e08d&t=78e9e4de-b34a-40ba-8bdd-077fa785e08d)
![notion image](https://www.notion.so/image/https%3A%2F%2Fprod-files-secure.s3.us-west-2.amazonaws.com%2Fb6f5e03a-51ea-46ef-946d-ada5d539c8a9%2F561a437c-083b-4f23-bfe4-f481c426837b%2F1010.png?table=block&id=cab3c259-258d-41c6-adf5-7c20465b79d8&t=cab3c259-258d-41c6-adf5-7c20465b79d8)
![notion image](https://www.notion.so/image/https%3A%2F%2Fprod-files-secure.s3.us-west-2.amazonaws.com%2Fb6f5e03a-51ea-46ef-946d-ada5d539c8a9%2Fdb8ad79f-2f92-4f8e-a27e-6b7e8aede71f%2F111111.png?table=block&id=710f15db-1c76-4f88-9507-8f651deae590&t=710f15db-1c76-4f88-9507-8f651deae590)
🤗 总结归纳
📎 参考文章
有关文章的问题,欢迎您在底部评论区留言,一起交流~