hadoop 3.1.2 源码编译
编译发行版:
编译最好是在 mac 或者 linux 环境下进行,windows 环境编译如下
编译环境:
* Windows System
* JDK 1.8
* Maven 3.0 or later
* ProtocolBuffer 2.5.0
* CMake 3.1 or newer
* Visual Studio 2010 Professional or Higher
* Windows SDK 8.1 (if building CPU rate control for the container executor)
* zlib headers (if building native code bindings for zlib)
* Internet connection for first build (to fetch all Maven and Hadoop dependencies)
* Unix command-line tools from GnuWin32: sh, mkdir, rm, cp, tar, gzip. These
tools must be present on your PATH.
* Python ( for generation of docs using 'mvn site')
从这个网站下载 windows 下支持执行 git,sh 之类命令的 git-scm http://git-scm.com/downloads
如果用 Visual Studio, 必须是专业版或者更高版本.
不要用 Visual Studio Express. 该版本不支持 64-bit, 编译源码是 64 位的
The Windows SDK 8.1 is available to download at:
http://msdn.microsoft.com/en-us/windows/bg162891.aspx
不需要 Cygwin.
构建:
因为 windows 的最大路径长度的限制,保持源码在一个较短的没有中文字符的路径中,如: C:\hdc
命令文件放置在 dev-support 目录下的 win-paths-eg.cmd.
你可以按照你的模式复制和修改其内容.
win-paths-eg.cmd 是为了设置可用环境. 你需要修改这个文件. 它会将所有需要的内容添加到命令行环境中, 按需修改.
Several tests require that the user must have the Create Symbolic Links
privilege.
All Maven goals are the same as described above with the exception that
native code is built by enabling the ‘native-win’ Maven profile. -Pnative-win
is enabled by default when building on Windows since the native components
are required (not optional) on Windows.
设置环境变量 ZLIB_HOME
set ZLIB_HOME=C:\zlib-1.2.7
zlib1.dll 必须在 PATH 中. Hadoop has been tested
with zlib 1.2.7, built using Visual Studio 2010 out of contrib\vstudio\vc10 in
the zlib 1.2.7 source tree.
Building distributions:
- Build distribution with native code : mvn package [-Pdist][-Pdocs][-Psrc][-Dtar][-Dmaven.javadoc.skip=true]
Running compatibility checks with checkcompatibility.py
Invoke ./dev-support/bin/checkcompatibility.py
to run Java API Compliance Checker
to compare the public Java APIs of two git objects. This can be used by release
managers to compare the compatibility of a previous and current release.
As an example, this invocation will check the compatibility of interfaces annotated as Public or LimitedPrivate:
./dev-support/bin/checkcompatibility.py –annotation org.apache.hadoop.classification.InterfaceAudience.Public –annotation org.apache.hadoop.classification.InterfaceAudience.LimitedPrivate –include “hadoop.*” branch-2.7.2 trunk
Changing the Hadoop version declared returned by VersionInfo
If for compatibility reasons the version of Hadoop has to be declared as a 2.x release in the information returned by
org.apache.hadoop.util.VersionInfo, set the property declared.hadoop.version to the desired version.
For example: mvn package -Pdist -Ddeclared.hadoop.version=2.11
If unset, the project version declared in the POM file is used.
ubuntu 条件下的编译方法
创建二进制文件不包含文档和native代码
$ mvn package -Pdist -DskipTests -Dtar -Dmaven.javadoc.skip=true
创建二进制文件包含文档和native代码:
$ mvn package -Pdist,native,docs -DskipTests -Dtar
构建发行版:
$ mvn package -Psrc -DskipTests
构建发行版包括文档和native代码:
$ mvn package -Pdist,native,docs,src -DskipTests -Dtar
在 /tmp/hadoop-site 目录中创建网页内容
$ mvn clean site -Preleasedocs; mvn site:stage -DstagingDirectory=/tmp/hadoop-site
本文内容由互联网用户自发贡献,版权归作者所有,本社区不拥有所有权,也不承担相关法律责任。如果您发现本社区中有涉嫌抄袭的内容,欢迎发送邮件至:jaagool@sina.cn 进行举报,并提供相关证据,一经查实,本社区将立刻删除涉嫌侵权内容。