Pitfalls and Lessons on Configuing and Tuning Hadoop
Posted on In TutorialThis post lists pitfalls and lessons learning when configuring and tuning Hadoop.
Table of Contents
Hadoop with IPv6
Hadoo doesn’t support IPv6 currently (up to 0.20.2 and 0.21.0): Hadoop and IPv6. The performance of the cluster may suffer from turning IPv6 on in clusters: mail archive.
One good practice is to disable IPv6 on servers in the Hadoop cluster.
Hostname vs. IP
Hostname is preferred. Using hostname instead of IP address may possible solve some problem magically.
Java Virtual Machine
Take care whether the JVM on all nodes are friendly with Hadoop. In case there are some nodes are configured with wrong JVM, problems may flow up.
Check the /etc/hosts file whether it contains wrong record. Always ensure it is correct.