How to Install and Configure Cloudera Manager on CentOS/RHEL 8?

Cloudera Manager is an enterprise-level software solution for managing Apache Hadoop clusters. It provides a web-based interface for deploying, configuring, and monitoring Hadoop clusters. Cloudera Manager is available in both open-source and enterprise editions. In this article, we will discuss how to install and configure Cloudera Manager on CentOS/RHEL 8.

Prerequisites

Before we proceed with installation, make sure that following prerequisites are met

• A fresh installation of CentOS/RHEL 8

• A user with sudo privileges

• Minimum 8GB RAM and 4 CPU cores

• At least 50GB free disk space

• A stable internet connection

Step 1: Install Java

Cloudera Manager requires Java to be installed on the system. While CentOS/RHEL 8 comes with OpenJDK pre-installed, Cloudera Manager recommends using Oracle JDK for optimal performance.

Download the latest version of Oracle JDK from the official website and extract it

tar zxvf jdk-<version>-linux-x64.tar.gz

Move the extracted directory to /usr/local

sudo mv jdk-<version> /usr/local

Set the JAVA_HOME environment variable by adding the following line to /etc/profile

export JAVA_HOME=/usr/local/jdk-<version>
export PATH=$PATH:$JAVA_HOME/bin

Reload the profile file and verify the installation

source /etc/profile
java -version

Step 2: Configure Cloudera Repository

Add the Cloudera repository to your system by creating a repository configuration file

sudo wget https://archive.cloudera.com/cm7/7.4.4/redhat8/yum/cloudera-manager.repo -P /etc/yum.repos.d/

Import the GPG key

sudo rpm --import https://archive.cloudera.com/cm7/7.4.4/redhat8/yum/RPM-GPG-KEY-cloudera

Step 3: Install Database

Cloudera Manager requires a database to store metadata. Install and configure PostgreSQL

sudo yum install -y postgresql-server postgresql-jdbc
sudo postgresql-setup --initdb
sudo systemctl enable postgresql
sudo systemctl start postgresql

Create a database and user for Cloudera Manager

sudo -u postgres psql -c "CREATE DATABASE scm DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;"
sudo -u postgres psql -c "CREATE USER 'scm'@'localhost' IDENTIFIED BY 'password';"
sudo -u postgres psql -c "GRANT ALL PRIVILEGES ON scm.* TO 'scm'@'localhost';"

Step 4: Install Cloudera Manager Server

Install the required dependencies and Cloudera Manager Server

sudo yum install -y cloudera-manager-daemons cloudera-manager-agent cloudera-manager-server

Initialize the database schema

sudo /opt/cloudera/cm/schema/scm_prepare_database.sh postgresql scm scm password

Start and enable Cloudera Manager Server

sudo systemctl start cloudera-scm-server
sudo systemctl enable cloudera-scm-server

Step 5: Install and Configure Cloudera Manager Agent

The Cloudera Manager Agent must be installed on all nodes in the cluster. On each node, install the agent

sudo yum install -y cloudera-manager-agent

Edit the agent configuration file /etc/cloudera-scm-agent/config.ini and set the server hostname

server_host=<cloudera_manager_server_hostname>

Start and enable the Cloudera Manager Agent

sudo systemctl start cloudera-scm-agent
sudo systemctl enable cloudera-scm-agent

Step 6: Accessing Cloudera Manager Web UI

Wait for the Cloudera Manager Server to start completely (this may take several minutes). Check the startup progress

sudo tail -f /var/log/cloudera-scm-server/cloudera-scm-server.log

Once the server is ready, access the web interface at http://<server-ip>:7180. The default credentials are

• Username: admin

• Password: admin

Key Features

Advanced Configuration

For production environments, consider these additional configurations

• High Availability: Configure multiple Cloudera Manager Server instances with load balancing

• SSL/TLS: Enable HTTPS for secure web access and encrypted communication

• LDAP Integration: Connect to your organization's directory service for user authentication

• Custom Metrics: Define application-specific monitoring metrics using JMX or the Cloudera Manager API

Troubleshooting

Common issues and their solutions

• Server startup failure: Check /var/log/cloudera-scm-server/cloudera-scm-server.log for database connection issues

• Agent connection problems: Verify firewall settings and ensure port 7182 is accessible

• Memory issues: Increase JVM heap size in /etc/default/cloudera-scm-server

Conclusion

Cloudera Manager provides a comprehensive platform for managing Hadoop clusters with enterprise-grade features. This installation guide covers the essential steps for setting up Cloudera Manager on CentOS/RHEL 8. With proper configuration and monitoring, Cloudera Manager ensures optimal performance and reliability of your big data infrastructure.