NiFi Parcel

This repository provides a parcel (https://github.com/cloudera/cm_ext) to install Apache NiFi as a service usable by Cloudera Manager.

Requirements

• CDH >=5.5 (tested with 5.15)
• HDFS
• Zookeeper installed by CDH. The embedded Zookeeper is not supported

The CSD provided here creates the following Items:

• a (system) user on each node in your Hadoop cluster
  • with /var/lib/nifi as home directory
• data and configuration files below /var/lib/nifi.
  • Custom configuration files (SSL, login, etc.) should also be placed there
  • It is posible to change most of those directories use the CDH GUI

Overview

• Quick install
  • below one this page
• Build Parcel and CSD
  • Build all
• Howtos
  • Setup SSL
  • Enable Ldap (Active Directory)
• Todos, finished features
  • bottom of this page

Quick install

The CSD provided can be installed like any other "Add-on" Service for CDH: CDH - Add-on Services

=== On the Node running Cloudera Manager ===

1. Download CSD latest release

$ cp NIFI-1.2.jar /opt/cloudera/csd
$ sudo service cloudera-scm-server restart
# Wait a min, go to Cloudera Manager -> Add a Service -> NiFi

2. Manually add your personal parcel download URL to Cloudera CDH Manager under Parcels->Configuration

• ❗ TODO: There is currently no public download for the Nifi parcel available, see Build all

3. Download, distribute and activate the parcel (Cloudera CDH Manager->Parcels)

Todos

• "Currently NiFi runs under the root user"
  • done. User is nifi.
• Expose config options under Cloudera Manager
  • Conf folder from parcels is used, this needs to be migrated to ConfigWriter - done for all basic settings
• simplify install
  • Provide precompiled CSD
    • see latest release
  • Provide parcel repo
  • Include parcel repo URL in CSD
• Expose metrics from NiFi in CDH
• Allow reading log files in CDH
• https config
  • document certificate Management
  • (optional) Puppet CA integration - blocked by https://tickets.puppetlabs.com/browse/SERVER-2338
• Login
  • this requires https
• Fix WebUI link in CDH
• Parcel building in Jenkins
  • Use Nifi releases
• multi distribution support for parcels

Bugs

• CDH cannot gracefully shutdown a Nifi node