Apache NiFi Quickstart Guide

Target Audience

  • If you want to learn Apache NiFi with examples, this tutorial is for you.

Notes

  • This tutorial is only for learning to use Apache NiFi, and using a local instance of NiFi is the best way to do that.
  • Data Spine NiFi is a central, cloud-native component in the EFPF ecosystem. If you are already familiar with using Apache NiFi, you can get started with the DS NiFi User Guide to use Data Spine NiFi directly.

Prerequisites

  • Docker Desktop/Engine is installed
  • Download this template to be loaded to NiFi later: Template

Introduction

  • In this tutorial, we will load an existing template onto NiFI’s (GUI) canvas and run it. Alternatively, you can also drag-and-drop processors onto the canvas, and connect them together to create a dataflow.
  • The dataflow in this template creates a reverse proxy endpoint for an existing external HTTP API endpoint: SMECluster Platform’s Marketplace Service that returns a list of available products and services. This service is publicly accessible.
  • That is, the dataflow listens for HTTP GET requests, and upon receiving a request, it calls SMECluster’s Marketplace Service, gets the response and returns it to the caller.

Steps

  1. Execute docker run -d --name nifi -p 8080:8080 -p 8091:8091 apache/nifi:latest to start NiFi.
  2. Navigate to http://localhost:8080/nifi/ in a Web browser. NiFi should serve its homepage:

img.png

  1. On the homepage of NiFi, click on the ‘Upload Template’ button on the ‘Operate Palette’ and select the template “ SMECluster_GetAllProducts_Proxy.xml” to be uploaded.

img.png

  1. To add this uploaded template to the ‘Canvas’, drag-and-drop the ‘Template (+)’ button from the ‘Components Toolbar’ onto the canvas and choose the template to be uploaded:

img_1.png

  1. Now, you should be able see the template as a Process Group on the Canvas.

img_2.png

  1. If you encounter an error while starting the workflow processors similar to ‘HTTP Context Map validated against xxxx is invalid because Controller Service xxxx is disabled’, you will need to enable the StandardHttpContextMap controller service.

img_4.png

  1. To enable the StandardHttpContextMap controller service, goto the home screen of NiFi (http://localhost:8080/nifi/), right click on the concerned process group and then click on ‘Configure’.

img_5.png

  1. On the configuration page, ‘CONTROLLER SERVICES’ tab, click on ‘Go To’ (→).

img_6.png

  1. On the ‘NiFi Flow Configuration’ page, ‘CONTROLLER SERVICES’ tab, click on ‘Enable’ (⚡).

img_7.png

  1. Finally, on the page, click on the ‘ENABLE’ button. The error should disappear now.

img_8.png

  1. Next, to start all the processors in this process group, select (single click) the process group and then click the ‘Start’ button on the ‘Operate Palette’:

img_2.png

  1. Alternatively, you can also view the NiFi workflow by double clicking on the process group and then select and start the processors individually:

img_3.png

  1. That’s it! Now, you can invoke http://localhost:8091 from the Web browser (or Postman, or using cURL) and get the response:

img_9.png

Data Spine NiFi Documentation

References

[1] Getting Started with Apache NiFi: https://nifi.apache.org/docs/nifi-docs/html/getting-started.html
[2] Apache NiFi Documentation: https://nifi.apache.org/docs.html

Previous
Next