Apache-nifi-links

Following is a set of links that i've gather while evaluating NiFi as an ETL tool.

The first place to look for NiFi info would probably be awesome-nifi but if you didnt find what you are looking for there... here is my additional personal list of links.

nifi git repo

Documentation

Nifi notes - greaat overview with lots of gathered knowladge

NiFi Developer’s Guide

Apache nifi Mailing Lists

Intro to apache nifi

Apache NIFI for dummies

Apache NiFi In Depth

Official Getting Started with Apache NiFi

Apache NiFi Expression Language Guide

Apache NiFi 1.x Cheatsheet

Awesome NiFi List

Getting Started with Apache Nifi

Apache NiFi Development Quickstart

Beginners guide to Apache NiFi flows

Moving data around with Apache NiFi

Data Distribution Patterns with Apache NiFi

Standard Nifi Processors and Controller Services

Syncfusion Data Integration Platform

Apache NiFi Not From Scratch

Apache NiFi Developer List

The Mine of Information

Medium

What is Apache NiFi?

INTRODUCTION TO APACHE NIFI


Hortonworks guides

Getting Started with Apache NiFi pdf

Apache NiFi User Guide pdf

Apache NiFi Administration pdf

Apache MiNiFi Administration pdf

Apache NiFi Expression Language Guide pdf

Apache NiFi Developer Guide pdf

Apache NiFi REST API Reference

HortonWorks Apachi NiFi videos

Hortonworks Community Connection

Additional resourses - videos

Hortonworks Technical Workshop: Hortonworks DataFlow - Apache NiFi


Articles

Leveraging Apache NiFi for Agile Dataflows

Introduction to NiFi with WebInterpret

Apache NiFi - Part 1 (Introduction)

Apache NiFi - Part 2 (Twitter Flow)

3 considerations for Apache NiFi in Financial Services

HDF/NIFI Best practices for setting up a high performance NiFi installation.

NiFi Sizing Guide & Deployment Best Practices

NiFi/HDF Dataflow Optimization

Enterprise NiFi: Implementing Reusable Components and an SDLC

How can we update NIFI workflow in production

automating NIFI template deployment

NiFi automatic deployment for NiFi 1.x versions in Java

ANALYZE TRANSIT PATTERNS WITH APACHE NIFI

Apache NiFi Flow Demo

How Do I Distribute Data Across an Apache NiFi Cluster

NiFi Ranger based policy descriptions

Apache NiFi 1.x Cheatsheet

nifi-notify-batch

How to generate N FlowFiles and set the content of each FlowFile according to the data in Elastic?

Apache NiFi - Processing Multi-line Logs

Multiple Versions of the Same Extension

Records

Apache NiFi - Records and Schema Registries

Convert CSV to JSON, Avro, XML using ConvertRecord

arsing Apache NiFi Records to HBase

Process Groups

Introduction into Process Group Variables

NiFi - Understanding how to use Process Groups and Remote Process Groups

debugging workflow in nifi processor group

Simple Tasks in NiFi – Turning Flows into Groups


Json support

NiFi and JSON

JSON-to-JSON Simplified with Apache NiFi and Jolt

Complex JSON Transformation with Jolt

how i extract attribute from json file using nifi

Merge json events based on property

Nifi Merge Json Files Then Turn into JsonArray

Best approach to re merging flowfiles in NiFi

Nifi Merge Json Files Then Turn into JsonArray

Combining Attributes in NiFi

nifi-examples - json processor

Jolt

In order to work with Jolt Json Transformation processor you should be familier with Jolt syntax

Jolt main page

Jolt Transform Demo is a great assistance to check your Jolt expressions

Getting Started

Jolt Example

Transforming a Array into Array with pairs

Apache NiFi Split JSON root array

Unit Tests

Developing a Custom Apache Nifi Processor-Unit Tests (Part I)

Data Aggregation

Add processor to perform simple aggregations

Data aggregation in Apache Nifi

Create a general purpose Aggregating processor

Grok

Logstash grok filter help - fixed position file

Intro to grok

ExtractGrok.java

Json Path

Json Path Tester

Json Path Evaluator

Intro to JsonPath

Unable to return a scalar value for the expression from NIFI EvaluateJsonPath processor

how i extract attribute from json file using nifi


Solutions

update JSON attributes from different flow file in Apache NiFi

NiFi and HTTP Post Configuration

A demo collecting wifi admin packets with a raspberry pi, and pushing them to nifi.

Split one Nifi flow file into Multiple flow file based on attribute

NiFi example on how to join CSV files to create a merged result

healthservices nifi solution

MP3 Jukebox with NIFI 1.x

NIFI 1.x For Automatic Music Playing Pipelines

Adding A Custom Processor to NiFi : LinkProcessor


Development

Apache Nifi Jira issues

NiFi Debugging Tutorial


Interesting NiFi repositories

Nifi Deployment

Apache NiFi - How do I deploy my flow?

Shell for scripting against Apache NiFi - Havent managed to get it to work but looked at its code and its interesting.

NiFi api deploy

Update NiFi Flow On-the-Fly via API

terraform-provider-nifi

nifi-config

A convenient Python wrapper for Apache NiFi

Client swagger for nifi with security

NIFI Deployment Via API's

Consume API with Nifi-Token authentication

Configuration Management of Flows

automating NIFI template deployment

Post a NIFI template via REST?

NiFi Deploy Process Group

Hortonworks HDF - Apache Nifi - how to deploy a flow from dev environment to production environment

Testing automation, Deployment in NiFi

[Upload a NiFi template via nifi-api](Upload a NiFi template via nifi-api)

NIFI Deployment Via API's

Building processors / Additional processors

nifi-network-processor

nifi-processor-examples

ambari-nifi-service

An Ambari Service for NiFi

nifi-streaming-examples

NiFi-Extensions

nifi-rabbitmq-bundle

nifi_amqp_processors

nifi-ConvertJSONtoCSV-bundle

Machine Learning in NiFi

nifi-box-connector

nifi-put-site-to-site-bundle

nifi-lookup-service

nifi-redis-bundle

nifi-websockets-bundle

nifi-msgpack-bundle

nifi-compose-bundle

Custom Processors for Apache NiFi

Developing A Custom Apache Nifi Processor (JSON)

Salesforce processors

How to invoke remote rest service in InvokeHttp nifi processor?

Integrating Solace with NiFi JMS Processor

Writing a custom NiFi Processor

NiFiByExample - post http with json body

Setting up for custom NiFi processor development

Apache NIFI Custom Processor Design

NiFi origin processor name or id as attribute

Custom Processors for Apache NiFi

Custom processor extended GenerateTableFetch

Nifi Custom processor exception

nifi-sample-processor-bundle

nifi-addons

building custom services

Custom Controller Service Debugging

nifi-dependency-example

Developing a Custom Apache Nifi Controller Service

Custom implementation of standard Controller Service interface

problems with custom controller services, and other comments

Cross NAR Controller Services

How to access Controller Service created in UI into Root processors in NiFi-1.1.1?

Custom NiFi processor gets "ControllerServices may be referenced only via their interfaces" error.

Records

Record-Oriented Data with NiFi

NiFI for Apache - the flow using records and registry

Record based processors in Apache NiFi 1.2

Apache NiFi Record Processing

Better Together: NiFi, Schema Registry and Streaming Analytics Manager (Part 1. NiFi and Schema Registry)

hortonworks/registry

HWX Schema Registry User Guide

InvokeScriptedProcessor template revisited (with Jython)

Using NiFi api

Creating NiFi Template via Rest API

Post a NIFI template via REST

Nifipi Ruby wrapper

How can find a nifi process group id by Nifi rest api

api example

Learning Nifi

nifi workshop

nifi-master-class

Example Dataflow Templates

NiFi confluence - Example Dataflow Templates

HortonWorks nifi templates

SQL-to-CSV_ExecuteScript

JSON_Array_to_String_List

nifi-streaming-examples

kylo data transformation templates

6020-salesforce-oauth2-login-example.xml

NiFi blogs

Apache NiFi official blog

Bryan Bende

Matt Burgess

Andy LoPresto

ijokarumawak

pierre villard

richards technotes

OAuth

OAuth test server

OAuth 2 Authorization support - Client Credentials Grant

Apache NiFi - Authentication with OpenID Connect

[NiFi and OAuth 2.0 to request WordPress API](https://pierrevillard.com/2017/01/31/nifi-and-oauth-2-0-to-request-wordpress-api/

OAuth 1.0A with Apache NiFi (Twitter API example)

Simple OAuth library for Java

Getting Started with scribejava

Apache Nifi: use OAuth 2.0 to Access Google Analytics APIs

How to set Twitter OAuth attributes to InvokeHttp processor for fetching twitter user profiles

using GetHTTP for Salesforce Integration with OAuth2.0 authentication parameter

NIFI-4246 - Client Credentials Grant based OAuth2 Controller Service

Nifi Oauth Groovy

Apache Nifi: use OAuth 2.0 to Access Google Analytics APIs

Salesforce oauth template

Schema Registry

Registry

schema-registry-examples

NiFi Registry

Apache NiFi Registry User Guide

Maven

Maven Projects for Extensions

Apache Nifi failure due to java.lang.NoClassDefFoundError in a local maven dependency jar.

Maven 2 assembly with dependencies: jar under scope “system” not included

Rest

How to invoke remote rest service in InvokeHttp nifi processor?

NiFi and HTTP Post Configuration

Post HTTP Attributes to Send as HTTP Headers (Regex)

HTTP communication with Apache NiFi

Detect dedup

Detect duplicate based on 2 attributes per second: NIFI

Inspecting your NiFi DistributedMapCacheServer with Groovy

DistributedMapCacheClient Groovy example

How to pass credential to NiFi rest api

NiFi and HTTP Post Configuration

SSL

Creating a SSL context Nifi

Connecting to DataSift HTTPS API using NiFi GetHTTP

Configuring Apache NiFi SSL Authentication

HTTPS Endpoint in NiFi Flow

Help with SSL Context Service for https post requests

Using SSL with NiFi

Merge

Nifi- MergeContent processor- Mergestrategy

Nifi Automation

Nifi Stop all Processors from Shell

NiPyApi

[NiPyApi doc](Welcome to Nipyapi’s documentation!)

Nipyapi documentation

NIFI - How to set properties of a Processor through java api?

Programmatically adding processors

Developing a Custom Apache Nifi Processor-Unit Tests (Part I)

Unit test failing for custom processor's 'optional' properties

Testing processor logging message in Junit tests

Cluster

Setup apache NiFi cluster

nifi-service-clustered

Data Replication

Apache NiFi 1.0.0 – Cluster setup

Apache NiFi 1.1.0 – Secured cluster setup

Scaling up/down a NiFi cluster

Integration of NiFi with LDAP

Authorizations with LDAP synchronization in Apache NiFi 1.4+

HAProxy load balancing in front of Apache NiFi

Using counters in Apache NiFi

List/Fetch pattern and Remote Process Group in Apache NiFi

XML

XML data processing with Apache NiFi

Monitoring

Monitoring NiFi – Bootstrap notifier

Monitoring NiFi – Logback configuration

Monitoring NiFi – Site2Site reporting tasks

Monitoring NiFi – Workflow SLA

Monitoring NiFi – Ambari & Grafana

Monitoring NiFi – Scripted Reporting Task

Mini NiFi

Monitoring NiFi – Introduction

NAR

NiFi NAR Files Explained

Other

Is possible to write an attribute into a file and also keep the original flow file?

Halecium - A Next Generation of NiFi

Class loader

Apache NiFi - Class Loading

Inspecting the NAR classloading hierarchy

Per-Instance ClassLoading

Extend std processors

nifi 1.0.0 - extend the behaviour of a standard processor

Duplicated processors when using nifi processors dependency

nifi 1.0 extending a standard processor

Testing

Faster way of developing and Testing new Nifi Processor

Execute scripts

ExecuteScript - Using Modules

ci cd

Configuration Management of Flows

Extension Repositories (aka Extension Registry) for Dynamically-loaded Extensions

Apache NiFi - Component Versioning

Change NiFi Flow Using Rest API - Part 1

Creating NiFi Template via Rest API

Versioned DataFlows with Apache NiFi 1.5 and Apache NiFi Registry 0.1.0

Apache NiFi - How do I deploy my flow?

New Features in Apache NiFi 1.5: Registry and Version Control

FDLC: towards Flow Development Life Cycle with NiFi registries

Installing registry video

Development Life Cycle for Apache NiFi

NiPyAPI - NiFi Python API - A convenient Python wrapper for Apache NiFi Automation and SDLC / CICD

NiFi SDLC Automation in Python with NiPyApi - Part 1

Nifi Stop all Processors from Shell

Nifi Stop all Processors from Shell

env configuration

Is there a fix for the “Too many open files in system” error on OS X 10.7.1?

HOW TO SET JAVA_HOME ON MAC OS X

other tools

AWS Glue

BatchIQ

Difference between Apache NiFi and StreamSets

list of etl tools

Twitter folks related to nifi

Bryan Bende

Matt Burgess

Milo Simpson

Interesting use cases

ANALYZE TRANSIT PATTERNS WITH APACHE NIFI

Apache NiFi: Bonnes pratiques et retours d'expérience

Google Vision & Apache NiFi - Making Advanced Computer Vision Feasible

Using Apache Nifi to Stream Live Twitter Feeds to Hadoop

Accessing Facebook Page Data from Apache NiFi 1.2

IOT AND DATA SCIENCE – A TRUCKING DEMO ON DSX LOCAL WITH APACHE NIFI

Data Ingestion using Apache NiFi for Building Data Lakes - Twitter Data - Part 1

Analyze Flickr user interests using Apache NiFi and Spark

IOT

Secure IoT Command,Control, and Exfil with Apache MiNiFi code

HTTP Listen

How to listen to https with ListenHTTP in NiFi and setup ssl

URL shortener service with Apache NiFi

To Sort

nifi-integration-tests

nifi-api-client-js

testing

kylo

Configuration Management of Flows

Apache nifi with Elasticsearch experiment

Let Your Data Flow with Apache NiFi

Orchestrating NLP Building Blocks With Apache NiFi For Named-Entity Extraction

Use NiFi to process Excel spreadsheets in automated workflows

Gentle Introduction to Apache NiFi for Data Flow... and Some Clojure