Apache NiFi Tutorial
Selected Reading
- Apache NiFi - Discussion
- Apache NiFi - Useful Resources
- Apache NiFi - Quick Guide
- Apache NiFi - Logging
- Apache NiFi - Custom Controllers Service
- Apache NiFi - Custom Processor
- Apache NiFi - Reporting Task
- Apache NiFi - Controller Settings
- Apache NiFi - Remote Process Group
- Apache NiFi - Upgrade
- Apache NiFi - Monitoring
- Apache NiFi - Data Provenance
- Apache NiFi - API
- Apache NiFi - Templates
- Apache NiFi - Creating Flows
- Apache NiFi - Administration
- Apache NiFi - Configuration
- Apache NiFi - Labels
- Apache NiFi - Process Groups
- Apache NiFi - Queues
- Apache NiFi - FlowFile
- Apache NiFi - Processors Relationship
- Apache NiFi - Processors Categorization
- Apache NiFi - Processors
- Apache NiFi - User Interface
- Apache NiFi - Environment Setup
- Apache NiFi - Basic Concepts
- Apache NiFi - Introduction
- Apache NiFi - Home
Selected Reading
- Who is Who
- Computer Glossary
- HR Interview Questions
- Effective Resume Writing
- Questions and Answers
- UPSC IAS Exams Notes
Apache NiFi - Data Provenance
Apache NiFi - Data Provenance
Apache NiFi logs and store every information about the events occur on the ingested data in the flow. Data provenance repository stores this information and provides UI to search this event information. Data provenance can be accessed for full NiFi level and processor level also.
The following table psts down the different fields in the NiFi Data Provenance event pst have following fields −
| S.No. | Field Name | Description |
|---|---|---|
| 1 | Date/Time | Date and time of event. |
| 2 | Type | Type of Event pke ‘CREATE’. |
| 3 | FlowFileUuid | UUID of the flowfile on which the event is performed. |
| 4 | Size | Size of the flowfile. |
| 5 | Component Name | Name of the component which performed the event. |
| 6 | Component Type | Type of the component. |
| 7 | Show pneage | Last column has the show pneage icon, which is used to see the flowfile pneage as shown in the below image. |
To get more information about the event, a user can cpck on the information icon present in the first column of the NiFi Data Provenance UI.
There are some properties in nifi.properties file, which are used to manage NiFi Data Provenance repository.
| S.No. | Property Name | Default Value | Description |
|---|---|---|---|
| 1 | nifi.provenance.repository.directory.default | ./provenance_repository | To specify the default path of NiFi data provenance . |
| 2 | nifi.provenance.repository.max.storage.time | 24 hours | To specify the maximum retention time of NiFi data provenance. |
| 3 | nifi.provenance.repository.max.storage.size | 1 GB | To specify the maximum storage of NiFi data provenance. |
| 4 | nifi.provenance.repository.rollover.time | 30 secs | To specify the rollover time of NiFi data provenance. |
| 5 | nifi.provenance.repository.rollover.size | 100 MB | To specify the rollover size of NiFi data provenance. |
| 6 | nifi.provenance.repository.indexed.fields | EventType, FlowFileUUID, Filename, ProcessorID, Relationship | To specify the fields used to search and index NiFi data provenance. |