feat(analytics): add MLflow integration for MLOps provenance tracking #5673

vellanki-santhosh · 2026-01-24T17:11:45Z

Summary

This PR adds MLflow Analytics integration to Flowise, enabling users to automatically log LLM chain executions, inputs/outputs, metrics, and latency to an MLflow tracking server.

Closes #5669

Changes

New Files

packages/components/credentials/MLflowApi.credential.ts - Credential definition supporting:
- Tracking URI
- Bearer token authentication (Databricks, managed MLflow)
- Basic auth (username/password)
packages/components/nodes/analytic/MLflow/MLflow.ts - Analytics node for the Flowise canvas
packages/components/nodes/analytic/MLflow/mlflow.svg - Node icon
packages/ui/src/assets/images/mlflow.svg - UI icon for analytics settings

Modified Files

packages/components/package.json - Added mlflow-tracing dependency
packages/components/src/handler.ts - Added:
- MLflowClient class with REST API integration
- MLflow provider handling in AnalyticHandler class
- Chain/LLM/Tool event tracking
packages/ui/src/ui-component/extended/AnalyseFlow.jsx - Added MLflow to analytics providers

Implementation Details

This integration uses a custom MLflowClient class that communicates directly with the MLflow REST API. This approach:

Is thread-safe - Each client instance maintains its own auth headers (no global process.env pollution)
Supports all MLflow backends - Works with local MLflow, Databricks, and any MLflow-compatible server
Tracks comprehensive data:
- Creates experiments and runs automatically
- Logs inputs/outputs as tags
- Records metrics: duration_ms, success, llm_calls, tool_calls
- Logs parameters: chain_name, session_id

How to Test

1. Setup MLflow Server

pip install mlflow
mlflow ui  # Starts server at http://127.0.0.1:5000

2. Configure Flowise

pnpm install
pnpm build
pnpm start

3. Enable MLflow Analytics

Open Flowise UI → Create/Edit a Chatflow
Go to Settings (gear icon) → Analytic
Expand MLflow section
Click Create New credential:
- Set Tracking URI: http://127.0.0.1:5000
- (Optional) Add auth credentials
Set Experiment Name: my-flowise-experiment
Toggle On/Off to enable
Save

4. Verify

Send a message in the chatflow
Open MLflow UI at http://127.0.0.1:5000
Check the experiment for new runs with logged data

Checklist

Code follows Flowise patterns (INode, INodeCredential interfaces)
No TypeScript errors
Authentication support (Bearer token + Basic auth)
Icons added for node and UI
Thread-safe implementation (no global state)

- Add MLflowApi credential with support for Bearer token and Basic auth - Add MLflow analytic node for Flowise canvas - Add MLflowClient class using MLflow REST API for thread-safe tracking - Add MLflow to AnalyticHandler for chain/LLM/tool event tracking - Add MLflow to UI analytics providers - Add mlflow-tracing dependency Closes FlowiseAI#5669

gemini-code-assist · 2026-01-24T17:12:04Z

Summary of Changes

Hello @vellanki-santhosh, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request significantly enhances Flowise's MLOps capabilities by integrating MLflow Analytics. It provides a robust framework for monitoring, comparing, and reproducing LLM experiments by automatically tracking chain executions, inputs, outputs, and performance metrics. This integration streamlines the development and deployment lifecycle of AI applications within Flowise, offering better visibility and control over experimental results.

Highlights

MLflow Integration: Introduces comprehensive MLflow Analytics integration to Flowise, enabling users to automatically log LLM chain executions, inputs/outputs, metrics, and latency to an MLflow tracking server.
Flexible Authentication: Adds a new MLflow API credential type supporting Tracking URI, Bearer token authentication (for Databricks/managed MLflow), and basic authentication (username/password).
Dedicated UI Components: Includes a new MLflow analytics node for the Flowise canvas and integrates MLflow into the analytics settings UI for easy configuration.
Custom MLflow Client: Implements a custom, thread-safe MLflowClient class that communicates directly with the MLflow REST API, ensuring compatibility with various MLflow backends (local, Databricks, etc.).
Comprehensive Data Tracking: Automatically creates experiments and runs, logs inputs/outputs as tags, and records metrics such as duration_ms, success, llm_calls, and tool_calls, along with parameters like chain_name and session_id.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	`/gemini review`	Performs a code review for the current pull request in its current state.
Pull Request Summary	`/gemini summary`	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	`/gemini help`	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

gemini-code-assist

Code Review

This pull request introduces MLflow integration for analytics, which is a valuable addition for MLOps provenance tracking. The implementation includes a new MLflowClient for direct communication with the MLflow REST API and its integration within the AnalyticHandler. The code is generally well-structured. However, I've identified a few issues that should be addressed: an unused dependency, a potential injection vulnerability in experiment name handling, an error handling improvement opportunity, and a bug in how metrics are counted. My review comments provide specific details and suggestions for these points.

gemini-code-assist · 2026-01-24T17:14:12Z

packages/components/src/handler.ts

+        try {
+            // Try to get or create the experiment
+            const searchResult = await this.fetch('/experiments/search', 'POST', {
+                filter: `name = '${this.experimentName}'`


The experimentName is used directly in the filter string for searching experiments. If an experiment name contains a single quote ('), it will break the filter's SQL-like syntax, leading to API errors. This poses a minor injection risk. The experiment name should be properly escaped to prevent this.

Suggested change

filter: `name = '${this.experimentName}'`

filter: `name = '${this.experimentName.replace(/'/g, "''")}'`

gemini-code-assist · 2026-01-24T17:14:12Z

packages/components/package.json

        "lunary": "^0.7.12",
        "mammoth": "^1.5.1",
        "meilisearch": "^0.41.0",
+        "mlflow-tracing": "^0.1.2",


The mlflow-tracing dependency is added but it doesn't appear to be used. The MLflow integration is implemented via a custom MLflowClient that uses the REST API directly. To keep dependencies clean and avoid unnecessary package bloat, this unused dependency should be removed.

gemini-code-assist · 2026-01-24T17:14:12Z

packages/components/src/handler.ts

+        } catch (err) {
+            if (process.env.DEBUG === 'true') console.error(`Error initializing MLflow experiment: ${err}`)
+        }


The catch block in the init method swallows errors, only logging them when in debug mode. If init fails to get or create an experiment, this.experimentId remains null, and subsequent operations like startRun will fail silently by returning null, obscuring the root cause. It's better to propagate the error from init so the caller can handle the initialization failure explicitly.

} catch (err) { if (process.env.DEBUG === 'true') console.error(`Error initializing MLflow experiment: ${err}`) throw err }

gemini-code-assist · 2026-01-24T17:14:12Z

packages/components/src/handler.ts

+
+            if (mlflow && chainRunId) {
+                // Log LLM call as a metric on the parent chain run
+                await mlflow.logMetric(chainRunId, 'llm_calls', 1)


The llm_calls metric is logged with a value of 1 for every LLM call. Since no step is provided to logMetric, it defaults to 0. This causes each new metric to overwrite the previous one within the same run. Consequently, llm_calls will always be 1 at the end of the run, regardless of the actual number of calls. To correctly count the calls, you should provide a unique or incremental step for each metric. Using Date.now() as the step is a simple way to record each call individually. This same issue applies to tool_calls.

await mlflow.logMetric(chainRunId, 'llm_calls', 1, Date.now())

gemini-code-assist · 2026-01-24T17:14:12Z

packages/components/src/handler.ts

+
+            if (mlflow && chainRunId) {
+                // Log tool call as a metric on the parent chain run
+                await mlflow.logMetric(chainRunId, 'tool_calls', 1)


The tool_calls metric is logged with a value of 1 for every tool call. Similar to llm_calls, the absence of a step argument causes each log to overwrite the previous one. To accurately track the number of tool calls, a unique step should be provided for each metric. Using Date.now() will ensure each call is recorded as a distinct data point.

await mlflow.logMetric(chainRunId, 'tool_calls', 1, Date.now())

… improved metrics - Add tokenCount tracking to activeRuns Map for token usage metrics - Add logParams() and logMetrics() batch methods for efficiency - Add setTags() batch method - Add addTokenUsage() for tracking token usage during runs - Add logChainExecution() comprehensive logging helper - endRun() now automatically logs: latency_ms, token_usage (if tracked), success (0/1) - Updated comments to clarify thread-safety (per-instance headers, no global process.env) - onChainStart now logs chain_id, chain_name, session_id as params - Use standard mlflow.note.content tag for inputs

- Fix llm_calls and tool_calls metrics to use Date.now() as step for proper counting - Escape single quotes in experiment name to prevent filter injection - Propagate init errors instead of silently swallowing them - Remove unused mlflow-tracing dependency

…etrics Major improvements to MLflow integration: ## New Features - Token usage tracking: Extract promptTokens, completionTokens, totalTokens from LLM responses - Batch logging: logBatch() method reduces HTTP requests by combining metrics, params, tags - Enhanced run info: Track llmCalls and toolCalls counts per run - incrementLLMCalls() and incrementToolCalls() for accumulating call counts - getRunInfo() for accessing current run state ## Bug Fixes (from code review) - Data overwrite bug: Use Date.now() as step for llm_calls/tool_calls metrics - SQL injection: Escape single quotes in experiment name filter - Error handling: Propagate init() errors instead of swallowing them ## Metrics Logged Final batch at chain end includes: - latency_ms: Total execution time - success: 1 for FINISHED, 0 for FAILED - total_tokens, prompt_tokens, completion_tokens: Token usage for cost analysis - total_llm_calls, total_tool_calls: Final call counts Per-call metrics (with unique timestamps): - llm_calls: Each LLM invocation - tool_calls: Each tool invocation ## Thread-Safety - All state is per-instance (activeRuns Map, headers, experimentId) - No global process.env usage for configuration

vellanki-santhosh · 2026-01-24T17:35:27Z

MLflow Analytics for Flowise

This integration allows you to log traces, metrics, and parameters from your Flowise chains directly to an MLflow tracking server.

Features

Tracing: Logs inputs, outputs, and intermediate chain steps.
Metrics: Automatically tracks token_usage (total/prompt/completion), latency, llm_calls, and tool_calls.
Tags: Logs session_id, chatflow_id, and agent_type.
Authentication: Supports Basic Auth (User/Pass) and Bearer Tokens (Databricks/Azure).

Setup Guide

Prerequisite: You must have an MLflow server running (e.g., mlflow ui or a hosted instance).
Add Node: In Flowise, drag the MLflow node from the Analytics category.
Connect: Connect it to the Analytics input of your Chain or Agent.
Configuration:
- Tracking URI: The URL of your server (e.g., http://localhost:5000).
- Experiment Name: The name of the experiment bucket (e.g., Chatbot_Prod).
Authentication (Optional):
- Click Connect Credential.
- Select MLflow API.
- Enter your Username/Password OR Tracking Token.

Troubleshooting

Docker: If running Flowise in Docker and MLflow on host, use http://host.docker.internal:5000.
Missing Tokens: Not all LLMs return token usage data. Ensure your LLM provider supports it.

gemini-code-assist bot reviewed Jan 24, 2026

View reviewed changes

vellanki-santhosh added 3 commits January 24, 2026 17:17

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat(analytics): add MLflow integration for MLOps provenance tracking #5673

feat(analytics): add MLflow integration for MLOps provenance tracking #5673

vellanki-santhosh commented Jan 24, 2026

Uh oh!

gemini-code-assist bot commented Jan 24, 2026

Uh oh!

gemini-code-assist bot left a comment

Uh oh!

gemini-code-assist bot Jan 24, 2026

Uh oh!

gemini-code-assist bot Jan 24, 2026

Uh oh!

gemini-code-assist bot Jan 24, 2026

Uh oh!

gemini-code-assist bot Jan 24, 2026

Uh oh!

gemini-code-assist bot Jan 24, 2026

Uh oh!

vellanki-santhosh commented Jan 24, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

	filter: `name = '${this.experimentName}'`
	filter: `name = '${this.experimentName.replace(/'/g, "''")}'`

Uh oh!

feat(analytics): add MLflow integration for MLOps provenance tracking #5673

Are you sure you want to change the base?

feat(analytics): add MLflow integration for MLOps provenance tracking #5673

Conversation

vellanki-santhosh commented Jan 24, 2026

Summary

Changes

New Files

Modified Files

Implementation Details

How to Test

1. Setup MLflow Server

2. Configure Flowise

3. Enable MLflow Analytics

4. Verify

Checklist

Uh oh!

gemini-code-assist bot commented Jan 24, 2026

Summary of Changes

Highlights

Footnotes

Uh oh!

gemini-code-assist bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

gemini-code-assist bot Jan 24, 2026

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist bot Jan 24, 2026

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist bot Jan 24, 2026

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist bot Jan 24, 2026

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist bot Jan 24, 2026

Choose a reason for hiding this comment

Uh oh!

vellanki-santhosh commented Jan 24, 2026

MLflow Analytics for Flowise

Features

Setup Guide

Troubleshooting

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant