Deprecate Mllama support in llm_ptq/vlm_ptq examples

[cjluo-nv]

Summary

• Removes Mllama (Llama 3.2 Vision) model-type branches from the llm_ptq example (hf_ptq.py, example_utils.py) and drops the now-unused MllamaImageProcessor wrapper from modelopt/torch/utils/.
• Drops the legacy MllamaImageProcessor path in modelopt/torch/utils/vlm_dataset_utils.py; the generic HF ProcessorMixin path handles the remaining cases.
• Adds a CHANGELOG entry under 0.44 Backward Breaking Changes.

Test plan

• CI lint / unit tests pass
• Smoke-run examples/llm_ptq/scripts/huggingface_example.sh --model <llm> --quant fp8 (text-only path, non-mllama)

Summary by CodeRabbit

• Chores
  • Removed Mllama (Llama 3.2 Vision) support from quantization examples. This includes removal of dedicated image processor implementation, specialized model handling, and related calibration logic.
  • Updated VLM image-text calibration guidance to use --calib_with_images flag with other supported VLMs instead of Mllama-specific processing paths.

[coderabbitai]

📝 Walkthrough

Walkthrough

This pull request removes Mllama (Llama 3.2 Vision) support from quantization examples and utilities by deleting the image processor abstraction, eliminating model-specific handling in example code, and updating documentation to direct users toward alternative approaches.

Changes

Cohort / File(s)	Summary
Release Notes CHANGELOG.rst	Added deprecation notice marking Mllama support as removed from llm_ptq and vlm_ptq examples, with guidance to use --calib_with_images with supported VLMs instead.
Image Processor Abstraction Removal modelopt/torch/utils/image_processor.py	Deleted entire file containing BaseImageProcessor and MllamaImageProcessor classes that provided image/text prompt construction and batch collation functionality.
Example Utilities examples/llm_ptq/example_utils.py	Removed device parameter from get_processor() function signature and eliminated MllamaImageProcessor import and instantiation logic; function now returns generic ProcessorMixin | None.
Quantization Example examples/llm_ptq/hf_ptq.py	Stripped Mllama-specific calibration dataloader branch, processor initialization, TensorRT-LLM export config rewriting, and special-case handling in input/output decoding; narrowed processor typing to ProcessorMixin.
Dataset Utilities modelopt/torch/utils/vlm_dataset_utils.py	Removed specialized dataloader construction branch for MllamaImageProcessor, consolidating to generic path that defers tokenization/image processing to batch collation.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

🚥 Pre-merge checks | ✅ 5 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 25.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (5 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly and concisely summarizes the primary change: deprecating Mllama support in llm_ptq/vlm_ptq examples, which aligns with the main objective of removing Mllama-specific code branches and helpers.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Security Anti-Patterns	✅ Passed	PR removes Mllama-specific code with no new security anti-patterns introduced; all changes maintain security hygiene standards.
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch chenjiel/deprecate_mllama

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[github-actions]

PR Preview Action v1.8.1
Preview removed because the pull request was closed.
2026-04-23 17:54 UTC

[coderabbitai]

🧹 Nitpick comments (1)

CHANGELOG.rst (1)

22-22: Prefer “Removed” over “Deprecated” in this breaking-change note.

This line says support is deprecated but also says the code paths were removed. Using “Removed” avoids ambiguity for users upgrading.

Proposed wording tweak

-- Deprecated Mllama (Llama 3.2 Vision) support in the ``llm_ptq`` and ``vlm_ptq`` examples.
+- Removed Mllama (Llama 3.2 Vision) support in the ``llm_ptq`` and ``vlm_ptq`` examples.

🤖 Prompt for AI Agents

Verify each finding against the current code and only fix it if needed.

In `@CHANGELOG.rst` at line 22, The changelog wording is ambiguous — it says
"Deprecated" but the code branches (model_type == "mllama") and uses of
MllamaImageProcessor were removed; update the sentence in CHANGELOG.rst to use
"Removed" instead of "Deprecated" and keep the remaining details intact
(mentioning removal from hf_ptq.py and example_utils.py, and advising use of
--calib_with_images with a supported VLM per the Nemotron VL section in
examples/llm_ptq/README.md) so users understand the functionality was removed,
not merely deprecated.

🤖 Prompt for all review comments with AI agents

Verify each finding against the current code and only fix it if needed.

Nitpick comments:
In `@CHANGELOG.rst`:
- Line 22: The changelog wording is ambiguous — it says "Deprecated" but the
code branches (model_type == "mllama") and uses of MllamaImageProcessor were
removed; update the sentence in CHANGELOG.rst to use "Removed" instead of
"Deprecated" and keep the remaining details intact (mentioning removal from
hf_ptq.py and example_utils.py, and advising use of --calib_with_images with a
supported VLM per the Nemotron VL section in examples/llm_ptq/README.md) so
users understand the functionality was removed, not merely deprecated.

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: a80fda8b-b46e-4839-9383-4bb9040e48a7

📥 Commits

Reviewing files that changed from the base of the PR and between 8663678 and c5561f9.

📒 Files selected for processing (5)

• CHANGELOG.rst
• examples/llm_ptq/example_utils.py
• examples/llm_ptq/hf_ptq.py
• modelopt/torch/utils/image_processor.py
• modelopt/torch/utils/vlm_dataset_utils.py

💤 Files with no reviewable changes (2)

• modelopt/torch/utils/vlm_dataset_utils.py
• modelopt/torch/utils/image_processor.py

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 69.06%. Comparing base (4e33368) to head (c5561f9).
⚠️ Report is 30 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1332      +/-   ##
==========================================
- Coverage   72.74%   69.06%   -3.68%     
==========================================
  Files         459      476      +17     
  Lines       48611    54931    +6320     
==========================================
+ Hits        35361    37938    +2577     
- Misses      13250    16993    +3743     

Flag	Coverage Δ
examples	41.39% <ø> (+1.95%)	⬆️
gpu	51.73% <ø> (-0.47%)	⬇️
unit	52.55% <ø> (+0.34%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[meenchen]

Clean deprecation PR. Removes Mllama-specific code paths from llm_ptq/vlm_ptq examples and deletes the now-unused MllamaImageProcessor class. All references to the removed imports and model_type == "mllama" branches are consistently cleaned up across the touched files. The remaining mllama references in modelopt/torch/export/ are for TRT-LLM export config mapping and are outside this PR's scope. CHANGELOG entry is present. Small, focused, and straightforward.

[shengliangxu]

good