[Pipelines] Add propagate_error argument

[kylesayrs]

Purpose

• Add research option to allow models to be calibrated using full precision values or values that reflect quantization error

propagate_error: Optional[bool] = field(
  default=True,
  metadata={
    "help": "A True value means that the activations used to calibrate layers "
    "will reflect the error induced by the quantization/optimization of "
    "previous layers of the model. A False value means that activations will "
    "be the same as activations produced by the original, full precision base "
    "model. Deafults to True"
  },
)

Changes

• Add propagate_error argument, defaults to True
• Modify basic, sequential, and layer sequential pipelines to conditionally propagate error induced by modifiers

[github-actions]

👋 Hi! Thank you for contributing to llm-compressor. Please add the ready label when the PR is ready for review.

Note: This is required to complete the testing suite, please only add the label once the PR is code complete and local testing has been performed.

[gemini-code-assist]

Summary of Changes

Hello @kylesayrs, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request introduces a new propagate_error argument to the calibration pipelines, providing a configurable option for how activations are handled during model calibration. This allows for research into the impact of propagating quantization errors versus using full-precision activations, offering greater control over the calibration process and its effect on model performance.

Highlights

• New Configuration Option for Calibration: I've introduced a new propagate_error boolean argument within DatasetArguments, which defaults to True. This argument provides a crucial research option, allowing models to be calibrated using either full-precision values or values that reflect the quantization error induced by previous layers in the model.
• Conditional Error Propagation in Pipelines: I've modified the basic, sequential, and layer_sequential calibration pipelines to conditionally handle error propagation. When propagate_error is set to False, the model's quantization is temporarily disabled during specific calibration passes. This ensures that activations used for calibrating subsequent layers are based on the original, full-precision model, rather than accumulating quantization errors.
• Refined Calibration Pass Logic: For the sequential and layer_sequential pipelines, I've adjusted the internal calibration loop. If propagate_error is False, the intermediate activation updates for the next layer now occur during the initial calibration pass (where quantization is disabled). Consequently, the second pass, which typically captures outputs of newly compressed modules, is skipped, streamlining the process for full-precision calibration.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point in your pull request via creating an issue comment (i.e. comment on the pull request page) using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in issue comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist is currently in preview and may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments to provide feedback.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

The pull request introduces a propagate_error argument to control whether quantization errors are propagated during calibration. The changes modify the basic, sequential, and layer sequential pipelines to conditionally propagate errors based on this argument. The code generally looks good, but there are a few suggestions for improvement, such as adding comments and considering performance optimizations.

[kylesayrs]

It seems like there's an issue with this current approach of doing the propagation step with quantization enabled. Namely, for sparsification -> quantization recipes, the follow events happen:

1. At initialize() time, the quant modifier initializes modules with randomly initialized qparams
2. Because of the independent pipeline, the sparsity modifier calibrates in its own pipeline
   a. One layer is calibrated and optimized
   b. When that layer's error is propagated, quantization is enabled and runs with randomly initialized qparams

This leads to forward passes with qparams before they have been initialized. Some solutions include

1. Applying qconfigs on start
   This is unideal, as it breaks support for recipes with multiple quantization modifiers, but only one of those modifiers specifying a config
2. Removing the independent pipeline and forcing all modifiers to run in the same pipelines
   This ensures that modules are calibrated before running forward, but removes functionality of the independent pipeline
3. Initialize qparams with valid values, rather than random values
   This means that initializing a module for quantization does not break the module, requiring that the module encounter calibration before running. However, this means that quantization error will occur earlier than is required, since you're still quantizing values.
4. Add a "calibrated" flag to modules, which means that the module runs with quantization disabled until it is calibrated
   This seems to be the best solution, although it is honorous to now maintain this flag for all of our quantization modifiers