[Qualcomm] Support native_layer_norm and affine-free LayerNorm in QNN backend by KevinUW114514 · Pull Request #18990 · pytorch/executorch

KevinUW114514 · 2026-04-19T07:29:09Z

[Qualcomm] Support native_layer_norm and affine-free LayerNorm in QNN backend

Summary

Adds QNN backend support for aten.native_layer_norm.default (which is the decomposed form of torch.nn.LayerNorm) and handles models where weight/bias are not provided (elementwise_affine=False).

Problem

When exporting models with torch.native_layer_norm or torch.nn.LayerNorm(affine=False) to the QNN backend, the following issues occur:

Missing native_layer_norm visitor: The original LayerNormVisitor only targets aten.layer_norm.default, but PyTorch decomposes torch.nn.LayerNorm to aten.native_layer_norm.default during export.
None weight/bias: When elementwise_affine=False, the weight and bias arguments are None. QNN x86_64 runtime cannot handle None tensor inputs, causing AttributeError when calling get_parameter().

Solution

1. Update visitor target (`op_layer_norm.py`)

Change the visitor target from aten.layer_norm.default to aten.native_layer_norm.default:

# Before
target = ["aten.layer_norm.default"]

# After
target = ["aten.native_layer_norm.default"]

This is correct because during ExecuTorch export, aten.layer_norm.default is decomposed to aten.native_layer_norm.default before the QNN lowering stage.

2. Handle None weight/bias (`op_layer_norm.py`)

When weight/bias are None, create synthetic tensors:

Missing weight → torch.ones(normalized_shapes) (identity transform)
Missing bias → torch.zeros(normalized_shapes) (no offset)

Create synthetic fx.Node objects to register these as QNN static tensors:

weight_tensor = torch.ones(normalized_shapes, dtype=torch.float32)
weight_node = torch.fx.Node(
    node.graph,
    node.name + "_runtime_weight",
    "call_function",
    exir_ops.edge.aten.tensor.default,
    (),
    {},
)
# Preserve quant_attrs with zero_point=0 for QNN compatibility

3. Use same annotator for both ops (`htp_rules.py`)

The quantizer annotator registers both aten.layer_norm.default and aten.native_layer_norm.default to the same LayerNorm class, since both ops have identical argument schemas:

@register_annotator(
    [torch.ops.aten.layer_norm.default, torch.ops.aten.native_layer_norm.default],
    QnnConstants.OpLayerNorm.op_name,
)

4. Add None check to `get_parameter()` (`utils.py`)

Guard against None nodes to prevent AttributeError:

if node is None:
    return None

Files Changed

File	Changes
`builders/op_layer_norm.py`	Add `native_layer_norm` support + handle None weight/bias
`builders/utils.py`	Add None guard in `get_parameter()`
`quantizer/annotators/htp_rules.py`	Register annotator for both ops
`tests/models.py`	Add `NativeLayerNorm` test model
`tests/test_qnn_delegate.py`	Add floating-point and quantized tests

Test Plan

Run QNN delegate tests for layer_norm:

python backends/qualcomm/tests/test_qnn_delegate.py \
    -k "test_qnn_backend_layer_norm or test_qnn_backend_native_layer_norm" \
    --soc_model SM8650 \
    --build_folder build-x86/ \
    --executorch_root . \
    --enable_x86_64

Expected: 4 tests pass (2 floating-point, 2 quantized).

Release Notes

Release notes: qualcomm

Related Issues

This resolves the issue where FLUX2 transformer export fails with:

[QNN Delegate Op Builder]: LayerNorm weight is None, skipping
AttributeError: 'NoneType' object has no attribute 'name'

Fixes #18989

Labels: bug, module:qnn

@abhinaykukkadapu

pytorch-bot · 2026-04-19T07:29:12Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/18990

📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❗ 1 Active SEVs

There are 1 currently active SEVs. If your PR is affected, please view them below:

Rolling out OSDC (ARC) runners on pull & trunk workflows in PyTorch main

This comment was automatically generated by Dr. CI and updates every 15 minutes.

meta-cla · 2026-04-19T07:29:14Z

Hi @KevinUW114514!

Thank you for your pull request and welcome to our community.

Action Required

In order to merge any pull request (code, docs, etc.), we require contributors to sign our Contributor License Agreement, and we don't seem to have one on file for you.

Process

In order for us to review and merge your suggested changes, please sign at https://code.facebook.com/cla. If you are contributing on behalf of someone else (eg your employer), the individual CLA may not be sufficient and your employer may need to sign the corporate CLA.

Once the CLA is signed, our tooling will perform checks and validations. Afterwards, the pull request will be tagged with CLA signed. The tagging process may take up to 1 hour after signing. Please give it that time before contacting us about it.

If you have received this in error or have any questions, please contact us at cla@meta.com. Thanks!

KevinUW114514 · 2026-04-19T07:31:15Z

@pytorchbot label "release notes: none"

meta-cla · 2026-04-19T07:31:22Z

Thank you for signing our Contributor License Agreement. We can now accept your code for this (and any) Meta Open Source project. Thanks!

Copilot

Pull request overview

Fixes a crash in the Qualcomm QNN PT2E quantizer by making _mark_nodes_as_annotated robust to None entries in node lists (e.g., when aten.layer_norm has optional affine args like weight=None).

Changes:

Skip None entries in _mark_nodes_as_annotated to avoid AttributeError when accessing node.meta.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@@ -29,6 +29,8 @@

 def _mark_nodes_as_annotated(nodes: List[Node]):


+        if node is None:
+            continue


abhinaykukkadapu · 2026-04-20T01:43:24Z

Hi @KevinUW114514 thank you for your contribution. I think the root cause is that we need to guard weight and bias creation in rules files for htp and lpai similar to #18219, let me know if you are willing to change it. Adding the guard might silently propagate bad configs like these in the pipeline and i think we should fail loudly. CC: @shewu-quic

KevinUW114514 · 2026-04-20T01:49:10Z

Hi @abhinaykukkadapu , thanks for the follow-up! Actually I also realized this root issue as I encountered the error in my downstream tasks. I am currently working on fixing this. I can edit the issue and PR to re-state the issue and submit a complete fix for it. Let me know if any concern. Thank you!

abhinaykukkadapu · 2026-04-20T03:01:02Z

Hi @abhinaykukkadapu , thanks for the follow-up! Actually I also realized this root issue as I encountered the error in my downstream tasks. I am currently working on fixing this. I can edit the issue and PR to re-state the issue and submit a complete fix for it. Let me know if any concern. Thank you!

Thanks, that would be awesome, will look forward to your changes.

Fixes AttributeError when aten.native_layer_norm has optional weight=None. Both weight and bias are guarded to handle the None case gracefully. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

… backend - add QNN layer norm support for aten.native_layer_norm.default - handle missing weight/bias by creating identity weight and zero bias - always provide bias tensor for QNN LayerNorm op - add floating-point and quantized tests for native_layer_norm - print generated pte filename after export

KevinUW114514 · 2026-04-22T21:05:03Z

Hi @abhinaykukkadapu and @shewu-quic , thanks for your help before! Could you please take a look at the PR to see whether I am doing the right fix on this? I appreciate your time 😃

abhinaykukkadapu · 2026-04-23T03:53:31Z


 def _mark_nodes_as_annotated(nodes: List[Node]):
    for node in nodes:
+        if node is None:


We might want to get rid of this, CC: @shewu-quic

I think the node should not be None in this function.

abhinaykukkadapu · 2026-04-23T03:54:09Z

@KevinUW114514 LGTM, will wait for a stamp from @shewu-quic too

Copilot

Pull request overview

This PR adds Qualcomm QNN backend support for aten.native_layer_norm.default (the decomposed form of torch.nn.LayerNorm) and improves robustness when optional weight/bias inputs are None (e.g., elementwise_affine=False).

Changes:

Update the QNN op builder to target aten.native_layer_norm.default and synthesize identity weight / zero bias when missing.
Make quantizer annotation/marking logic resilient to optional None nodes and register the HTP annotator for both layer_norm and native_layer_norm.
Add new test model + delegate tests intended to cover native layer norm (float + quantized).

Reviewed changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
backends/qualcomm/builders/op_layer_norm.py	Switch visitor target to native_layer_norm and add synthetic weight/bias handling for optional inputs.
backends/qualcomm/builders/utils.py	Allow `get_parameter()` to safely handle `None` inputs.
backends/qualcomm/quantizer/rules.py	Skip `None` entries when marking nodes as annotated.
backends/qualcomm/quantizer/annotators/htp_rules.py	Register LayerNorm annotator for both `layer_norm` and `native_layer_norm`; avoid annotating missing optional args.
backends/qualcomm/quantizer/annotators/lpai_rules.py	Make LayerNorm annotator tolerant of missing optional args (but still only registered for `layer_norm`).
backends/qualcomm/tests/models.py	Add `NativeLayerNorm` test module.
backends/qualcomm/tests/test_qnn_delegate.py	Add float + quantized tests for `NativeLayerNorm`.
backends/qualcomm/export_utils.py	Print a success message after writing the generated `.pte`.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

+        self.weight = torch.nn.Parameter(torch.ones(768))
+        self.bias = torch.nn.Parameter(torch.zeros(768))
+        self.normalized_shape = [768]
+        self.eps = 1e-6
+
+    def forward(self, x):
+        if self.affine:
+            return torch.native_layer_norm(
+                x, self.normalized_shape, self.weight, self.bias, self.eps
+            )[0]
+        else:
+            return torch.native_layer_norm(
+                x, self.normalized_shape, self.weight, self.bias, self.eps
+            )[0]


+        for i, module in enumerate(modules):
+            with self.subTest(i=i):
+                self.lower_module_and_test_output(module, sample_input)
+


@@ -29,7 +29,9 @@ def is_parameter(

 def get_parameter(
    node: torch.fx.Node, edge_program: torch.export.ExportedProgram


    @staticmethod
    def annotate(node: Node, quantization_config: QuantizationConfig) -> None:
        act_node = node.args[0]
-        weight_node = node.args[2]
-        bias_node = None
-        if len(node.args) > 2:
-            bias_node = node.args[3]
+        weight_node = node.args[2] if len(node.args) > 2 else None
+        bias_node = node.args[3] if len(node.args) > 3 else None


shewu-quic

Thank you for your effort.

shewu-quic · 2026-04-24T02:01:13Z

+        self.eps = 1e-6
+
+    def forward(self, x):
+        if self.affine:


These two branches seem to be the same. Would it be possible to extend the current LayerNorm with torch.nn.LayerNorm(elementwise_affine=False) as a test case?

shewu-quic · 2026-04-24T06:22:44Z

-
-        bias_node = self.get_node(node.args[3])
-        if bias_node is not None:
+        # Fake node: even when original bias is absent, QNN still needs it


I think the bias is optional for QNN and can be kept as in the original design.
https://docs.qualcomm.com/doc/80-63442-10/topic/MasterOpDef.html#layernorm

shewu-quic · 2026-04-24T06:24:45Z

 def get_parameter(
    node: torch.fx.Node, edge_program: torch.export.ExportedProgram
-) -> torch.Tensor:
+) -> Optional[torch.Tensor]:


This function shouldn't return None. Perhaps we should ensure that the node is not None before this function is called.

shewu-quic · 2026-04-24T06:26:58Z


 def _mark_nodes_as_annotated(nodes: List[Node]):
    for node in nodes:
+        if node is None:


I think the node should not be None in this function.

Fix AttributeError in _mark_nodes_as_annotated when node is None

ee9a981

Copilot AI review requested due to automatic review settings April 19, 2026 07:29

KevinUW114514 requested a review from abhinaykukkadapu as a code owner April 19, 2026 07:29

Copilot started reviewing on behalf of KevinUW114514 April 19, 2026 07:29 View session

pytorch-bot Bot added the release notes: none Do not include this in the release notes label Apr 19, 2026

meta-cla Bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Apr 19, 2026

Copilot AI reviewed Apr 19, 2026

View reviewed changes

Comment thread backends/qualcomm/quantizer/rules.py

@@ -29,6 +29,8 @@

def _mark_nodes_as_annotated(nodes: List[Node]):

Comment thread backends/qualcomm/quantizer/rules.py

Comment on lines +32 to +33

if node is None:

continue

KevinUW114514 changed the title ~~Fix AttributeError in _mark_nodes_as_annotated when node is None~~ [QNN] Fix AttributeError in _mark_nodes_as_annotated when node is None Apr 20, 2026

[QNN] Guard get_parameter against node=None in LayerNormVisitor

40404fd

Fixes AttributeError when aten.native_layer_norm has optional weight=None. Both weight and bias are guarded to handle the None case gracefully. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

nil-is-all assigned abhinaykukkadapu Apr 20, 2026

nil-is-all added the module: qnn Issues related to Qualcomm's QNN delegate and code under backends/qualcomm/ label Apr 20, 2026

KevinUW114514 changed the title ~~[QNN] Fix AttributeError in _mark_nodes_as_annotated when node is None~~ [Qualcomm] Support native_layer_norm and affine-free LayerNorm in QNN backend Apr 22, 2026

abhinaykukkadapu reviewed Apr 23, 2026

View reviewed changes

Fix QNN LayerNorm optional arg annotation

0a2c42c

Copilot AI review requested due to automatic review settings April 24, 2026 01:47

Copilot started reviewing on behalf of KevinUW114514 April 24, 2026 01:47 View session

Copilot AI reviewed Apr 24, 2026

View reviewed changes

shewu-quic reviewed Apr 24, 2026

View reviewed changes

		@@ -29,6 +29,8 @@

		def _mark_nodes_as_annotated(nodes: List[Node]):

		@@ -29,7 +29,9 @@ def is_parameter(

		def get_parameter(
		node: torch.fx.Node, edge_program: torch.export.ExportedProgram

Conversation

KevinUW114514 commented Apr 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

[Qualcomm] Support native_layer_norm and affine-free LayerNorm in QNN backend

Summary

Problem

Solution

1. Update visitor target (op_layer_norm.py)

2. Handle None weight/bias (op_layer_norm.py)

3. Use same annotator for both ops (htp_rules.py)

4. Add None check to get_parameter() (utils.py)

Files Changed

Test Plan

Release Notes

Related Issues

Uh oh!

pytorch-bot Bot commented Apr 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/18990

❗ 1 Active SEVs

Uh oh!

meta-cla Bot commented Apr 19, 2026

Action Required

Process

Uh oh!

KevinUW114514 commented Apr 19, 2026

Uh oh!

meta-cla Bot commented Apr 19, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

abhinaykukkadapu commented Apr 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

KevinUW114514 commented Apr 20, 2026

Uh oh!

abhinaykukkadapu commented Apr 20, 2026

Uh oh!

KevinUW114514 commented Apr 22, 2026

Uh oh!

abhinaykukkadapu Apr 23, 2026

Choose a reason for hiding this comment

Uh oh!

shewu-quic Apr 24, 2026

Choose a reason for hiding this comment

Uh oh!

abhinaykukkadapu commented Apr 23, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

shewu-quic left a comment

Choose a reason for hiding this comment

Uh oh!

shewu-quic Apr 24, 2026

Choose a reason for hiding this comment

Uh oh!

shewu-quic Apr 24, 2026

Choose a reason for hiding this comment

Uh oh!

shewu-quic Apr 24, 2026

Choose a reason for hiding this comment

Uh oh!

shewu-quic Apr 24, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

KevinUW114514 commented Apr 19, 2026 •

edited

Loading

1. Update visitor target (`op_layer_norm.py`)

2. Handle None weight/bias (`op_layer_norm.py`)

3. Use same annotator for both ops (`htp_rules.py`)

4. Add None check to `get_parameter()` (`utils.py`)

pytorch-bot Bot commented Apr 19, 2026 •

edited

Loading

abhinaykukkadapu commented Apr 20, 2026 •

edited

Loading