Skip to content

🐛 [Bug] Encountered bug when using torch.ops.aten.cat converter #1862

Closed
@gs-olive

Description

@gs-olive

Bug Description

When compiling the T5-Base Model model via the aten path, the following error is encountered:

IndexError: tuple index out of range
While executing %cat_default : [#users=1] = call_function[target=torch.ops.aten.cat.default](args = ([%arg0, %arg1, %arg2],), kwargs = {_itensor_to_tensor_meta: {<tensorrt.tensorrt.ITensor object at 0x7ff49c2fae30>: None, <tensorrt.tensorrt.ITensor object at 0x7ff49c2e6e30>: None, <tensorrt.tensorrt.ITensor object at 0x7ff49c2e1f30>: None}})

To Reproduce

Steps to reproduce the behavior:

  1. Initialize model: T5Model.from_pretrained("t5-base").eval().cuda()
  2. Initialize three input tensors, for example: torch.randint(0, 1, (1, 14), dtype=torch.int32).to("cuda") ("input_ids", "attention_mask", "decoder_input_ids")
  3. (Optional) Use the transformers tools to trace the model via: transformers.utils.fx.symbolic_trace(model, input_names=["input_ids", "attention_mask", "decoder_input_ids"])
  4. Compile the model using FX

Expected behavior

Model should compile via the aten path

Environment

  • Transformers: 4.26.1
  • Torch-TensorRT Version (e.g. 1.0.0): b3f433a
  • PyTorch Version (e.g. 1.0): 2.1.0.dev20230419+cu117
  • CPU Architecture: Intel Xeon CPU
  • OS: Ubuntu 20.04
  • How you installed PyTorch: pip
  • Build command you used: python setup.py develop
  • Are you using local sources or building from archives: local
  • Python version: 3.8.13
  • CUDA version: 11.7

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions