Push for BasicBertSequenceClassification_basic E2E

[silvasean]

@gpetters94 @dan-garvey @cathyzhyi (pls add others)

Creating this bug to centralize task tracking on getting BasicBertSequenceClassification_basic working after #341 lands. BERT is a milestone model to run -- let's do this!

@cathyzhyi can you post an updated todo op list here? Folks please indicate which ops/bugs you are working on so we don't step on each other's toes :)

[cathyzhyi]

I summarized the ops that need to be implemented from the IR output into this list. Please update the table accordingly when starting to implement an op.

For the status column:
- unimplemented
+ started
X finished

num	op	status	owner	notes
1	aten::dropout	X	dan-garvey	PR398
2	aten::embedding	X	cathyzhyi	PR362
3	aten::gelu	X	sogartar	PR374
4	aten::layer_norm	X	cathyzhyi	PR346
5	aten::matmul	X	pashu123	PR367
6	aten::softmax.int	X	cathyzhyi	PR362
7	aten::permute	X	gprateek93	PR376
8	aten::rsub.Scalar	X	gpetters94	PR375
9	aten::slice.Tensor	X	dan-garvey	PR398
10	aten::select.int	X	dan-garvey	PR398
11	aten::view	X	Shukla-Gaurav	PR384
12	aten::ones	X	gpetters94	PR366
13	aten::size.int	X	cathyzhyi	PR362
14	aten::contiguous	X	gpetters94	PR373
15	aten::Int.Tensor	X	pashu123	PR387
16	aten::to.dtype	X	gprateek93	PR392
17	prim::NumToTensor.Scalar	X	gpetters94	PR393
18	aten::expand	X	Shukla-Gaurav	PR396

Tasks:

num	task	status	owner	notes
1	adding a pass DecomposeComplexOps	X	cathyzhyi	PR362

[gpetters94]

Okay, I'll take ones.

[cathyzhyi]

@gpetters94 Thanks! A kind reminder that you could edit the table with your id directly as well.

[silvasean]

quick note: for softmax, we probably want to decompose it into softmax(x, dim) => tmp=exp(x); tmp / reduce(tmp, dim, keepdim=True)
https://pytorch.org/docs/stable/generated/torch.nn.functional.softmax.html

I would recommend adding a pass lib/Dialect/Torch/Transforms/DecomposeComplexOps.cpp (there are many others like this -- and eventually we will want to automate this, such as by deriving the patterns there from python snippets like def softmax(x, dim): tmp = torch.exp(x); return torch.reduce(tmp, dim, keepdim=True))

[sogartar]

Hi, I recently joined nod.ai's team. I will implement the GELU activation.

[cathyzhyi]

@sogartar Welcome! Thanks for contributing! FYI, you can update the table entry directly with your github id.

[sogartar]

@cathyzhyi, I can edit your comment. Is this done in some other manner than the usual way?

[cathyzhyi]

@sogartar @gpetters94 sry I didn't realize you folks can't edit the comment. I moved the list to a new issue here cathyzhyi/torch-mlir-ops-tracking#1. Let's use that list for now. I've sent you the invitations for that repo.

[cathyzhyi]

@sogartar @gpetters94 you folks should be able to update the table in #342 (comment) now.

[Shukla-Gaurav]

Hi, I recently joined nod.ai's team. I will take care of aten.view op. How can I edit the list?

[pashu123]

I will take aten::matmul op.

[silvasean]

@Shukla-Gaurav @pashu123 I gave you permissions for editing the list

[gprateek93]

Hi, I recently joined nod.ai's team. I will take care of aten.permute op. Can someone provide me edit access to the list above? Thanks!

[silvasean]

Hi, I recently joined nod.ai's team. I will take care of aten.permute op. Can someone provide me edit access to the list above? Thanks!

Done. Thanks!