Standard Dialect: The Final Chapter

MLIR
1

Hi all,

Over the past year a lot of great effort has been made in the process of splitting and killing the standard dialect. We are now down to 10 operations (well 9 if we can land D118202)! At this point, I’d like to pull people in so that we can hash out the last few steps in finally killing this beast.

The remaining operations (barring the one killed by D118202) are:

  • AssertOp, BranchOp, CallOp, CallIndirectOp, CondBranchOp, ConstantOp(function reference), ConstantOp(unit attribute), ReturnOp, SelectOp, SwitchOp

There are a few obvious groups that stand out here:

Misc

  • AssertOp

^ Not entirely sure where this fits into anything that exists, but could be the start of something new?

Control Flow/Conditionals

  • BranchOp, CallIndirectOp, CondBranchOp, SelectOp, SwitchOp

These are all generic-ish control flow/conditional operations. We’ve talked before about a potential cf(Control Flow) dialect, and I think these would make a great fit for an initial starting set.

(Before anyone says “Hey wait a minute, SelectOp shouldn’t be there”, I’d like to point out the scf.If basically supports a region-based select, so ha… joking aside, we could also also move it to arith or somewhere else, not much of a preference)

FuncOp and Friends

  • FuncOp, ReturnOp, CallOp, ConstantOp (function reference)

These are all operations that are hardcoded to interact with FuncOp. There are a few ideas that we could explore here, for example: CallOp(and I guess conceptually ReturnOp) could be updated allow referring to any FunctionOpInterface? and moved to a hypothetical cf dialect. ConstantOp(function reference)/FuncOp/ReturnOp could be moved to what is currently SCF(likely after we clean up some weird dependencies in that dialect).

Note: FuncOp is currently builtin (but shouldn’t be)

Things that should just be removed

  • ContantOp (UnitAttr)

This is unused in-tree and it isn’t obvious what semantics are supposed to be attached to this. I would propose that we just remove this behavior altogether.

Closing

We are so close to the end, let’s make one final push and end it once and for all! It would be nice if we can brainstorm in here what we think the final steps should be.

– River

5 Likes
2

I’d consider select to be a better fit in the arith dialect really. If you think about what this is modeling, there is no control associated with select, even down to the HW it is really a simple ternary arithmetic operation.

Very supportive otherwise!

It seems OK to me to have FuncOp and associated constructs in the SCF dialect (functions are really one of the most basic structured control flow abstraction, aren’t they? ;))

3

I agree with that. Though when I was considering where to put it, the values that are selected are not constrained in any way (just the condition type). Conceptually there are many high level types that don’t lower easily to hardware instructions. Either way I can see it fitting in arithmetic just fine, happy to move it there.

– River

4

Exciting and +1 in general. I see Mehdi is already discussing a couple of the points I had on my mind.

My main feedback is that we converge on both the end state and a move schedule. Some of these ops are more in the “just ops” category and are fairly easy to move (i.e. select comes to mind). However, some of these bring a lot of C++ level entanglements that would be easier to digest if moved in isolation or a related block (with some notice). I’ll think on more concrete to suggest here as the technical discussion of details resolves.

5

Right, moving FuncOp would be an organizational nightmare. That is part of this discussion, I’d like to converge on what we think the next steps are and then plot a course to do it (which I’m more than happy to do all the work here). We have so few ops left, that it shouldn’t be too difficult for us to figure something out.

– River

6

We currently use this to signify a missing operand for operation with multiple optional operands. So it is used as a “null” value (given none type).

7

I’m definitely pro making the move and finishing this “v0” chapter of the codebase. The sooner we do it, the sooner we can have confidence in what it looks like over the long term. I think my priority is more along the lines of “no more than one nightmare per week, please” :slight_smile:

Of these, I think that the FuncOp/ReturnOp/CallOp cluster is the main one that I would choreograph separately, and maybe even see if there are affordances to migration. I’m just even thinking of the potential for weird bugs as things move (there are a lot of special carve-outs for dialect prefixes and such, iirc, and we’ve run into wrinkles there before).

IREE has a check module/dialect that seems vaguely related, perhaps at least as inspiration for the name?

We also know that there needs to be a functional assert (i.e. AssertWith that accepts and returns values), but initiative to add things to standard right now is pretty low.

8

(Assuming this means in TensorFlow) Could those not be updated to use operand_segment_sizes? Either way, I think it’d be better to have the TF related dialects define their own thing to signify that (if they want to keep doing it). It gives me some slight “undef” vibes.

– River

9

Indeed, the plan is not to torture downstream users while I sip wine from behind my screen. The goal of this is to get convergence on a plan so the discussion/push back/“but wait” can be out of the way, and we can focus on the technical aspects of a smooth transition.

Agreed. There will likely be some number of temporary hacks inserted to make a smooth transition (whatever those may be is unknown). We won’t know how to make it easier for downstream until we have some example patches on hand, but we can’t get there without agreement on a plan.

– River

10

It’s worth mentioning that FuncOp is a builtin and not in the standard dialect right now: we can likely get through all of standard without touching FuncOp.

11

Yes, thanks. Added a note to the OP. I agree that we can (from a techincal standpoint) move most of the rest of the operations without touching FuncOp, but what we do with FuncOp factors in to where the related operations should go.

– River

12

Sounds good. And FTR - I am pro sipping wine behind the screen while working on this stuff :slight_smile:

(and sorry if it sounds like I am belaboring the point: I really do want to see this stuff fixed and appreciate the work you do to make it happen and me playing on the word “nightmare” was just extending your turn of phrase, not implying intent or anything)

13

I’m +1 on a cf dialect for all of these (but am undecided on FuncOp). As mentioned elsewhere, my initial reaction is that select fits better in arith - but indeed, it does operate on AnyType and cf doesn’t seem so bad from that perspective.

Odd idea, but thinking about ops that some day might co-exist with AssertOp, would it make sense to bundle these wherever FuncOp lands? As written, it is a form of non-local exit. Other related variants sometimes take the form of specialty terminators (i.e. hypothetical AbortOp or UnreachableOp), which might make sense to be co-located with FuncOp in order to make them verify and not introduce dialect dependencies.

14

I see assert and abort quote differently than FuncOp though: interrupting the program is really a different property than structuring the flow of control.
Also assert is mostly either a debugging tool or an optimization hint: to me we’re touching on abstractions that aren’t fundamentally in the same box here.

15

Yeah, I don’t disagree and suspected you would say that. I thought it was a stretch. Maybe a debug dialect?

16

That said if we bundle FuncOp in a cf dialect with other things, I would likely be fine adding assert, unreachable, etc. there.
It would be weird to me if we created a dialect for FuncOp and the only ops alongside it would be assert & co.

1 Like
17

Okay, conceptual update based on some of the discussion above:

Arithmetic:

  • SelectOp

Control Flow (new dialect):

  • AssertOp, BranchOp, CallOp, CallIndirectOp, CondBranchOp, SwitchOp

CallOp is currently hardcoded to FuncOp, but given that CallIndirectOp is already agnostic to the callee(and that the documentation of CallOp doesn’t mention FuncOp) I think we should at least consider changing the explicit FuncOp to FunctionOpInterface. The alternative to that of course would be to move CallOp(and likely +CallIndirectOp) to SCF.

SCF:

  • ConstantOp (function reference), FuncOp, ReturnOp

To Remove:

  • ContantOp (UnitAttr)
18

Very big fan of the Control Flow dialect!

Personally wouldn’t want FuncOp and ReturnOp be in SCF as they’re very fundamental but I wouldn’t know any better. It’s also not that big of a deal.

19

TFLite side, and they were a very early adopter of this work. I’ll need to check if DRR nicely expresses matching on missing operands if done via segments nicely (PDL/PDLL conceptually too, but it isn’t utilized yet, so would just be migration blocker). But starting with specific op there is easiest short term.

Not really undef, there is only one value for unit and that is presence. So constant of unit, is indicating presence (and in this case the presence is used to signify absence ;-)).

20

Thank you @River707 for pushing this to a final conclusion. While it is fairly painful downstream, it is a cost we have to pay.

+1 to the updated suggestion. I wonder whether functions are so fundamental that we could also give them their own dialect. It would contain fairly few ops but functions in our pipelines outlive uses of scf, which is a (maybe weak) indication that they are a thing of their own.