Fixing up PLIO compiler support and creating an example #1623

eddierichter-amd · 2024-07-17T04:11:21Z

Adding the necessary passes to have a plio attribute in an aie2.py and have that lower to connecting the PLIO on the VCK5000 platform.

This will be coupled with a PR in the platforms repo with a platform that has PLIO connected on the VCK5000.

github-actions · 2024-07-17T17:45:38Z

Coverage Report

Created: 2024-07-23 02:13

Click here for information about interpreting this report.

Filename	Function Coverage	Line Coverage	Region Coverage	Branch Coverage
AIECreatePathFindFlows.cpp	100.00%	92.71%	85.42%	72.71%
AIEObjectFifoStatefulTransform.cpp	100.00%	94.03%	92.49%	87.91%
AIEPathFinder.cpp	100.00%	95.64%	88.38%	76.88%
Totals	100.00%	93.84%	88.77%	79.52%

Generated by llvm-cov -- llvm version 14.0.0

stephenneuendorffer · 2024-07-19T17:09:17Z

include/aie/Dialect/AIE/IR/AIEOps.td

@@ -1634,7 +1636,8 @@ def AIE_ObjectFifoCreateOp: AIE_Op<"objectfifo", [HasParent<"DeviceOp">, Symbol]
        TypeAttrOf<AIE_ObjectFifoType>:$elemType,
        BDDimLayoutArrayAttr:$dimensionsToStream,
        BDDimLayoutArrayArrayAttr:$dimensionsFromStreamPerConsumer,
-        DefaultValuedAttr<BoolAttr, "false">:$via_DMA
+        DefaultValuedAttr<BoolAttr, "false">:$via_DMA,
+        DefaultValuedAttr<BoolAttr, "false">:$plio


This seems a bit odd to me.. Can't the objectFifo figure out that it's connected to PLIO based on the shimDMAAllocation op? Could I have an objectFifo connecting 2 PLIOs?

I am using this as a way for the user to specify whether the corresponding shim tile(s) of an object FIFO should be connected to GMIO or PLIO. Here is an example: https://github.com/Xilinx/mlir-aie/blob/plio-rebase/programming_examples/basic/passthrough_dmas_plio/aie2-input-plio.py#L49.

We do also need to encode this in the shimDMAAllocationOP but that is later in the compilation, and I believe usually not exposed via the IRON python bindings.

I believe an object FIFO could connect 2 PLIOs but have never tested it. It can connect two GMIOs, correct?

The shimDMAAllocationOp is currently being generated by the AIEObjectFifoStatefulTransform and as the object FIFO also abstracts the AIEFlowOps between tiles I think it can be confusing to know where to add new attributes to guide the configuration. A cleaner solution might be to "allow" shimDMAAllocationOps to exist before the object FIFO lowering pass and have the lowering pass check for them and either: adopt whatever configuration they specify for the shim tile (including the channel index), or completing any missing information if it was not provided (following the same generation it currently uses). Another possibility would be to rely on the current AIEFlowOp in the same manner as above for a more general solution that would allow users to specify the ports they want for any tile not just the shim tile.

Either works for me the only important thing to me is that this is user facing. That being said, that change in allowing shimDMAAllocationOps to be before the object FIFO lowering pass sounds like it could require some refactoring. If it is non-trivial, should that be a separate PR? If so, I can create an issue so we can track it.

If we go this route, it will indeed require a bit of refactoring and I do think it will be better to have it in a separate PR that this PR would then build on. Let's discuss further and see if any of those solutions would be preferred.

A cleaner solution might be to "allow" shimDMAAllocationOps to exist before the object FIFO lowering pass

+1 because it makes things more composable

stephenneuendorffer · 2024-07-19T17:12:53Z

lib/Targets/AIETargetXAIEV2.cpp

+        if (connectOp.getDestBundle() == WireBundle::North)
+          // mux
+          output << "__mlir_aie_try(XAie_PlToAieIntfEnable(" << deviceInstRef
+                 << ", " << tileLocStr("x", "y") << ", "
+                 << connectOp.destIndex() << ", PLIF_WIDTH_64));\n";
+      }


What about the case of being connected TO a PLIO?

Currently we are only supporting the 0th and 1st channel as mentioned in

mlir-aie/lib/Targets/AIETargetXAIEV2.cpp

Line 757 in e05bff4

"Currently only PLIO channel 0 and 1 are supported.");

. For AIE -> PL using those channels it is a straightshot to the PL and we actually don't need to configure anything. From PL -> AIE we need a little configuration which is what this line is doing. This is all tested in hardware.

OK, would like to see the proper configuration happen here, since there's no fundamental reason why the other cases can't be implemented as well. At the very least, this should be an error rather than a warning.

Good point, it was originally an error but that made one of the plio unit tests fail. The reason that I didn't put them in there is that it is difficult to test as I am not sure how at platform build time to force a PLIO connection to use a specific channel. I will look into it and if that doesn't work, I will at least make it an error and change the unit test that was failing.

0f50197 Removed the channel limitation in this commit.

stephenneuendorffer · 2024-07-19T17:14:15Z

lib/Dialect/AIE/Transforms/AIECreatePathFindFlows.cpp

-                  rewriter, cast<Interconnect>(shimMuxOp.getOperation()),
-                  flowOp, srcBundle, srcChannel, WireBundle::North, shimCh);
-            }
+            ShimMuxOp shimMuxOp = analyzer.getShimMux(rewriter, srcSB.col);


It's unclear to me what the point of this change is, but I don't understand the original code necessarily. Is this related to the fact that you only 'support' channel 0 and 1 in the code below?

I don't know how correct the previous code was, but in the pass here:

mlir-aie/lib/Targets/AIETargetXAIEV2.cpp

Lines 709 to 767 in d08090b

for (auto op : targetOp.getOps<ShimMuxOp>()) {

Region &r = op.getConnections();

Block &b = r.front();

bool isEmpty = b.getOps<ConnectOp>().empty();

if (isa<TileOp>(op.getTile().getDefiningOp())) {

int col = op.colIndex();

int row = op.rowIndex();

if (!isEmpty) {

output << "// ShimMux column " << col << " row " << row << "\n";

output << "// NOTE ShimMux always connects from the south as "

<< "directions are defined relative to the tile stream "

<< "switch\n";

output << "x = " << col << ";\n";

output << "y = " << row << ";\n";

}

}

for (auto connectOp : b.getOps<ConnectOp>()) {

if(connectOp.getSourceBundle() == WireBundle::DMA || connectOp.getDestBundle() == WireBundle::DMA) {

if (connectOp.getSourceBundle() == WireBundle::North)

// demux!

output

<< "__mlir_aie_try(XAie_EnableAieToShimDmaStrmPort("

<< deviceInstRef << ", " << tileLocStr("x", "y")

<< ", "

// <<

// stringifyWireBundle(connectOp.sourceBundle()).upper()

<< connectOp.sourceIndex() << "));\n";

else if (connectOp.getDestBundle() == WireBundle::North)

// mux

output

<< "__mlir_aie_try(XAie_EnableShimDmaToAieStrmPort("

<< deviceInstRef << ", " << tileLocStr("x", "y")

<< ", "

// <<

// stringifyWireBundle(connectOp.sourceBundle()).upper()

<< connectOp.destIndex() << "));\n";

}

else if(connectOp.getSourceBundle() == WireBundle::PLIO || connectOp.getDestBundle() == WireBundle::PLIO) {

// Note: Right now this just works with PLIO channel 0 and 1 as those don't require to program

// the shim mux

if(connectOp.destIndex() != 0 && connectOp.destIndex() != 1) {

return connectOp.emitOpError("Currently only PLIO channel 0 and 1 are supported.");

}

if (connectOp.getDestBundle() == WireBundle::North)

// mux

output

<< "__mlir_aie_try(XAie_PlToAieIntfEnable("

<< deviceInstRef << ", " << tileLocStr("x", "y")

<< ", "

<< connectOp.destIndex()

<< ", PLIF_WIDTH_64));\n";

}

}

}

we use the information in the shim mux to know whether we should enable PLIO or GMIO.

OK, I get it. I think the original code here was conceptually incorrect. The intention is/was to be explicit about all the connections that exist even the non-programmable ones. Conceptually the 'shimMux' op is intended to represent all of the connectivity in the shim south of the stream switch.

stephenneuendorffer

Thanks for pulling this together!

stephenneuendorffer · 2024-07-19T21:57:28Z

lib/Dialect/AIE/Transforms/AIECreatePathFindFlows.cpp

-                  rewriter, cast<Interconnect>(shimMuxOp.getOperation()),
-                  flowOp, srcBundle, srcChannel, WireBundle::North, shimCh);
-            }
+            ShimMuxOp shimMuxOp = analyzer.getShimMux(rewriter, srcSB.col);


OK, I get it. I think the original code here was conceptually incorrect. The intention is/was to be explicit about all the connections that exist even the non-programmable ones. Conceptually the 'shimMux' op is intended to represent all of the connectivity in the shim south of the stream switch.

stephenneuendorffer · 2024-07-19T22:00:37Z

lib/Targets/AIETargetXAIEV2.cpp

+        if (connectOp.getDestBundle() == WireBundle::North)
+          // mux
+          output << "__mlir_aie_try(XAie_PlToAieIntfEnable(" << deviceInstRef
+                 << ", " << tileLocStr("x", "y") << ", "
+                 << connectOp.destIndex() << ", PLIF_WIDTH_64));\n";
+      }


OK, would like to see the proper configuration happen here, since there's no fundamental reason why the other cases can't be implemented as well. At the very least, this should be an error rather than a warning.

lib/Dialect/AIE/Transforms/AIEObjectFifoStatefulTransform.cpp

AndraBisca · 2024-07-22T12:31:24Z

include/aie/Dialect/AIE/IR/AIEOps.td

@@ -1634,7 +1636,8 @@ def AIE_ObjectFifoCreateOp: AIE_Op<"objectfifo", [HasParent<"DeviceOp">, Symbol]
        TypeAttrOf<AIE_ObjectFifoType>:$elemType,
        BDDimLayoutArrayAttr:$dimensionsToStream,
        BDDimLayoutArrayArrayAttr:$dimensionsFromStreamPerConsumer,
-        DefaultValuedAttr<BoolAttr, "false">:$via_DMA
+        DefaultValuedAttr<BoolAttr, "false">:$via_DMA,
+        DefaultValuedAttr<BoolAttr, "false">:$plio


The shimDMAAllocationOp is currently being generated by the AIEObjectFifoStatefulTransform and as the object FIFO also abstracts the AIEFlowOps between tiles I think it can be confusing to know where to add new attributes to guide the configuration. A cleaner solution might be to "allow" shimDMAAllocationOps to exist before the object FIFO lowering pass and have the lowering pass check for them and either: adopt whatever configuration they specify for the shim tile (including the channel index), or completing any missing information if it was not provided (following the same generation it currently uses). Another possibility would be to rely on the current AIEFlowOp in the same manner as above for a more general solution that would allow users to specify the ports they want for any tile not just the shim tile.

…check in plio_shimmux test

… a tile where there is no shim dma

AndraBisca

Approved with the intent to refactor some of the code once the PR (to be linked..) generalizing the use of shimDMAAllocationOps as discussed is merged.

Thank you for the changes Eddie!

* update llvm * update for Xilinx/mlir-aie#1623 * update for aiex.runtime_sequence * bump llvm, mlir-aie * bump mlir-aie

jgmelber approved these changes Jul 17, 2024

View reviewed changes

eddierichter-amd mentioned this pull request Jul 18, 2024

Adding initial PLIO support to the platform Xilinx/ROCm-air-platforms#13

Merged

eddierichter-amd marked this pull request as ready for review July 19, 2024 15:37

eddierichter-amd requested review from makslevental, AndraBisca, denolf, fifield and stephenneuendorffer as code owners July 19, 2024 15:37

stephenneuendorffer requested changes Jul 19, 2024

View reviewed changes

stephenneuendorffer reviewed Jul 19, 2024

View reviewed changes

AndraBisca reviewed Jul 22, 2024

View reviewed changes

eddierichter-amd added 14 commits July 22, 2024 19:45

Fixing up PLIO compiler support and creating an example

e280dec

clang-format

750aa09

python format

2781398

downgrading error to warning

ef1fae0

No longer returning when we emit a warning and removing an incorrect …

c81b751

…check in plio_shimmux test

Making it so PLIO flows get lowered to a shim mux even if they are in…

8973628

… a tile where there is no shim dma

clang-format

49bc24c

Only check for PLIO if it is a PL tile

965403a

clang-format

a29234b

Test changes and compiler pass fixes

ea0aef1

clang-format

c5f2f29

Removing limitation of only working for channels 0 and 1

3aad40c

clang-format

a915fb8

Python formatting

16dd328

eddierichter-amd force-pushed the plio-rebase branch from 73d9591 to 16dd328 Compare July 23, 2024 01:52

AndraBisca approved these changes Jul 23, 2024

View reviewed changes

AndraBisca mentioned this pull request Jul 23, 2024

Compose shimDMAAllocationOps with ObjectFifoStatefulTransform #1635

Open

stephenneuendorffer approved these changes Jul 23, 2024

View reviewed changes

eddierichter-amd added this pull request to the merge queue Jul 23, 2024

Merged via the queue into main with commit a1fb650 Jul 23, 2024
51 checks passed

eddierichter-amd deleted the plio-rebase branch July 23, 2024 17:13

fifield added a commit to fifield/mlir-air that referenced this pull request Jul 29, 2024

update for Xilinx/mlir-aie#1623

76ad19a

fifield added a commit to fifield/mlir-air that referenced this pull request Jul 29, 2024

update for Xilinx/mlir-aie#1623

a1a8da0

fifield added a commit to fifield/mlir-air that referenced this pull request Jul 31, 2024

update for Xilinx/mlir-aie#1623

18f4d75

fifield added a commit to Xilinx/mlir-air that referenced this pull request Jul 31, 2024

Update llvm and mlir-aie (#687)

0b4b996

* update llvm * update for Xilinx/mlir-aie#1623 * update for aiex.runtime_sequence * bump llvm, mlir-aie * bump mlir-aie

jgmelber mentioned this pull request Oct 22, 2024

Interfacing with the PL #1105

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fixing up PLIO compiler support and creating an example #1623

Fixing up PLIO compiler support and creating an example #1623

eddierichter-amd commented Jul 17, 2024

github-actions bot commented Jul 17, 2024 •

edited

Loading

stephenneuendorffer Jul 19, 2024

eddierichter-amd Jul 19, 2024

AndraBisca Jul 22, 2024

eddierichter-amd Jul 22, 2024

AndraBisca Jul 22, 2024 •

edited

Loading

fifield Jul 22, 2024

stephenneuendorffer Jul 19, 2024

eddierichter-amd Jul 19, 2024

stephenneuendorffer Jul 19, 2024

eddierichter-amd Jul 19, 2024

eddierichter-amd Jul 22, 2024

stephenneuendorffer Jul 19, 2024

eddierichter-amd Jul 19, 2024

stephenneuendorffer Jul 19, 2024

stephenneuendorffer left a comment

stephenneuendorffer Jul 19, 2024

stephenneuendorffer Jul 19, 2024

AndraBisca Jul 22, 2024

AndraBisca left a comment

	for (auto op : targetOp.getOps<ShimMuxOp>()) {
	Region &r = op.getConnections();
	Block &b = r.front();
	bool isEmpty = b.getOps<ConnectOp>().empty();

	if (isa<TileOp>(op.getTile().getDefiningOp())) {
	int col = op.colIndex();
	int row = op.rowIndex();
	if (!isEmpty) {
	output << "// ShimMux column " << col << " row " << row << "\n";
	output << "// NOTE ShimMux always connects from the south as "
	<< "directions are defined relative to the tile stream "
	<< "switch\n";
	output << "x = " << col << ";\n";
	output << "y = " << row << ";\n";
	}
	}

	for (auto connectOp : b.getOps<ConnectOp>()) {

	if(connectOp.getSourceBundle() == WireBundle::DMA \|\| connectOp.getDestBundle() == WireBundle::DMA) {
	if (connectOp.getSourceBundle() == WireBundle::North)
	// demux!
	output
	<< "__mlir_aie_try(XAie_EnableAieToShimDmaStrmPort("
	<< deviceInstRef << ", " << tileLocStr("x", "y")
	<< ", "
	// <<
	// stringifyWireBundle(connectOp.sourceBundle()).upper()
	<< connectOp.sourceIndex() << "));\n";
	else if (connectOp.getDestBundle() == WireBundle::North)
	// mux
	output
	<< "__mlir_aie_try(XAie_EnableShimDmaToAieStrmPort("
	<< deviceInstRef << ", " << tileLocStr("x", "y")
	<< ", "
	// <<
	// stringifyWireBundle(connectOp.sourceBundle()).upper()
	<< connectOp.destIndex() << "));\n";
	}

	else if(connectOp.getSourceBundle() == WireBundle::PLIO \|\| connectOp.getDestBundle() == WireBundle::PLIO) {
	// Note: Right now this just works with PLIO channel 0 and 1 as those don't require to program
	// the shim mux
	if(connectOp.destIndex() != 0 && connectOp.destIndex() != 1) {
	return connectOp.emitOpError("Currently only PLIO channel 0 and 1 are supported.");
	}

	if (connectOp.getDestBundle() == WireBundle::North)
	// mux
	output
	<< "__mlir_aie_try(XAie_PlToAieIntfEnable("
	<< deviceInstRef << ", " << tileLocStr("x", "y")
	<< ", "
	<< connectOp.destIndex()
	<< ", PLIF_WIDTH_64));\n";
	}
	}
	}

Fixing up PLIO compiler support and creating an example #1623

Fixing up PLIO compiler support and creating an example #1623

Conversation

eddierichter-amd commented Jul 17, 2024

github-actions bot commented Jul 17, 2024 • edited Loading

Coverage Report

Created: 2024-07-23 02:13

Generated by llvm-cov -- llvm version 14.0.0

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

AndraBisca Jul 22, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

stephenneuendorffer left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

AndraBisca left a comment

Choose a reason for hiding this comment

github-actions bot commented Jul 17, 2024 •

edited

Loading

AndraBisca Jul 22, 2024 •

edited

Loading