[DAG] Add wrappers for insert and extract sub-vector [nfc] #137230
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Mechanical change to introduce the new utilities, and add enough users to make the usage pattern clear. Once this lands, I'm going to do a further pass to adjust many more callsites as a separate change.
|
@llvm/pr-subscribers-backend-risc-v Author: Philip Reames (preames) ChangesMechanical change to introduce the new utilities, and add enough users to make the usage pattern clear. Once this lands, I'm going to do a further pass to adjust more callsites as Patch is 22.11 KiB, truncated to 20.00 KiB below, full version: https://github.com/llvm/llvm-project/pull/137230.diff 3 Files Affected:
diff --git a/llvm/include/llvm/CodeGen/SelectionDAG.h b/llvm/include/llvm/CodeGen/SelectionDAG.h
index c183149b0863a..2b0d928035851 100644
--- a/llvm/include/llvm/CodeGen/SelectionDAG.h
+++ b/llvm/include/llvm/CodeGen/SelectionDAG.h
@@ -924,6 +924,18 @@ class SelectionDAG {
/// Example: shuffle A, B, <0,5,2,7> -> shuffle B, A, <4,1,6,3>
SDValue getCommutedVectorShuffle(const ShuffleVectorSDNode &SV);
+ /// Insert SubVec into the lowest sub-vector of Vec.
+ SDValue getInsertLowSubvector(const SDLoc &DL, SDValue Vec, SDValue SubVec) {
+ return getNode(ISD::INSERT_SUBVECTOR, DL, Vec.getValueType(), Vec, SubVec,
+ getVectorIdxConstant(0, DL));
+ }
+
+ /// Return the lowest VT typed sub-vector of Vec.
+ SDValue getExtractLowSubvector(const SDLoc &DL, EVT VT, SDValue Vec) {
+ return getNode(ISD::EXTRACT_SUBVECTOR, DL, VT, Vec,
+ getVectorIdxConstant(0, DL));
+ }
+
/// Convert Op, which must be of float type, to the
/// float type VT, by either extending or rounding (by truncation).
SDValue getFPExtendOrRound(SDValue Op, const SDLoc &DL, EVT VT);
diff --git a/llvm/lib/CodeGen/SelectionDAG/SelectionDAG.cpp b/llvm/lib/CodeGen/SelectionDAG/SelectionDAG.cpp
index cf88c1f4ae937..f10ad5224346e 100644
--- a/llvm/lib/CodeGen/SelectionDAG/SelectionDAG.cpp
+++ b/llvm/lib/CodeGen/SelectionDAG/SelectionDAG.cpp
@@ -12671,8 +12671,7 @@ SelectionDAG::matchBinOpReduction(SDNode *Extract, ISD::NodeType &BinOp,
if (!TLI->isExtractSubvectorCheap(SubVT, OpVT, 0))
return SDValue();
BinOp = (ISD::NodeType)CandidateBinOp;
- return getNode(ISD::EXTRACT_SUBVECTOR, SDLoc(Op), SubVT, Op,
- getVectorIdxConstant(0, SDLoc(Op)));
+ return getExtractLowSubvector(SDLoc(Op), SubVT, Op);
};
// At each stage, we're looking for something that looks like:
@@ -13042,8 +13041,7 @@ SelectionDAG::SplitVector(const SDValue &N, const SDLoc &DL, const EVT &LoVT,
N.getValueType().getVectorMinNumElements() &&
"More vector elements requested than available!");
SDValue Lo, Hi;
- Lo =
- getNode(ISD::EXTRACT_SUBVECTOR, DL, LoVT, N, getVectorIdxConstant(0, DL));
+ Lo = getExtractLowSubvector(DL, LoVT, N);
// For scalable vectors it is safe to use LoVT.getVectorMinNumElements()
// (rather than having to use ElementCount), because EXTRACT_SUBVECTOR scales
// IDX with the runtime scaling factor of the result vector type. For
@@ -13075,8 +13073,7 @@ SDValue SelectionDAG::WidenVector(const SDValue &N, const SDLoc &DL) {
EVT VT = N.getValueType();
EVT WideVT = EVT::getVectorVT(*getContext(), VT.getVectorElementType(),
NextPowerOf2(VT.getVectorNumElements()));
- return getNode(ISD::INSERT_SUBVECTOR, DL, WideVT, getUNDEF(WideVT), N,
- getVectorIdxConstant(0, DL));
+ return getInsertLowSubvector(DL, getUNDEF(WideVT), N);
}
void SelectionDAG::ExtractVectorElements(SDValue Op,
diff --git a/llvm/lib/Target/RISCV/RISCVISelLowering.cpp b/llvm/lib/Target/RISCV/RISCVISelLowering.cpp
index c440df5a3e638..17e2c3b6f82f3 100644
--- a/llvm/lib/Target/RISCV/RISCVISelLowering.cpp
+++ b/llvm/lib/Target/RISCV/RISCVISelLowering.cpp
@@ -2780,8 +2780,7 @@ static SDValue convertToScalableVector(EVT VT, SDValue V, SelectionDAG &DAG,
assert(V.getValueType().isFixedLengthVector() &&
"Expected a fixed length vector operand!");
SDLoc DL(V);
- SDValue Zero = DAG.getVectorIdxConstant(0, DL);
- return DAG.getNode(ISD::INSERT_SUBVECTOR, DL, VT, DAG.getUNDEF(VT), V, Zero);
+ return DAG.getInsertLowSubvector(DL, DAG.getUNDEF(VT), V);
}
// Shrink V so it's just big enough to maintain a VT's worth of data.
@@ -3625,12 +3624,9 @@ static SDValue matchSplatAsGather(SDValue SplatVal, MVT VT, const SDLoc &DL,
// Put Vec in a VT sized vector
if (SrcContainerVT.getVectorMinNumElements() <
ContainerVT.getVectorMinNumElements())
- Src = DAG.getNode(ISD::INSERT_SUBVECTOR, DL, ContainerVT,
- DAG.getUNDEF(ContainerVT), Src,
- DAG.getVectorIdxConstant(0, DL));
+ Src = DAG.getInsertLowSubvector(DL, DAG.getUNDEF(ContainerVT), Src);
else
- Src = DAG.getNode(ISD::EXTRACT_SUBVECTOR, DL, ContainerVT, Src,
- DAG.getVectorIdxConstant(0, DL));
+ Src = DAG.getExtractLowSubvector(DL, ContainerVT, Src);
// We checked that Idx fits inside VT earlier
auto [Mask, VL] = getDefaultVLOps(VT, ContainerVT, DL, DAG, Subtarget);
@@ -4555,17 +4551,14 @@ static SDValue lowerScalarInsert(SDValue Scalar, SDValue VL, MVT VT,
ExtractedVal, DAG, Subtarget);
}
if (ExtractedContainerVT.bitsLE(VT))
- return DAG.getNode(ISD::INSERT_SUBVECTOR, DL, VT, Passthru,
- ExtractedVal, DAG.getVectorIdxConstant(0, DL));
- return DAG.getNode(ISD::EXTRACT_SUBVECTOR, DL, VT, ExtractedVal,
- DAG.getVectorIdxConstant(0, DL));
+ return DAG.getInsertLowSubvector(DL, Passthru, ExtractedVal);
+ return DAG.getExtractLowSubvector(DL, VT, ExtractedVal);
}
}
-
if (VT.isFloatingPoint())
- return DAG.getNode(RISCVISD::VFMV_S_F_VL, DL, VT,
- DAG.getUNDEF(VT), Scalar, VL);
+ return DAG.getNode(RISCVISD::VFMV_S_F_VL, DL, VT, DAG.getUNDEF(VT), Scalar,
+ VL);
// Avoid the tricky legalization cases by falling back to using the
// splat code which already handles it gracefully.
@@ -4764,8 +4757,7 @@ static SDValue getDeinterleaveShiftAndTrunc(const SDLoc &DL, MVT VT,
Res = DAG.getNode(ISD::TRUNCATE, DL, ResVT, Res);
MVT CastVT = ResVT.changeVectorElementType(VT.getVectorElementType());
Res = DAG.getBitcast(CastVT, Res);
- return DAG.getNode(ISD::INSERT_SUBVECTOR, DL, VT, DAG.getUNDEF(VT), Res,
- DAG.getVectorIdxConstant(0, DL));
+ return DAG.getInsertLowSubvector(DL, DAG.getUNDEF(VT), Res);
}
/// Match a single source shuffle which is an identity except that some
@@ -5225,8 +5217,7 @@ static SDValue lowerBitreverseShuffle(ShuffleVectorSDNode *SVN,
// to insert it into the larger vector and then shift up the reversed bits
// afterwards to get rid of the gap introduced.
if (ViaEltSize > NumElts)
- V = DAG.getNode(ISD::INSERT_SUBVECTOR, DL, ViaBitVT, DAG.getUNDEF(ViaBitVT),
- V, DAG.getVectorIdxConstant(0, DL));
+ V = DAG.getInsertLowSubvector(DL, DAG.getUNDEF(ViaBitVT), V);
SDValue Res =
DAG.getNode(ISD::BITREVERSE, DL, ViaVT, DAG.getBitcast(ViaVT, V));
@@ -5240,8 +5231,7 @@ static SDValue lowerBitreverseShuffle(ShuffleVectorSDNode *SVN,
Res = DAG.getBitcast(ViaBitVT, Res);
if (ViaEltSize > NumElts)
- Res = DAG.getNode(ISD::EXTRACT_SUBVECTOR, DL, VT, Res,
- DAG.getVectorIdxConstant(0, DL));
+ Res = DAG.getExtractLowSubvector(DL, VT, Res);
return Res;
}
@@ -5761,8 +5751,7 @@ static SDValue lowerVECTOR_SHUFFLE(SDValue Op, SelectionDAG &DAG,
return Concat;
SDValue Vec = DAG.getUNDEF(VT);
- return DAG.getNode(ISD::INSERT_SUBVECTOR, DL, VT, Vec, Concat,
- DAG.getVectorIdxConstant(0, DL));
+ return DAG.getInsertLowSubvector(DL, Vec, Concat);
}
}
}
@@ -5811,10 +5800,8 @@ static SDValue lowerVECTOR_SHUFFLE(SDValue Op, SelectionDAG &DAG,
// Prefer vzip2a if available.
// TODO: Extend to matching zip2b if EvenSrc and OddSrc allow.
if (Subtarget.hasVendorXRivosVizip()) {
- EvenV = DAG.getNode(ISD::INSERT_SUBVECTOR, DL, VT, DAG.getUNDEF(VT),
- EvenV, DAG.getVectorIdxConstant(0, DL));
- OddV = DAG.getNode(ISD::INSERT_SUBVECTOR, DL, VT, DAG.getUNDEF(VT), OddV,
- DAG.getVectorIdxConstant(0, DL));
+ EvenV = DAG.getInsertLowSubvector(DL, DAG.getUNDEF(VT), EvenV);
+ OddV = DAG.getInsertLowSubvector(DL, DAG.getUNDEF(VT), OddV);
return lowerVZIP(RISCVISD::RI_VZIP2A_VL, EvenV, OddV, DL, DAG, Subtarget);
}
return getWideningInterleave(EvenV, OddV, DL, DAG, Subtarget);
@@ -5949,8 +5936,7 @@ static SDValue lowerVECTOR_SHUFFLE(SDValue Op, SelectionDAG &DAG,
if (isSpreadMask(Mask, Factor, Index)) {
MVT NarrowVT =
MVT::getVectorVT(VT.getVectorElementType(), NumElts / Factor);
- SDValue Src = DAG.getNode(ISD::EXTRACT_SUBVECTOR, DL, NarrowVT, V1,
- DAG.getVectorIdxConstant(0, DL));
+ SDValue Src = DAG.getExtractLowSubvector(DL, NarrowVT, V1);
return getWideningSpread(Src, Factor, Index, DL, DAG);
}
}
@@ -5971,12 +5957,10 @@ static SDValue lowerVECTOR_SHUFFLE(SDValue Op, SelectionDAG &DAG,
std::max((uint64_t)MinVLMAX, PowerOf2Ceil(MaxIdx + 1));
if (NewNumElts != NumElts) {
MVT NewVT = MVT::getVectorVT(VT.getVectorElementType(), NewNumElts);
- SDValue ZeroIdx = DAG.getVectorIdxConstant(0, DL);
- V1 = DAG.getNode(ISD::EXTRACT_SUBVECTOR, DL, NewVT, V1, ZeroIdx);
+ V1 = DAG.getExtractLowSubvector(DL, NewVT, V1);
SDValue Res = DAG.getVectorShuffle(NewVT, DL, V1, DAG.getUNDEF(NewVT),
Mask.take_front(NewNumElts));
- return DAG.getNode(ISD::INSERT_SUBVECTOR, DL, VT, DAG.getUNDEF(VT), Res,
- ZeroIdx);
+ return DAG.getInsertLowSubvector(DL, DAG.getUNDEF(VT), Res);
}
}
@@ -6056,9 +6040,7 @@ static SDValue lowerVECTOR_SHUFFLE(SDValue Op, SelectionDAG &DAG,
// source register group. TODO: This generalizes to m2, and m4.
const MVT M1VT = getLMUL1VT(ContainerVT);
EVT SubIndexVT = M1VT.changeVectorElementType(IndexVT.getScalarType());
- SDValue SubIndex =
- DAG.getNode(ISD::EXTRACT_SUBVECTOR, DL, SubIndexVT, LHSIndices,
- DAG.getVectorIdxConstant(0, DL));
+ SDValue SubIndex = DAG.getExtractLowSubvector(DL, SubIndexVT, LHSIndices);
auto [InnerTrueMask, InnerVL] =
getDefaultScalableVLOps(M1VT, DL, DAG, Subtarget);
int N = ContainerVT.getVectorMinNumElements() /
@@ -6091,11 +6073,8 @@ static SDValue lowerVECTOR_SHUFFLE(SDValue Op, SelectionDAG &DAG,
int N = ContainerVT.getVectorMinNumElements() /
M1VT.getVectorMinNumElements();
assert(isPowerOf2_32(N) && N <= 8);
- SDValue SubV1 = DAG.getNode(ISD::EXTRACT_SUBVECTOR, DL, M1VT, V1,
- DAG.getVectorIdxConstant(0, DL));
- SDValue SubIndex =
- DAG.getNode(ISD::EXTRACT_SUBVECTOR, DL, SubIndexVT, LHSIndices,
- DAG.getVectorIdxConstant(0, DL));
+ SDValue SubV1 = DAG.getExtractLowSubvector(DL, M1VT, V1);
+ SDValue SubIndex = DAG.getExtractLowSubvector(DL, SubIndexVT, LHSIndices);
SDValue SubVec = DAG.getNode(GatherVVOpc, DL, M1VT, SubV1, SubIndex,
DAG.getUNDEF(M1VT), InnerTrueMask, InnerVL);
Gather = DAG.getUNDEF(ContainerVT);
@@ -6121,17 +6100,14 @@ static SDValue lowerVECTOR_SHUFFLE(SDValue Op, SelectionDAG &DAG,
Gather = DAG.getUNDEF(ContainerVT);
SDValue SlideAmt =
DAG.getElementCount(DL, XLenVT, M1VT.getVectorElementCount());
- SDValue SubV1 =
- DAG.getNode(ISD::EXTRACT_SUBVECTOR, DL, M1VT, V1,
- DAG.getVectorIdxConstant(0, DL));
+ SDValue SubV1 = DAG.getExtractLowSubvector(DL, M1VT, V1);
for (int i = 0; i < N; i++) {
if (i != 0)
LHSIndices = getVSlidedown(DAG, Subtarget, DL, IndexContainerVT,
DAG.getUNDEF(IndexContainerVT), LHSIndices,
SlideAmt, TrueMask, VL);
SDValue SubIndex =
- DAG.getNode(ISD::EXTRACT_SUBVECTOR, DL, SubIndexVT, LHSIndices,
- DAG.getVectorIdxConstant(0, DL));
+ DAG.getExtractLowSubvector(DL, SubIndexVT, LHSIndices);
SDValue SubVec =
DAG.getNode(GatherVVOpc, DL, M1VT, SubV1, SubIndex,
DAG.getUNDEF(M1VT), InnerTrueMask, InnerVL);
@@ -9899,8 +9875,7 @@ SDValue RISCVTargetLowering::lowerEXTRACT_VECTOR_ELT(SDValue Op,
if (auto SmallerVT =
getSmallestVTForIndex(ContainerVT, *MaxIdx, DL, DAG, Subtarget)) {
ContainerVT = *SmallerVT;
- Vec = DAG.getNode(ISD::EXTRACT_SUBVECTOR, DL, ContainerVT, Vec,
- DAG.getConstant(0, DL, XLenVT));
+ Vec = DAG.getExtractLowSubvector(DL, ContainerVT, Vec);
}
}
@@ -10843,12 +10818,11 @@ static SDValue lowerReductionSeq(unsigned RVVOpcode, MVT ResVT,
// prove it is non-zero. For the AVL=0 case, we need the scalar to
// be the result of the reduction operation.
auto InnerVL = NonZeroAVL ? VL : DAG.getConstant(1, DL, XLenVT);
- SDValue InitialValue = lowerScalarInsert(StartValue, InnerVL, InnerVT, DL,
- DAG, Subtarget);
+ SDValue InitialValue =
+ lowerScalarInsert(StartValue, InnerVL, InnerVT, DL, DAG, Subtarget);
if (M1VT != InnerVT)
InitialValue =
- DAG.getNode(ISD::INSERT_SUBVECTOR, DL, M1VT, DAG.getUNDEF(M1VT),
- InitialValue, DAG.getVectorIdxConstant(0, DL));
+ DAG.getInsertLowSubvector(DL, DAG.getUNDEF(M1VT), InitialValue);
SDValue PassThru = NonZeroAVL ? DAG.getUNDEF(M1VT) : InitialValue;
SDValue Policy = DAG.getTargetConstant(RISCVVType::TAIL_AGNOSTIC, DL, XLenVT);
SDValue Ops[] = {PassThru, Vec, InitialValue, Mask, VL, Policy};
@@ -11098,9 +11072,7 @@ SDValue RISCVTargetLowering::lowerINSERT_SUBVECTOR(SDValue Op,
Vec = convertToScalableVector(ContainerVT, Vec, DAG, Subtarget);
}
- SubVec = DAG.getNode(ISD::INSERT_SUBVECTOR, DL, ContainerVT,
- DAG.getUNDEF(ContainerVT), SubVec,
- DAG.getVectorIdxConstant(0, DL));
+ SubVec = DAG.getInsertLowSubvector(DL, DAG.getUNDEF(ContainerVT), SubVec);
SDValue Mask =
getDefaultVLOps(VecVT, ContainerVT, DL, DAG, Subtarget).first;
@@ -11226,9 +11198,7 @@ SDValue RISCVTargetLowering::lowerINSERT_SUBVECTOR(SDValue Op,
DAG.getVectorIdxConstant(AlignedIdx, DL));
}
- SubVec = DAG.getNode(ISD::INSERT_SUBVECTOR, DL, InterSubVT,
- DAG.getUNDEF(InterSubVT), SubVec,
- DAG.getVectorIdxConstant(0, DL));
+ SubVec = DAG.getInsertLowSubvector(DL, DAG.getUNDEF(InterSubVT), SubVec);
auto [Mask, VL] = getDefaultVLOps(VecVT, ContainerVecVT, DL, DAG, Subtarget);
@@ -11342,8 +11312,7 @@ SDValue RISCVTargetLowering::lowerEXTRACT_SUBVECTOR(SDValue Op,
if (auto ShrunkVT =
getSmallestVTForIndex(ContainerVT, LastIdx, DL, DAG, Subtarget)) {
ContainerVT = *ShrunkVT;
- Vec = DAG.getNode(ISD::EXTRACT_SUBVECTOR, DL, ContainerVT, Vec,
- DAG.getVectorIdxConstant(0, DL));
+ Vec = DAG.getExtractLowSubvector(DL, ContainerVT, Vec);
}
SDValue Mask =
@@ -11356,8 +11325,7 @@ SDValue RISCVTargetLowering::lowerEXTRACT_SUBVECTOR(SDValue Op,
getVSlidedown(DAG, Subtarget, DL, ContainerVT,
DAG.getUNDEF(ContainerVT), Vec, SlidedownAmt, Mask, VL);
// Now we can use a cast-like subvector extract to get the result.
- Slidedown = DAG.getNode(ISD::EXTRACT_SUBVECTOR, DL, SubVecVT, Slidedown,
- DAG.getVectorIdxConstant(0, DL));
+ Slidedown = DAG.getExtractLowSubvector(DL, SubVecVT, Slidedown);
return DAG.getBitcast(Op.getValueType(), Slidedown);
}
@@ -11444,8 +11412,7 @@ SDValue RISCVTargetLowering::lowerEXTRACT_SUBVECTOR(SDValue Op,
// Now the vector is in the right position, extract our final subvector. This
// should resolve to a COPY.
- Slidedown = DAG.getNode(ISD::EXTRACT_SUBVECTOR, DL, SubVecVT, Slidedown,
- DAG.getVectorIdxConstant(0, DL));
+ Slidedown = DAG.getExtractLowSubvector(DL, SubVecVT, Slidedown);
// We might have bitcast from a mask type: cast back to the original type if
// required.
@@ -11548,16 +11515,15 @@ SDValue RISCVTargetLowering::lowerVECTOR_DEINTERLEAVE(SDValue Op,
if (SDValue Src = foldConcatVector(V1, V2);
Src && getLMUL1VT(VT).bitsGT(VT)) {
EVT NewVT = VT.getDoubleNumVectorElementsVT();
- SDValue ZeroIdx = DAG.getVectorIdxConstant(0, DL);
- Src = DAG.getNode(ISD::EXTRACT_SUBVECTOR, DL, NewVT, Src, ZeroIdx);
+ Src = DAG.getExtractLowSubvector(DL, NewVT, Src);
// Freeze the source so we can increase its use count.
Src = DAG.getFreeze(Src);
SDValue Even = lowerVZIP(RISCVISD::RI_VUNZIP2A_VL, Src,
DAG.getUNDEF(NewVT), DL, DAG, Subtarget);
SDValue Odd = lowerVZIP(RISCVISD::RI_VUNZIP2B_VL, Src,
DAG.getUNDEF(NewVT), DL, DAG, Subtarget);
- Even = DAG.getNode(ISD::EXTRACT_SUBVECTOR, DL, VT, Even, ZeroIdx);
- Odd = DAG.getNode(ISD::EXTRACT_SUBVECTOR, DL, VT, Odd, ZeroIdx);
+ Even = DAG.getExtractLowSubvector(DL, VT, Even);
+ Odd = DAG.getExtractLowSubvector(DL, VT, Odd);
return DAG.getMergeValues({Even, Odd}, DL);
}
@@ -11601,13 +11567,11 @@ SDValue RISCVTargetLowering::lowerVECTOR_DEINTERLEAVE(SDValue Op,
SDValue EvenSplat = DAG.getConstant(0b01010101, DL, MVT::nxv8i8);
EvenSplat = DAG.getBitcast(MVT::nxv64i1, EvenSplat);
- SDValue EvenMask = DAG.getNode(ISD::EXTRACT_SUBVECTOR, DL, MaskVT,
- EvenSplat, DAG.getVectorIdxConstant(0, DL));
+ SDValue EvenMask = DAG.getExtractLowSubvector(DL, MaskVT, EvenSplat);
SDValue OddSplat = DAG.getConstant(0b10101010, DL, MVT::nxv8i8);
OddSplat = DAG.getBitcast(MVT::nxv64i1, OddSplat);
- SDValue OddMask = DAG.getNode(ISD::EXTRACT_SUBVECTOR, DL, MaskVT, OddSplat,
- DAG.getVectorIdxConstant(0, DL));
+ SDValue OddMask = DAG.getExtractLowSubvector(DL, MaskVT, OddSplat);
// vcompress the even and odd elements into two separate vectors
SDValue EvenWide = DAG.getNode(ISD::VECTOR_COMPRESS, DL, ConcatVT, Concat,
@@ -11987,9 +11951,7 @@ SDValue RISCVTargetLowering::lowerVECTOR_REVERSE(SDValue Op,
Hi = DAG.getNode(ISD::VECTOR_REVERSE, DL, HiVT, Hi);
// Reassemble the low and high pieces reversed.
// FIXME: This is a CONCAT_VECTORS.
- SDValue Res =
- DAG.getNode(ISD::INSERT_SUBVECTOR, DL, VecVT, DAG.getUNDEF(VecVT), Hi,
- DAG.getVectorIdxConstant(0, DL));
+ SDValue Res = DAG.getInsertLowSubvector(DL, DAG.getUNDEF(VecVT), Hi);
return DAG.getNode(
ISD::INSERT_SUBVECTOR, DL, VecVT, Res, Lo,
DAG.getVectorIdxConstant(LoVT.getVectorMinNumElements(), DL));
@@ -12138,8 +12100,7 @@ RISCVTargetLowering::lowerFixedLengthVectorStoreToRVV(SDValue Op,
if (VT.getVectorElementType() == MVT::i1 && VT.getVectorNumElements() < 8) {
VT = MVT::v8i1;
StoreVal =
- DAG.getNode(ISD::INSERT_SUBVECTOR, DL, VT, DAG.getConstant(0, DL, VT),
- StoreVal, DAG.getVectorIdxConstant(0, DL));
+ DAG.getInsertLowSubvector(DL, DAG.getConstant(0, DL, VT), StoreVal);
}
MVT ContainerVT = getContainerForFixedLengthVector(VT);
@@ -14571,8 +14532,7 @@ static SDValue combineBinOpToReduce(SDNode *N, SelectionDAG &DAG,
// If we looked through an INSERT_SUBVECTOR we need to restore it.
if (ScalarVT != ScalarV.getValueType())
NewScalarV =
- DAG.getNode(ISD::INSERT_SUBVECTOR, DL, ScalarVT, DAG.getUNDEF(ScalarVT),
- NewScalarV, DAG.getVectorIdxConstant(0, DL));
+ DAG.getInsertLowSubvector(DL, DAG.getUNDEF(ScalarVT), NewScalarV);
SDValue Ops[] = {Reduce.getOperand(0), Reduce.getOperand(1),
NewScalarV, Reduce.getOperand(3),
@@ -19702,13 +19662,10 @@ SDValue RISCVTargetLowering::PerformDAGCombine(SDNode *N,
// Use M1 or smaller to avoid over constraining register allocation
const MVT M1VT = getLMUL1VT(VT);
if (M1VT.bitsLT(VT)) {
- SDValue M1Passthru =
- DAG.getNode(ISD::EXTRACT_SUBVECTOR, DL, M1VT, Passthru,
- DAG.getVectorIdxConstant(0, DL));
+ SDValue M1Passthru = DAG.getExtractLowSubvector(DL, M1VT, Passthru);
SDValue Result =
DA...
[truncated]
|
|
@llvm/pr-subscribers-llvm-selectiondag Author: Philip Reames (preames) ChangesMechanical change to introduce the new utilities, and add enough users to make the usage pattern clear. Once this lands, I'm going to do a further pass to adjust more callsites as Patch is 22.11 KiB, truncated to 20.00 KiB below, full version: https://github.com/llvm/llvm-project/pull/137230.diff 3 Files Affected:
diff --git a/llvm/include/llvm/CodeGen/SelectionDAG.h b/llvm/include/llvm/CodeGen/SelectionDAG.h
index c183149b0863a..2b0d928035851 100644
--- a/llvm/include/llvm/CodeGen/SelectionDAG.h
+++ b/llvm/include/llvm/CodeGen/SelectionDAG.h
@@ -924,6 +924,18 @@ class SelectionDAG {
/// Example: shuffle A, B, <0,5,2,7> -> shuffle B, A, <4,1,6,3>
SDValue getCommutedVectorShuffle(const ShuffleVectorSDNode &SV);
+ /// Insert SubVec into the lowest sub-vector of Vec.
+ SDValue getInsertLowSubvector(const SDLoc &DL, SDValue Vec, SDValue SubVec) {
+ return getNode(ISD::INSERT_SUBVECTOR, DL, Vec.getValueType(), Vec, SubVec,
+ getVectorIdxConstant(0, DL));
+ }
+
+ /// Return the lowest VT typed sub-vector of Vec.
+ SDValue getExtractLowSubvector(const SDLoc &DL, EVT VT, SDValue Vec) {
+ return getNode(ISD::EXTRACT_SUBVECTOR, DL, VT, Vec,
+ getVectorIdxConstant(0, DL));
+ }
+
/// Convert Op, which must be of float type, to the
/// float type VT, by either extending or rounding (by truncation).
SDValue getFPExtendOrRound(SDValue Op, const SDLoc &DL, EVT VT);
diff --git a/llvm/lib/CodeGen/SelectionDAG/SelectionDAG.cpp b/llvm/lib/CodeGen/SelectionDAG/SelectionDAG.cpp
index cf88c1f4ae937..f10ad5224346e 100644
--- a/llvm/lib/CodeGen/SelectionDAG/SelectionDAG.cpp
+++ b/llvm/lib/CodeGen/SelectionDAG/SelectionDAG.cpp
@@ -12671,8 +12671,7 @@ SelectionDAG::matchBinOpReduction(SDNode *Extract, ISD::NodeType &BinOp,
if (!TLI->isExtractSubvectorCheap(SubVT, OpVT, 0))
return SDValue();
BinOp = (ISD::NodeType)CandidateBinOp;
- return getNode(ISD::EXTRACT_SUBVECTOR, SDLoc(Op), SubVT, Op,
- getVectorIdxConstant(0, SDLoc(Op)));
+ return getExtractLowSubvector(SDLoc(Op), SubVT, Op);
};
// At each stage, we're looking for something that looks like:
@@ -13042,8 +13041,7 @@ SelectionDAG::SplitVector(const SDValue &N, const SDLoc &DL, const EVT &LoVT,
N.getValueType().getVectorMinNumElements() &&
"More vector elements requested than available!");
SDValue Lo, Hi;
- Lo =
- getNode(ISD::EXTRACT_SUBVECTOR, DL, LoVT, N, getVectorIdxConstant(0, DL));
+ Lo = getExtractLowSubvector(DL, LoVT, N);
// For scalable vectors it is safe to use LoVT.getVectorMinNumElements()
// (rather than having to use ElementCount), because EXTRACT_SUBVECTOR scales
// IDX with the runtime scaling factor of the result vector type. For
@@ -13075,8 +13073,7 @@ SDValue SelectionDAG::WidenVector(const SDValue &N, const SDLoc &DL) {
EVT VT = N.getValueType();
EVT WideVT = EVT::getVectorVT(*getContext(), VT.getVectorElementType(),
NextPowerOf2(VT.getVectorNumElements()));
- return getNode(ISD::INSERT_SUBVECTOR, DL, WideVT, getUNDEF(WideVT), N,
- getVectorIdxConstant(0, DL));
+ return getInsertLowSubvector(DL, getUNDEF(WideVT), N);
}
void SelectionDAG::ExtractVectorElements(SDValue Op,
diff --git a/llvm/lib/Target/RISCV/RISCVISelLowering.cpp b/llvm/lib/Target/RISCV/RISCVISelLowering.cpp
index c440df5a3e638..17e2c3b6f82f3 100644
--- a/llvm/lib/Target/RISCV/RISCVISelLowering.cpp
+++ b/llvm/lib/Target/RISCV/RISCVISelLowering.cpp
@@ -2780,8 +2780,7 @@ static SDValue convertToScalableVector(EVT VT, SDValue V, SelectionDAG &DAG,
assert(V.getValueType().isFixedLengthVector() &&
"Expected a fixed length vector operand!");
SDLoc DL(V);
- SDValue Zero = DAG.getVectorIdxConstant(0, DL);
- return DAG.getNode(ISD::INSERT_SUBVECTOR, DL, VT, DAG.getUNDEF(VT), V, Zero);
+ return DAG.getInsertLowSubvector(DL, DAG.getUNDEF(VT), V);
}
// Shrink V so it's just big enough to maintain a VT's worth of data.
@@ -3625,12 +3624,9 @@ static SDValue matchSplatAsGather(SDValue SplatVal, MVT VT, const SDLoc &DL,
// Put Vec in a VT sized vector
if (SrcContainerVT.getVectorMinNumElements() <
ContainerVT.getVectorMinNumElements())
- Src = DAG.getNode(ISD::INSERT_SUBVECTOR, DL, ContainerVT,
- DAG.getUNDEF(ContainerVT), Src,
- DAG.getVectorIdxConstant(0, DL));
+ Src = DAG.getInsertLowSubvector(DL, DAG.getUNDEF(ContainerVT), Src);
else
- Src = DAG.getNode(ISD::EXTRACT_SUBVECTOR, DL, ContainerVT, Src,
- DAG.getVectorIdxConstant(0, DL));
+ Src = DAG.getExtractLowSubvector(DL, ContainerVT, Src);
// We checked that Idx fits inside VT earlier
auto [Mask, VL] = getDefaultVLOps(VT, ContainerVT, DL, DAG, Subtarget);
@@ -4555,17 +4551,14 @@ static SDValue lowerScalarInsert(SDValue Scalar, SDValue VL, MVT VT,
ExtractedVal, DAG, Subtarget);
}
if (ExtractedContainerVT.bitsLE(VT))
- return DAG.getNode(ISD::INSERT_SUBVECTOR, DL, VT, Passthru,
- ExtractedVal, DAG.getVectorIdxConstant(0, DL));
- return DAG.getNode(ISD::EXTRACT_SUBVECTOR, DL, VT, ExtractedVal,
- DAG.getVectorIdxConstant(0, DL));
+ return DAG.getInsertLowSubvector(DL, Passthru, ExtractedVal);
+ return DAG.getExtractLowSubvector(DL, VT, ExtractedVal);
}
}
-
if (VT.isFloatingPoint())
- return DAG.getNode(RISCVISD::VFMV_S_F_VL, DL, VT,
- DAG.getUNDEF(VT), Scalar, VL);
+ return DAG.getNode(RISCVISD::VFMV_S_F_VL, DL, VT, DAG.getUNDEF(VT), Scalar,
+ VL);
// Avoid the tricky legalization cases by falling back to using the
// splat code which already handles it gracefully.
@@ -4764,8 +4757,7 @@ static SDValue getDeinterleaveShiftAndTrunc(const SDLoc &DL, MVT VT,
Res = DAG.getNode(ISD::TRUNCATE, DL, ResVT, Res);
MVT CastVT = ResVT.changeVectorElementType(VT.getVectorElementType());
Res = DAG.getBitcast(CastVT, Res);
- return DAG.getNode(ISD::INSERT_SUBVECTOR, DL, VT, DAG.getUNDEF(VT), Res,
- DAG.getVectorIdxConstant(0, DL));
+ return DAG.getInsertLowSubvector(DL, DAG.getUNDEF(VT), Res);
}
/// Match a single source shuffle which is an identity except that some
@@ -5225,8 +5217,7 @@ static SDValue lowerBitreverseShuffle(ShuffleVectorSDNode *SVN,
// to insert it into the larger vector and then shift up the reversed bits
// afterwards to get rid of the gap introduced.
if (ViaEltSize > NumElts)
- V = DAG.getNode(ISD::INSERT_SUBVECTOR, DL, ViaBitVT, DAG.getUNDEF(ViaBitVT),
- V, DAG.getVectorIdxConstant(0, DL));
+ V = DAG.getInsertLowSubvector(DL, DAG.getUNDEF(ViaBitVT), V);
SDValue Res =
DAG.getNode(ISD::BITREVERSE, DL, ViaVT, DAG.getBitcast(ViaVT, V));
@@ -5240,8 +5231,7 @@ static SDValue lowerBitreverseShuffle(ShuffleVectorSDNode *SVN,
Res = DAG.getBitcast(ViaBitVT, Res);
if (ViaEltSize > NumElts)
- Res = DAG.getNode(ISD::EXTRACT_SUBVECTOR, DL, VT, Res,
- DAG.getVectorIdxConstant(0, DL));
+ Res = DAG.getExtractLowSubvector(DL, VT, Res);
return Res;
}
@@ -5761,8 +5751,7 @@ static SDValue lowerVECTOR_SHUFFLE(SDValue Op, SelectionDAG &DAG,
return Concat;
SDValue Vec = DAG.getUNDEF(VT);
- return DAG.getNode(ISD::INSERT_SUBVECTOR, DL, VT, Vec, Concat,
- DAG.getVectorIdxConstant(0, DL));
+ return DAG.getInsertLowSubvector(DL, Vec, Concat);
}
}
}
@@ -5811,10 +5800,8 @@ static SDValue lowerVECTOR_SHUFFLE(SDValue Op, SelectionDAG &DAG,
// Prefer vzip2a if available.
// TODO: Extend to matching zip2b if EvenSrc and OddSrc allow.
if (Subtarget.hasVendorXRivosVizip()) {
- EvenV = DAG.getNode(ISD::INSERT_SUBVECTOR, DL, VT, DAG.getUNDEF(VT),
- EvenV, DAG.getVectorIdxConstant(0, DL));
- OddV = DAG.getNode(ISD::INSERT_SUBVECTOR, DL, VT, DAG.getUNDEF(VT), OddV,
- DAG.getVectorIdxConstant(0, DL));
+ EvenV = DAG.getInsertLowSubvector(DL, DAG.getUNDEF(VT), EvenV);
+ OddV = DAG.getInsertLowSubvector(DL, DAG.getUNDEF(VT), OddV);
return lowerVZIP(RISCVISD::RI_VZIP2A_VL, EvenV, OddV, DL, DAG, Subtarget);
}
return getWideningInterleave(EvenV, OddV, DL, DAG, Subtarget);
@@ -5949,8 +5936,7 @@ static SDValue lowerVECTOR_SHUFFLE(SDValue Op, SelectionDAG &DAG,
if (isSpreadMask(Mask, Factor, Index)) {
MVT NarrowVT =
MVT::getVectorVT(VT.getVectorElementType(), NumElts / Factor);
- SDValue Src = DAG.getNode(ISD::EXTRACT_SUBVECTOR, DL, NarrowVT, V1,
- DAG.getVectorIdxConstant(0, DL));
+ SDValue Src = DAG.getExtractLowSubvector(DL, NarrowVT, V1);
return getWideningSpread(Src, Factor, Index, DL, DAG);
}
}
@@ -5971,12 +5957,10 @@ static SDValue lowerVECTOR_SHUFFLE(SDValue Op, SelectionDAG &DAG,
std::max((uint64_t)MinVLMAX, PowerOf2Ceil(MaxIdx + 1));
if (NewNumElts != NumElts) {
MVT NewVT = MVT::getVectorVT(VT.getVectorElementType(), NewNumElts);
- SDValue ZeroIdx = DAG.getVectorIdxConstant(0, DL);
- V1 = DAG.getNode(ISD::EXTRACT_SUBVECTOR, DL, NewVT, V1, ZeroIdx);
+ V1 = DAG.getExtractLowSubvector(DL, NewVT, V1);
SDValue Res = DAG.getVectorShuffle(NewVT, DL, V1, DAG.getUNDEF(NewVT),
Mask.take_front(NewNumElts));
- return DAG.getNode(ISD::INSERT_SUBVECTOR, DL, VT, DAG.getUNDEF(VT), Res,
- ZeroIdx);
+ return DAG.getInsertLowSubvector(DL, DAG.getUNDEF(VT), Res);
}
}
@@ -6056,9 +6040,7 @@ static SDValue lowerVECTOR_SHUFFLE(SDValue Op, SelectionDAG &DAG,
// source register group. TODO: This generalizes to m2, and m4.
const MVT M1VT = getLMUL1VT(ContainerVT);
EVT SubIndexVT = M1VT.changeVectorElementType(IndexVT.getScalarType());
- SDValue SubIndex =
- DAG.getNode(ISD::EXTRACT_SUBVECTOR, DL, SubIndexVT, LHSIndices,
- DAG.getVectorIdxConstant(0, DL));
+ SDValue SubIndex = DAG.getExtractLowSubvector(DL, SubIndexVT, LHSIndices);
auto [InnerTrueMask, InnerVL] =
getDefaultScalableVLOps(M1VT, DL, DAG, Subtarget);
int N = ContainerVT.getVectorMinNumElements() /
@@ -6091,11 +6073,8 @@ static SDValue lowerVECTOR_SHUFFLE(SDValue Op, SelectionDAG &DAG,
int N = ContainerVT.getVectorMinNumElements() /
M1VT.getVectorMinNumElements();
assert(isPowerOf2_32(N) && N <= 8);
- SDValue SubV1 = DAG.getNode(ISD::EXTRACT_SUBVECTOR, DL, M1VT, V1,
- DAG.getVectorIdxConstant(0, DL));
- SDValue SubIndex =
- DAG.getNode(ISD::EXTRACT_SUBVECTOR, DL, SubIndexVT, LHSIndices,
- DAG.getVectorIdxConstant(0, DL));
+ SDValue SubV1 = DAG.getExtractLowSubvector(DL, M1VT, V1);
+ SDValue SubIndex = DAG.getExtractLowSubvector(DL, SubIndexVT, LHSIndices);
SDValue SubVec = DAG.getNode(GatherVVOpc, DL, M1VT, SubV1, SubIndex,
DAG.getUNDEF(M1VT), InnerTrueMask, InnerVL);
Gather = DAG.getUNDEF(ContainerVT);
@@ -6121,17 +6100,14 @@ static SDValue lowerVECTOR_SHUFFLE(SDValue Op, SelectionDAG &DAG,
Gather = DAG.getUNDEF(ContainerVT);
SDValue SlideAmt =
DAG.getElementCount(DL, XLenVT, M1VT.getVectorElementCount());
- SDValue SubV1 =
- DAG.getNode(ISD::EXTRACT_SUBVECTOR, DL, M1VT, V1,
- DAG.getVectorIdxConstant(0, DL));
+ SDValue SubV1 = DAG.getExtractLowSubvector(DL, M1VT, V1);
for (int i = 0; i < N; i++) {
if (i != 0)
LHSIndices = getVSlidedown(DAG, Subtarget, DL, IndexContainerVT,
DAG.getUNDEF(IndexContainerVT), LHSIndices,
SlideAmt, TrueMask, VL);
SDValue SubIndex =
- DAG.getNode(ISD::EXTRACT_SUBVECTOR, DL, SubIndexVT, LHSIndices,
- DAG.getVectorIdxConstant(0, DL));
+ DAG.getExtractLowSubvector(DL, SubIndexVT, LHSIndices);
SDValue SubVec =
DAG.getNode(GatherVVOpc, DL, M1VT, SubV1, SubIndex,
DAG.getUNDEF(M1VT), InnerTrueMask, InnerVL);
@@ -9899,8 +9875,7 @@ SDValue RISCVTargetLowering::lowerEXTRACT_VECTOR_ELT(SDValue Op,
if (auto SmallerVT =
getSmallestVTForIndex(ContainerVT, *MaxIdx, DL, DAG, Subtarget)) {
ContainerVT = *SmallerVT;
- Vec = DAG.getNode(ISD::EXTRACT_SUBVECTOR, DL, ContainerVT, Vec,
- DAG.getConstant(0, DL, XLenVT));
+ Vec = DAG.getExtractLowSubvector(DL, ContainerVT, Vec);
}
}
@@ -10843,12 +10818,11 @@ static SDValue lowerReductionSeq(unsigned RVVOpcode, MVT ResVT,
// prove it is non-zero. For the AVL=0 case, we need the scalar to
// be the result of the reduction operation.
auto InnerVL = NonZeroAVL ? VL : DAG.getConstant(1, DL, XLenVT);
- SDValue InitialValue = lowerScalarInsert(StartValue, InnerVL, InnerVT, DL,
- DAG, Subtarget);
+ SDValue InitialValue =
+ lowerScalarInsert(StartValue, InnerVL, InnerVT, DL, DAG, Subtarget);
if (M1VT != InnerVT)
InitialValue =
- DAG.getNode(ISD::INSERT_SUBVECTOR, DL, M1VT, DAG.getUNDEF(M1VT),
- InitialValue, DAG.getVectorIdxConstant(0, DL));
+ DAG.getInsertLowSubvector(DL, DAG.getUNDEF(M1VT), InitialValue);
SDValue PassThru = NonZeroAVL ? DAG.getUNDEF(M1VT) : InitialValue;
SDValue Policy = DAG.getTargetConstant(RISCVVType::TAIL_AGNOSTIC, DL, XLenVT);
SDValue Ops[] = {PassThru, Vec, InitialValue, Mask, VL, Policy};
@@ -11098,9 +11072,7 @@ SDValue RISCVTargetLowering::lowerINSERT_SUBVECTOR(SDValue Op,
Vec = convertToScalableVector(ContainerVT, Vec, DAG, Subtarget);
}
- SubVec = DAG.getNode(ISD::INSERT_SUBVECTOR, DL, ContainerVT,
- DAG.getUNDEF(ContainerVT), SubVec,
- DAG.getVectorIdxConstant(0, DL));
+ SubVec = DAG.getInsertLowSubvector(DL, DAG.getUNDEF(ContainerVT), SubVec);
SDValue Mask =
getDefaultVLOps(VecVT, ContainerVT, DL, DAG, Subtarget).first;
@@ -11226,9 +11198,7 @@ SDValue RISCVTargetLowering::lowerINSERT_SUBVECTOR(SDValue Op,
DAG.getVectorIdxConstant(AlignedIdx, DL));
}
- SubVec = DAG.getNode(ISD::INSERT_SUBVECTOR, DL, InterSubVT,
- DAG.getUNDEF(InterSubVT), SubVec,
- DAG.getVectorIdxConstant(0, DL));
+ SubVec = DAG.getInsertLowSubvector(DL, DAG.getUNDEF(InterSubVT), SubVec);
auto [Mask, VL] = getDefaultVLOps(VecVT, ContainerVecVT, DL, DAG, Subtarget);
@@ -11342,8 +11312,7 @@ SDValue RISCVTargetLowering::lowerEXTRACT_SUBVECTOR(SDValue Op,
if (auto ShrunkVT =
getSmallestVTForIndex(ContainerVT, LastIdx, DL, DAG, Subtarget)) {
ContainerVT = *ShrunkVT;
- Vec = DAG.getNode(ISD::EXTRACT_SUBVECTOR, DL, ContainerVT, Vec,
- DAG.getVectorIdxConstant(0, DL));
+ Vec = DAG.getExtractLowSubvector(DL, ContainerVT, Vec);
}
SDValue Mask =
@@ -11356,8 +11325,7 @@ SDValue RISCVTargetLowering::lowerEXTRACT_SUBVECTOR(SDValue Op,
getVSlidedown(DAG, Subtarget, DL, ContainerVT,
DAG.getUNDEF(ContainerVT), Vec, SlidedownAmt, Mask, VL);
// Now we can use a cast-like subvector extract to get the result.
- Slidedown = DAG.getNode(ISD::EXTRACT_SUBVECTOR, DL, SubVecVT, Slidedown,
- DAG.getVectorIdxConstant(0, DL));
+ Slidedown = DAG.getExtractLowSubvector(DL, SubVecVT, Slidedown);
return DAG.getBitcast(Op.getValueType(), Slidedown);
}
@@ -11444,8 +11412,7 @@ SDValue RISCVTargetLowering::lowerEXTRACT_SUBVECTOR(SDValue Op,
// Now the vector is in the right position, extract our final subvector. This
// should resolve to a COPY.
- Slidedown = DAG.getNode(ISD::EXTRACT_SUBVECTOR, DL, SubVecVT, Slidedown,
- DAG.getVectorIdxConstant(0, DL));
+ Slidedown = DAG.getExtractLowSubvector(DL, SubVecVT, Slidedown);
// We might have bitcast from a mask type: cast back to the original type if
// required.
@@ -11548,16 +11515,15 @@ SDValue RISCVTargetLowering::lowerVECTOR_DEINTERLEAVE(SDValue Op,
if (SDValue Src = foldConcatVector(V1, V2);
Src && getLMUL1VT(VT).bitsGT(VT)) {
EVT NewVT = VT.getDoubleNumVectorElementsVT();
- SDValue ZeroIdx = DAG.getVectorIdxConstant(0, DL);
- Src = DAG.getNode(ISD::EXTRACT_SUBVECTOR, DL, NewVT, Src, ZeroIdx);
+ Src = DAG.getExtractLowSubvector(DL, NewVT, Src);
// Freeze the source so we can increase its use count.
Src = DAG.getFreeze(Src);
SDValue Even = lowerVZIP(RISCVISD::RI_VUNZIP2A_VL, Src,
DAG.getUNDEF(NewVT), DL, DAG, Subtarget);
SDValue Odd = lowerVZIP(RISCVISD::RI_VUNZIP2B_VL, Src,
DAG.getUNDEF(NewVT), DL, DAG, Subtarget);
- Even = DAG.getNode(ISD::EXTRACT_SUBVECTOR, DL, VT, Even, ZeroIdx);
- Odd = DAG.getNode(ISD::EXTRACT_SUBVECTOR, DL, VT, Odd, ZeroIdx);
+ Even = DAG.getExtractLowSubvector(DL, VT, Even);
+ Odd = DAG.getExtractLowSubvector(DL, VT, Odd);
return DAG.getMergeValues({Even, Odd}, DL);
}
@@ -11601,13 +11567,11 @@ SDValue RISCVTargetLowering::lowerVECTOR_DEINTERLEAVE(SDValue Op,
SDValue EvenSplat = DAG.getConstant(0b01010101, DL, MVT::nxv8i8);
EvenSplat = DAG.getBitcast(MVT::nxv64i1, EvenSplat);
- SDValue EvenMask = DAG.getNode(ISD::EXTRACT_SUBVECTOR, DL, MaskVT,
- EvenSplat, DAG.getVectorIdxConstant(0, DL));
+ SDValue EvenMask = DAG.getExtractLowSubvector(DL, MaskVT, EvenSplat);
SDValue OddSplat = DAG.getConstant(0b10101010, DL, MVT::nxv8i8);
OddSplat = DAG.getBitcast(MVT::nxv64i1, OddSplat);
- SDValue OddMask = DAG.getNode(ISD::EXTRACT_SUBVECTOR, DL, MaskVT, OddSplat,
- DAG.getVectorIdxConstant(0, DL));
+ SDValue OddMask = DAG.getExtractLowSubvector(DL, MaskVT, OddSplat);
// vcompress the even and odd elements into two separate vectors
SDValue EvenWide = DAG.getNode(ISD::VECTOR_COMPRESS, DL, ConcatVT, Concat,
@@ -11987,9 +11951,7 @@ SDValue RISCVTargetLowering::lowerVECTOR_REVERSE(SDValue Op,
Hi = DAG.getNode(ISD::VECTOR_REVERSE, DL, HiVT, Hi);
// Reassemble the low and high pieces reversed.
// FIXME: This is a CONCAT_VECTORS.
- SDValue Res =
- DAG.getNode(ISD::INSERT_SUBVECTOR, DL, VecVT, DAG.getUNDEF(VecVT), Hi,
- DAG.getVectorIdxConstant(0, DL));
+ SDValue Res = DAG.getInsertLowSubvector(DL, DAG.getUNDEF(VecVT), Hi);
return DAG.getNode(
ISD::INSERT_SUBVECTOR, DL, VecVT, Res, Lo,
DAG.getVectorIdxConstant(LoVT.getVectorMinNumElements(), DL));
@@ -12138,8 +12100,7 @@ RISCVTargetLowering::lowerFixedLengthVectorStoreToRVV(SDValue Op,
if (VT.getVectorElementType() == MVT::i1 && VT.getVectorNumElements() < 8) {
VT = MVT::v8i1;
StoreVal =
- DAG.getNode(ISD::INSERT_SUBVECTOR, DL, VT, DAG.getConstant(0, DL, VT),
- StoreVal, DAG.getVectorIdxConstant(0, DL));
+ DAG.getInsertLowSubvector(DL, DAG.getConstant(0, DL, VT), StoreVal);
}
MVT ContainerVT = getContainerForFixedLengthVector(VT);
@@ -14571,8 +14532,7 @@ static SDValue combineBinOpToReduce(SDNode *N, SelectionDAG &DAG,
// If we looked through an INSERT_SUBVECTOR we need to restore it.
if (ScalarVT != ScalarV.getValueType())
NewScalarV =
- DAG.getNode(ISD::INSERT_SUBVECTOR, DL, ScalarVT, DAG.getUNDEF(ScalarVT),
- NewScalarV, DAG.getVectorIdxConstant(0, DL));
+ DAG.getInsertLowSubvector(DL, DAG.getUNDEF(ScalarVT), NewScalarV);
SDValue Ops[] = {Reduce.getOperand(0), Reduce.getOperand(1),
NewScalarV, Reduce.getOperand(3),
@@ -19702,13 +19662,10 @@ SDValue RISCVTargetLowering::PerformDAGCombine(SDNode *N,
// Use M1 or smaller to avoid over constraining register allocation
const MVT M1VT = getLMUL1VT(VT);
if (M1VT.bitsLT(VT)) {
- SDValue M1Passthru =
- DAG.getNode(ISD::EXTRACT_SUBVECTOR, DL, M1VT, Passthru,
- DAG.getVectorIdxConstant(0, DL));
+ SDValue M1Passthru = DAG.getExtractLowSubvector(DL, M1VT, Passthru);
SDValue Result =
DA...
[truncated]
|
arsenm
reviewed
Apr 24, 2025
| @@ -924,6 +924,18 @@ class SelectionDAG { | |||
| /// Example: shuffle A, B, <0,5,2,7> -> shuffle B, A, <4,1,6,3> | |||
| SDValue getCommutedVectorShuffle(const ShuffleVectorSDNode &SV); | |||
|
|
|||
| /// Insert SubVec into the lowest sub-vector of Vec. | |||
There was a problem hiding this comment.
Doesn't this assume little endian vector element ordering? The endianness of elements was always unclear to me
There was a problem hiding this comment.
Indices are ordered from zero up, whether that's left to right or right to left. If you have a word you'd prefer other than "low", I'm open to suggestions.
There was a problem hiding this comment.
Sorry for the bike shedding - but maybe just getWidenSubvector/getNarrowSubvector and just handle the (INSERT_SUBVECTOR undef, sub, 0) case for getWidenSubvector?
There was a problem hiding this comment.
I would find these names extremely confusing, as widen and narrow would be things I'd expect to be done element wise to the elements given those names. I'm fine exploring the case with a undef passthru (that is fairly common), but these names really don't work.
There was a problem hiding this comment.
We could make it getInsertSubvector/getExtractSubvector with an unsigned index. Then we don't need Low and just pass 0 explicitly. I imagine that would let us update even more places?
There was a problem hiding this comment.
+1 to that. We already have some helpers for other vector nodes like getSplatVector, getShuffleVector, getBuildVector etc.
lukel97
approved these changes
May 8, 2025
Co-authored-by: Luke Lau <[email protected]>
|
Buildkite is claiming that this change introduces a failure in Flang.Driver/mcmodel.f90 and the flang's parsing of the codemodel command line. This seems sufficiently unlikely that I'm going to ignore the failure and land the change anyways. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Mechanical change to introduce the new wrappers, and add enough users to make the usage pattern clear. Once this lands, I'm going to do a further pass to adjust more callsites as
separate changes.