Add load-time configuration to resize the node creation buffer #2102
Conversation
There was a problem hiding this comment.
Let's keep this PR simple
Today our matrices dimensions (NxN) are equal to the graph's node datablock itemCap, which is: number of blocks * DATABLOCK_BLOCK_CAP
I believe the idea was to allow the user to specify a different value which will become a replacement to the DATABLOCK_BLOCK_CAP specifically for the graph node's datablock.
consider
DataBlock *DataBlock_New
(
uint64_t blockCap, // max number of items in a single block
uint64_t itemCount, // initial number of items required
uint itemSize, fpDestructor fp
);to be compact we can use a small value for the blockCap in addition to passing 0 as itemCount
DataBlock *DataBlock_New(128, 0, (fpDestructor)FreeEntity);
tests/unit/test_graph.cpp
Outdated
| @@ -57,6 +57,7 @@ class GraphTest : public ::testing::Test { | |||
| GrB_Index ncols, nrows, nvals; | |||
|
|
|||
| // Create nodes. | |||
| Graph_AllocateNodes(g, node_count); | |||
0fd3ad1 to
773d249
Compare
Codecov Report
@@ Coverage Diff @@
## master #2102 +/- ##
==========================================
+ Coverage 92.79% 92.81% +0.01%
==========================================
Files 252 251 -1
Lines 22425 22451 +26
==========================================
+ Hits 20810 20837 +27
+ Misses 1615 1614 -1
Continue to review full report at Codecov.
|
0084708 to
34dde15
Compare
| @@ -140,10 +140,14 @@ This configuration can only be set when the module loads. | |||
|
|
|||
| `NODE_CREATION_BUFFER` is 16,384 by default. | |||
|
|
|||
| ### Minimum | |||
|
|
|||
| The minimum value for `NODE_CREATION_BUFFER` is 128. Values lower than this will be accepted as arguments, but will internally be converted to 128. | |||
There was a problem hiding this comment.
consider always rounding up to a power of 2
so 200 will become 256
| dataBlock->itemSize = itemSize + ITEM_HEADER_SIZE; | ||
| dataBlock->itemCount = 0; | ||
| dataBlock->blockCount = 0; | ||
| dataBlock->blockCap = blockCap; |
| @@ -13,23 +13,28 @@ | |||
| #include <stdbool.h> | |||
|
|
|||
| // Computes the number of blocks required to accommodate n items. | |||
| #define ITEM_COUNT_TO_BLOCK_COUNT(n) \ | |||
| ceil((double)n / DATABLOCK_BLOCK_CAP) | |||
| #define ITEM_COUNT_TO_BLOCK_COUNT(n, cap) \ | |||
There was a problem hiding this comment.
why need cap? can't we just write dataBlock->blockCap?
There was a problem hiding this comment.
this macro answers the question:
given n items how many blocks do we need to accommodate them?
There was a problem hiding this comment.
yes but we always pass dataBlock->blockCap to this macro
| @@ -27,3 +27,4 @@ void *DataBlock_AllocateItemOutOfOrder(DataBlock *dataBlock, uint64_t idx); | |||
| * 1. Item distructor is not invoked in this call. | |||
| * 2. This call does not decrease the number of items in the data block. */ | |||
| void DataBlock_MarkAsDeletedOutOfOrder(DataBlock *dataBlock, uint64_t idx); | |||
There was a problem hiding this comment.
if the c file deleted why not delete this file too?
There was a problem hiding this comment.
clear separation of the API, but we could delete this one as-well, I just wanted to keep this API separated.
* Add load-time configuration to resize the node creation buffer * Revert changes to graph files * Node creation buffer modifies datablock sizes * refactor datablock, removed unused features * Modify NODE_CREATION_BUFFER tests * Node creation buffer is always a power of 2 Co-authored-by: swilly22 <[email protected]> Co-authored-by: Roi Lipman <[email protected]>
* Add load-time configuration to resize the node creation buffer (#2102) * Add load-time configuration to resize the node creation buffer * Revert changes to graph files * Node creation buffer modifies datablock sizes * refactor datablock, removed unused features * Modify NODE_CREATION_BUFFER tests * Node creation buffer is always a power of 2 Co-authored-by: swilly22 <[email protected]> Co-authored-by: Roi Lipman <[email protected]> * Address PR comments * Bump readies dependency to latest version * Fixed docker builder (build/docker) Co-authored-by: Jeffrey Lovitz <[email protected]> Co-authored-by: swilly22 <[email protected]> Co-authored-by: Roi Lipman <[email protected]> Co-authored-by: rafie <[email protected]>
…Graph#2102) * Add load-time configuration to resize the node creation buffer * Revert changes to graph files * Node creation buffer modifies datablock sizes * refactor datablock, removed unused features * Modify NODE_CREATION_BUFFER tests * Node creation buffer is always a power of 2 Co-authored-by: swilly22 <[email protected]> Co-authored-by: Roi Lipman <[email protected]>
This PR adds the load-time configuration
NODE_CREATION_BUFFERto allow users to modify the amount of empty space is reserved in matrices for future node creations.