Enriching Data Quality Through SQL in Supply Chain Management (and not only)

As we all know, data quality plays a pivotal role in effective supply chain management, as it directly
impacts decision-making, operational efficiency, and any implementation project success. The accuracy
and integrity of data can be significantly improved through smart SQL statements, particularly in the
context of data cleansing projects.

Here are 11 SQL statements with examples from real world, that I came to use while creating the
business rules for a data cleansing project covering the end-to-end Supply Chain Management Processes.
These turned out to be key in getting an accurate output and not missing any datapoints on the way.

1. TO_VARCHAR() + CAST() + SUBSTRING()

Change number format from 99999.99 to 99,999.99 - this SQL statement converts a string representing a
number in the format '99999.99' to '99,999.99'. It extracts the integer part before the decimal, converts
it to a number, and formats it with commas. The decimal part is then appended to create the formatted
number:

select TO_VARCHAR(CAST(SUBSTRING('23563463.34', 1, POSITION('.' in '23563463.34') - 1) as NUMBER),

|| '.' || SUBSTRING('23563463.34', POSITION('.' in '23563463.34') + 1) AS your_column ;

2. TO_VARCHAR() + REPLACE()

Change number format from 99,999.99 to 99999,99 - this SQL statement changes a number's format
from '99,999.99' to '99999,99'. It first removes commas, treating the number as a continuous string,
then replaces the decimal point with a comma, aligning with formats used in many non-English formats:

select replace(TO_VARCHAR(replace('33,563,463.345', ',', '')), '.', ',') as you_column ;

3. COALESCE()

The COALESCE() function returns the first non-null value in a list and handles the NULL values in math
operations. In this example, the SQL expression calculates the percentage of Issued_Qty to
T_Issued_Qty. The COALESCE function ensures any NULL Issued_Qty is considered as 0, avoiding division
by zero errors:

(COALESCE(T1.Issued_Qty, 0) / T1.T_Issued_Qty)*100 ;

4. TO_VARCHAR() + TO_DATE()

This SQL statement converts a date string in 'YYYYMMDD' format into a 'YYYY-MM-DD' format. It first
interprets the string as a date using TO_DATE and then formats it into a more standard and readable
date format using TO_VARCHAR. Useful when data is coming from different ERPs / ERP Instances and it is
not standardized:

TO_VARCHAR(TO_DATE(t1.ADATU,'YYYYMMDD'),'YYYY-MM-DD') ;

5. LISTAGG()

Used to aggregate strings from data in columns in a database table – in this example, it concatenates
child_part_number values from multiple rows into a single string, separated by semicolons. The WITHIN
GROUP() clause specifies that the aggregation should be ordered by child_part_number in ascending
order. Of course, can be ordered in descending order by using DESC:

listagg(T1.child_part_number, ';') WITHIN GROUP (ORDER BY T1.child_part_number ASC) as

6. ARRAY_AGG()

Returns the input values, pivoted into an array. If the input is empty, the function returns an empty array.
In our example, it aggregates the values of child_part_number from multiple rows into a single array. If
there are no rows, it returns an empty array, ensuring consistency in output regardless of input. To get a
list of unique values, use ARRAY_UNIQUE_AGG():

array_agg(T1.child_part_number) ;

7. ARRAY_TO_STRING()

Returns an input array converted to a string by casting all values to strings – example, it converts an array
of child_part_number values into a single string, with each element separated by a semicolon. This
allows for easy viewing or processing of array contents as a simple text format.

ARRAY_TO_STRING(T1.child_part_number, ';') ;

8. ARRAY_CONTAINS() + CAST()

Query a subset of an array. Returns True if the specified variant is found in the specified array – i.e., it
checks if a PART_NO, cast to a variant type, is present in the BOM_child_list array. The ARRAY_CONTAINS
function returns True if the part number is found, facilitating checks within dynamically built or
predefined arrays.

ARRAY_CONTAINS(CAST(T1.PART_NO as variant), BOM_child_list) ; where BOM_child_list is a built array

9. DENSE_RANK()
A (my favourite) ranking function in SQL that assigns a unique rank value within a specified partition:
each unique value is given a distinct rank, and identical values receive the same rank. The below SQL
statement uses DENSE_RANK() to assign ranks to work_ord_num within groups defined by item_cd.
Identical order numbers receive the same rank, and there are no gaps in ranking between different
ranks:

DENSE_RANK() OVER(PARTITION BY T1.item_cd ORDER BY T1.work_ord_num DESC) as

10. DIV0()

This SQL function DIV0 safely handles division by zero. It divides the dividend by the divisor and returns 0
instead of an error when the divisor is zero, avoiding disruptions in query execution:

DIV0(<dividend> , <divisor>) ;

11. REGEXP_LIKE()

Used for pattern matching. It compares whether the given strings match a regular expression or not, like
checking if Material_Number conforms to a specific pattern. In this particular example, the pattern
requires that the string starts with exactly three letters (either uppercase or lowercase), followed by any
sequence of characters, ensuring flexibility in matching more complex string patterns.

REGEXP_LIKE(T1.Material_Number, '[a-zA-Z]{3}.*') ; - - only alphanumeric characters

Where

[a-zA-Z]: Only matches letter characters, both upper and lowercase

{3}: Requires three of those letters at the start of the string

.*: Allows any number of any characters after those three letters

Similar example : REGEXP_LIKE(right(BOM_TREE.parent_part_number, 2), '[a-zA-Z]{2}.*') ;

To wrap it up, look at your data and check if there is any need to use any of these SQL tricks. Use them
not only to keep your data clean and concise, but also to ensure the data works for you, delivering
insights that are both accurate and actionable. I hope these will make someone’s life easier and the data
a whole lot sharper. Happy querying!