✅ Identifying duplicates in columns

[lbrown]

How do I identify duplicates in Xcalar? For example, I want to identify the duplicate rows present in 2 or 3 columns of my table. How do I go about this??

[jennyyan]

Hi Larry!

Do you want to simply identify the duplicates or do you want to remove them?

Thanks,
Jenny

[lbrown]

I would actually like to remove the duplicates.

[jennyyan]

Hi Larry,

Here's something that can work:
1. Let's call the column that you would like to dedup ColA, and ColB.
2. Select ColA, ColB and from the dropdown, select groupBy. Now select count as the function, and choose ColA to count on (this field doesn't really matter).
3. Fill in all the necessary names. Click on Advanced Options and select the checkbox that says Keep some columns
4. Click on Select all and click on Group By

In the resultant table, you will notice that the rows will be deduplicated based on the 2 columns that you have chosen. The new column that gets created even has the count of the number of duplicates!

Hope this helps! Happy Xcaling!

Cheers,
Jenny

[dshetty]