How to find duplicate values in a table

Sometimes duplicate values in tables can create a major problem when we do not make a primary key or a unique key on a table. In these cases we need to find out the duplicate records and need to delete them. We can use the Having Clause to find out that duplicate records.we show this with the help of an example.

Suppose we have a table named EmployeeDeptInfo which have the column Employeid and Departmentid. The query for creating this table is given below:-

Create table EmployeeDeptInfo ( Employeeid int, Departmentid int)

Employeeid contains the Id of the employee and Departmentid contains the Id of the department to which he/she is belonged. This table do not have any primary key defined on it and also it doesnt have any unique key constraint defined on any column.

Suppose the table contains the following data

Employeeid              Departmentid
1                                  1
2                                  2
3                                  2
4                                  3
3                                  2
2                                  2
5                                  4

2                                  2
In this table, entries for the employee having employeeid 2 & 3 are get repeated.In this example the data in the table is not much for the example point of view but in real time scenario it can be billions of rows and duplication of rows can be a very big concern. Therefore it is necessary to find out the duplicate rows in the table. We can use the Having Clause to find out the duplicate rows:-

Query for finding out the duplicate rows in the table:-

Select Employeeid, Departmentid from EmployeeDeptInfo Group By Employeeid, DepartmentId having (count(employeeid) >1 and count (Departmentid) >1)

This query will give us the following results:-

Employeeid         Departmentid
2                              2
3                              2

We can also find out that how many times a record is repeated in the table. For example, The following query gives us the result of how many times a given record is repeated

Select Employeeid, Departmentid, count (Employeeid) as NoOfRepeat from EmployeeDeptInfo Group By Employeeid, DepartmentId having (count(employeeid) >1 and count (Departmentid) >1)

This query will give us the following results

Employeeid           Departmentid        NoOfRepeat
    2                                 2                           3
    3                                 2                           2

Here NoOfRepeat shows the number of times the records are duplicated in the table

Summary

This article shows that we can find the duplicate records with the help of the Group By and Having Clause.

About vivekjohari

Database Consultant with more than 11.5 years of experience in database designing & programming and DBA related activities.  Had good experience on different databases like SQL Server, MySQL & Oracle, Azure SQL &  Big Data.
This entry was posted in Interview Questions, SQL Basic Concepts, SQL Server and tagged , . Bookmark the permalink.

33 Responses to How to find duplicate values in a table

  1. Anonymous says:

    excellent!! have been looking for this!!

  2. Anonymous says:

    thans simple and superb mamoooooo

  3. mahesh says:

    its nice, But i have a doubt is there any procedure to eliminate duplicate records with out using distinct key word.

  4. Anonymous says:

    Yes there is a way to delete duplicate records without using the distinct keyword

    Here is the Query

    with i as (select *,row_number() over (partition by Dep_ID order by DEP_ID ) as rank from emp)
    delete from i where rank>1

  5. Pingback: hop over to this website

  6. Pingback: agencia de viajes en managua

  7. Pingback: p10496#forum.krajowy.biz

  8. Pingback: ONLINE TRAINING BUILD INTERNET BUSINESS MAKE MONEY

  9. Pingback: Sex doll movie—- Lars and the Real Girl

  10. Pingback: redirected here

  11. Pingback: new website

  12. Pingback: HyphenMax Ecommerce Refreshed

  13. Pingback: Jadwal Piala Eropa

  14. Pingback: click here for info

  15. Pingback: a lot

  16. Pingback: http://thetoptrainticketbiz.strikingly.com/blog/getting-cheap-train-tickets-in-the-uk

  17. Pingback: https://getthebesttraintickets.weebly.com/blog/tips-to-find-cheap-train-tickets

  18. Pingback: aimimchems

  19. Pingback: vao w88

  20. Pingback: didi

  21. Pingback: laser hair removal

  22. Pingback: เล่นพนันออนไลน์

  23. Pingback: botox

  24. Pingback: ทางเข้าUFABET

  25. Pingback: 메이저안전놀이터

  26. Pingback: refresher on film

  27. Pingback: http://apartamentynaibizie.pl/gazeta-wyborcza-kielce-163-2016-2563.php

  28. Pingback: QE4! Stock Market Economy Gold Silver and Crazy News Important Update

  29. Pingback: movies

  30. Pingback: Pre-Market Stocks Gold Silver Update Today New Futures Exchange for Gold

  31. Pingback: شركات التنظيف بالرياض

Leave a Reply