Web Analytics Made Easy - Statcounter

Technology with Vivek Johari

Interview Questions SQL Basic Concepts SQL Server

How to find duplicate values in a table

Byvivekjohari

Jan 31, 2010 #CodeProject, #SQL Duplicate Values
Sometimes duplicate values in tables can create a major problem when we do not make a primary key or a unique key on a table. In these cases we need to find out the duplicate records and need to delete them. We can use the Having Clause to find out that duplicate records.we show this with the help of an example.

Suppose we have a table named EmployeeDeptInfo which have the column Employeid and Departmentid. The query for creating this table is given below:-

Create table EmployeeDeptInfo ( Employeeid int, Departmentid int)

Employeeid contains the Id of the employee and Departmentid contains the Id of the department to which he/she is belonged. This table do not have any primary key defined on it and also it doesnt have any unique key constraint defined on any column.

Suppose the table contains the following data

Employeeid              Departmentid
1                                  1
2                                  2
3                                  2
4                                  3
3                                  2
2                                  2
5                                  4

2                                  2
In this table, entries for the employee having employeeid 2 & 3 are get repeated.In this example the data in the table is not much for the example point of view but in real time scenario it can be billions of rows and duplication of rows can be a very big concern. Therefore it is necessary to find out the duplicate rows in the table. We can use the Having Clause to find out the duplicate rows:-

Trending
Exception handling in SQL Server

Query for finding out the duplicate rows in the table:-

Select Employeeid, Departmentid from EmployeeDeptInfo Group By Employeeid, DepartmentId having (count(employeeid) >1 and count (Departmentid) >1)

This query will give us the following results:-

Employeeid         Departmentid
2                              2
3                              2

We can also find out that how many times a record is repeated in the table. For example, The following query gives us the result of how many times a given record is repeated

Select Employeeid, Departmentid, count (Employeeid) as NoOfRepeat from EmployeeDeptInfo Group By Employeeid, DepartmentId having (count(employeeid) >1 and count (Departmentid) >1)

This query will give us the following results

Employeeid           Departmentid        NoOfRepeat
    2                                 2                           3
    3                                 2                           2

Here NoOfRepeat shows the number of times the records are duplicated in the table

Summary

This article shows that we can find the duplicate records with the help of the Group By and Having Clause.

Discover more from Technology with Vivek Johari

Subscribe to get the latest posts sent to your email.

By vivekjohari

I am currently working as a Senior Database Professional and have around 18 years of experience in database. Degree:- Master Degree in Computer(MCA) Certification course in Data Science & Machine Learning from Indian Institute of Technology (IIT), Delhi Work experience:- Designing of the database. Database Optimization. Writing Complex Stored Procedures,Functions,Triggers etc. Designing and developing SSIS & DTS packages. Designing SQL Reports using SSRS. Database Server Maintenance. Certification:- MCTS: DA-100: Analysing Data with Microsoft Power BI MCTS: DP-300: Administering Relational Databases on Microsoft Azure Microsoft certified Sql DBA in Sql server 2008 (MCTS). Microsoft certified BI professional in Sql server 2008 (MCTS). Oracle certified profession DBA in ORACLE 10g (OCP) certified profession DBA in ORACLE 9i (OCP) My other publication Technical Blog:- Technologies with Vivek Johari Guest Author and Blogger at sqlservercentral.com

Related Post

Miscellaneous SQL Server

🎉✨ Happy New Year 2025! ✨🎉

Jan 1, 2025 vivekjohari
SQL Server

Database Partitioning in SQL Server

Dec 31, 2024 vivekjohari
SQL Server

What are the important python libraries?

Dec 31, 2024 vivekjohari
33 thought on “How to find duplicate values in a table”

  4. Yes there is a way to delete duplicate records without using the distinct keyword

    Here is the Query

    with i as (select *,row_number() over (partition by Dep_ID order by DEP_ID ) as rank from emp)
    delete from i where rank>1

    Reply

  11. … [Trackback]

    […] There you can find 54101 more Information on that Topic: techmixing.com/2010/01/how-to-find-duplicate-values-in-table.html […]

    Reply

  15. … [Trackback]

    […] Find More to that Topic: techmixing.com/2010/01/how-to-find-duplicate-values-in-table.html […]

    Reply

  19. … [Trackback]

    […] Here you can find 4696 additional Information to that Topic: techmixing.com/2010/01/how-to-find-duplicate-values-in-table.html […]

    Reply

  20. … [Trackback]

    […] Information on that Topic: techmixing.com/2010/01/how-to-find-duplicate-values-in-table.html […]

    Reply

  23. … [Trackback]

    […] Read More Information here to that Topic: techmixing.com/2010/01/how-to-find-duplicate-values-in-table.html […]

    Reply

Leave a Reply

You missed

Miscellaneous SQL Server

🎉✨ Happy New Year 2025! ✨🎉

January 1, 2025 vivekjohari No Comments
SQL Server

Database Partitioning in SQL Server

December 31, 2024 vivekjohari No Comments
SQL Server

What are the important python libraries?

December 31, 2024 vivekjohari No Comments
SQL Server

Understanding Estimated and Actual Query Plans in SQL Server

December 31, 2024 vivekjohari No Comments

Discover more from Technology with Vivek Johari

Subscribe now to keep reading and get access to the full archive.

Continue reading