I have an old table with legacy data and approx 10,000 rows and a new table with about 500 rows. The columns are the same in both tables. I need to compare a few columns in the new table with the old one and report on data that is duplicated in the new table.

I’ve researched articles with similar issues, attempted table joins and where exists / where not exists clauses but I just can’t get the SQL right. I have included my latest version.

One issue causing trouble for me, I think, is that there is no “Key” as such like a userid or similar unique identifier in either table.

What I want to do is find the data in the “new” table where all rows except for the “reference_number” (doesn’t matter if it does or does not) is duplicated, i.e. exists already in the “old” table.

I have this so far…

select 
old.reference_number
new.reference_number
new.component
new.privileges
new.protocol
new.authority
new.score
new.means
new.difficulty
new.hierarchy
new.interaction
new.scope
new.conf
new.integrity
new.availability
new.version
from old, new
where
old.component = new.component
old.privileges = new.privileges
old.protocol = new.protocol
old.authority = new.authority
old.score = new.score
old.means = new.means
old.difficulty = new.difficulty
old.hierarchy = new.hierarchy
old.interaction = new.interaction
old.scope = new.scope
old.conf = new.conf
old.integrity = new.integrity
old.availability = new.availability
old.version = new.version

I have tried this here but it doesn’t seem to pull out ALL of the data for some reason.

It is evident that actually there are MORE rows in the old table that are duplicated in the new table but I’m only getting a small number of rows returned from the query.

Can anyone spot why that might be, is there another way I should be approaching this?

If it matters, this is Postgresql.

Thanks for any help given.