KARAOKE SCENE MAGAZINE ONLINE! - Removing Duplicate Songs Public Forums Karaoke Discussions Karaoke Scene's Karaoke Forums Home | Contact Us | Site Map  

Karaoke Forums

Karaoke Scene Karaoke Forums

Karaoke Scene

   
  * Login
  * Register

  * FAQ
  * Search

Custom Search

Social Networks


wordpress-hosting

Offsite Links


It is currently Fri Jan 10, 2025 3:01 pm

All times are UTC - 8 hours





Post new topic Reply to topic  [ 57 posts ]  Go to page 1, 2, 3  Next
Author Message
 Post subject: Removing Duplicate Songs
PostPosted: Thu Jan 23, 2014 1:20 pm 
Offline
Extreme Plus Poster
Extreme Plus Poster
User avatar

Joined: Fri Dec 02, 2011 12:38 pm
Posts: 6086
Images: 1
Location: Redmond, WA
Been Liked: 1665 times
I am considering eliminating multiple versions of songs that I don't believe will ever get requested. I estimate I can reduce the number of tracks I have by 60% or more by doing this. What I am looking for are suggestions on how to do this easily and quickly. All of my files are in the following format:

DiscID-Track# - Artist - Title.zip
Example - SC10001-01 - Haley, Bill & The Comets - Rock Around The Clock.zip


or for downloads which don't have a track #


TrackID - Artist - Title.zip
Example - KV287814 - ZZ Ward - Put The Gun Down.zip

Videos use the same format with the appropriate extension.

Any suggestions?

-Chris

_________________
-Chris


Top
 Profile Personal album Singer's Showcase Profile 
 
PostPosted: Thu Jan 23, 2014 1:40 pm 
Offline
Extreme Plus Poster
Extreme Plus Poster
User avatar

Joined: Fri Dec 02, 2011 12:38 pm
Posts: 6086
Images: 1
Location: Redmond, WA
Been Liked: 1665 times
I may have an answer for this (doh!)

I have always been afraid to use the Duplicate Song Finder Option in the Karoasoft KJ File Manager tool. Just didn't want to wipe out files by accident. I just ran it using the Duplicate Search and it appears to have found all files that are duplicated by Artist + Title. Pretty much exactly what I am looking for.

Now it appears I have the manual task of identifying which version of a track to keep and just Moving them unwanted files to another folder. Yet another LONG TEDIOUS task, but one that will pay off big time in the long run.

Is this what other folks have done to clean things up?
Any other better system to use?

-Chris

_________________
-Chris


Top
 Profile Personal album Singer's Showcase Profile 
 
PostPosted: Thu Jan 23, 2014 6:36 pm 
Offline
Advanced Poster
Advanced Poster
User avatar

Joined: Tue May 03, 2011 2:17 pm
Posts: 413
Location: United Kingdom
Been Liked: 38 times
I would suggest, if it's possible, to do it in such a way that you can easily call them up at the drop of a hat should a particular version get requested. Depends what the reason for removing the duplicates is I guess. If it's to try and make your computer run smoother, maybe you could put the dupes on a USB hard drive/memory stick of their own, so if you do get a "You got X version of that song? I'm set on that brand" type of singer (such as yours truly!) it's just a case of plug and play.


Top
 Profile Singer's Showcase Profile 
 
PostPosted: Thu Jan 23, 2014 8:06 pm 
Offline
Super Poster
Super Poster

Joined: Tue May 31, 2011 7:09 pm
Posts: 839
Location: Myrtle Beach, SC
Been Liked: 224 times
Chris,

I think the question is why you want to get rid of dupes? I tried that once and set up two karaoke folders in karma - one main karaoke and one extra karaoke. That way I could quickly search either of the folders. If you have a lot of SC, which I think you do, there are a ton of duplicates throughout their library.

Is it to make your book smaller? With hard drive space being so cheap I'm not sure what your motivation is to delete them. I always like having my whole library available at a show.


Top
 Profile Singer's Showcase Profile 
 
PostPosted: Thu Jan 23, 2014 9:53 pm 
Offline
Extreme Plus Poster
Extreme Plus Poster
User avatar

Joined: Fri Dec 02, 2011 12:38 pm
Posts: 6086
Images: 1
Location: Redmond, WA
Been Liked: 1665 times
I am operating at scale which means -

-overall backup/restore times can be dramatically reduced by eliminating duplicates
-training new hosts on what version to select is eliminated (I don't get too many 'must sing the Druid Performance Tracks version of "Cathy's Clown" by Reba McEntire' types)
-identify duplication to allow me to expand without further duplication (reduce music purchasing)
-sell off discs I simply will never use to recoup expenses (I need exactly ZERO BackStage discs and have something like 8 complete sets now)

....and.....

I have my own OCD ticks and this one has started to twitch.

-Chris

_________________
-Chris


Top
 Profile Personal album Singer's Showcase Profile 
 
PostPosted: Fri Jan 24, 2014 5:30 am 
Offline
Super Plus Poster
Super Plus Poster
User avatar

Joined: Thu Jan 18, 2007 6:30 pm
Posts: 1806
Images: 0
Been Liked: 631 times
chrisavis wrote:
....and.....

I have my own OCD ticks and this one has started to twitch.

-Chris


:clapper: Don't we all?

_________________
Music speaks to the heart in ways words cannot express.


Top
 Profile Singer's Showcase Profile 
 
PostPosted: Fri Jan 24, 2014 7:19 am 
Offline
Advanced Poster
Advanced Poster
User avatar

Joined: Sun Nov 13, 2011 7:37 am
Posts: 304
Location: Washington D.C.
Been Liked: 31 times
chrisavis wrote:
I may have an answer for this (doh!)

I have always been afraid to use the Duplicate Song Finder Option in the Karoasoft KJ File Manager tool. Just didn't want to wipe out files by accident. I just ran it using the Duplicate Search and it appears to have found all files that are duplicated by Artist + Title. Pretty much exactly what I am looking for.

<snip>

Is this what other folks have done to clean things up?
Any other better system to use?

-Chris

Here's what I'd do: Create a "Duplicate" folder with a subfolder called "BackStage", sort on Disc ID and MOVE the duplicate BackStage songs into it. Rinse and Repeat. Warning: "Delete" in KJFM is absolute!

_________________
Karaoke that just sounds better!

CompuHost | KJ Media Pro | Karaoke Kiosk | Bose L1 Model II system | B2 bass | ToneMatch audio engine | Shure VHF SM58 wireless mics | Countryman E6 Earset with Shure UHF wireless beltpack


Top
 Profile Singer's Showcase Profile 
 
PostPosted: Fri Jan 24, 2014 8:25 am 
Offline
Extreme Poster
Extreme Poster
User avatar

Joined: Thu Jul 10, 2008 5:13 pm
Posts: 3801
Images: 1
Location: Florida
Been Liked: 1612 times
I have removed dupes very simply by dropping all songs into 1 folder, then I remove disc data except for manu codes (SC,MM, CB, etc) using KJFM (split at 3rd character from left in disc ID and discard the right). Then when KJFM renames them with short ID names and an identical songs already exists, it will ask you to move the dupe files to another location. Done in minutes! What you will end up with is only one of every manu that you have, and your spares will still have the full ID.


Top
 Profile Personal album Singer's Showcase Profile 
 
PostPosted: Fri Jan 24, 2014 4:37 pm 
Offline
Super Extreme Poster
Super Extreme Poster
User avatar

Joined: Mon Dec 10, 2001 3:57 pm
Posts: 22978
Songs: 35
Images: 3
Location: Tacoma, WA
Been Liked: 2126 times
I agree with not deleting from the hard drive. Books, yes.
True 9 out of 10 times you are going to go to your go to pick that you feel is the best version, but it's nice when you can offer another version to someone who is used to something else and doesn't like the go to pick.

_________________
LIKE Lonman on Facebook - Lonman Productions Karaoke & my main site via my profile!
Image


Top
 Profile Personal album Singer's Showcase Profile 
 
PostPosted: Fri Jan 24, 2014 10:57 pm 
Offline
Extreme Plus Poster
Extreme Plus Poster
User avatar

Joined: Fri Dec 02, 2011 12:38 pm
Posts: 6086
Images: 1
Location: Redmond, WA
Been Liked: 1665 times
Lonman wrote:
I agree with not deleting from the hard drive. Books, yes.
True 9 out of 10 times you are going to go to your go to pick that you feel is the best version, but it's nice when you can offer another version to someone who is used to something else and doesn't like the go to pick.


I would feel the same way if I only ran a single rig. But it just isn't cost effective to have every rig be identical. And the backup/restore process across so many rigs is very time consuming.

Not to mention, where we host, we have that younger crowd that is far less sensitive to brands than the older generation of singers.

-Chris

_________________
-Chris


Top
 Profile Personal album Singer's Showcase Profile 
 
PostPosted: Sat Jan 25, 2014 12:13 am 
Offline
Extreme Plus Poster
Extreme Plus Poster
User avatar

Joined: Wed Feb 07, 2007 1:12 pm
Posts: 5046
Been Liked: 334 times
Chris, pardon me if I'm asking a dumb question, but after reading the thread I still have no idea what songbook software you are using- that is, if it's the BOOKS from which you wish to delete. I'm assuming ( being PC based) you want to keep dupe versions from multiple mfrs. on your HD.

_________________
"No Contests, No Divas, Just A Good Time!"

" Disc based and loving it..."


Top
 Profile Singer's Showcase Profile 
 
PostPosted: Sat Jan 25, 2014 1:49 am 
Offline
Major Poster
Major Poster

Joined: Sat Dec 28, 2013 1:19 am
Posts: 50
Location: From UK but living in Cyprus
Been Liked: 1 time
I Export list to excel and remove dups from my book only

_________________
facebook - PJs Karaoke Disco


Top
 Profile Singer's Showcase Profile 
 
PostPosted: Sat Jan 25, 2014 3:20 am 
Offline
Extreme Plus Poster
Extreme Plus Poster
User avatar

Joined: Fri Dec 02, 2011 12:38 pm
Posts: 6086
Images: 1
Location: Redmond, WA
Been Liked: 1665 times
I am removing the duplicates from my hard drive. I removed probably 98% of the dupes from my books a year ago. Now I want to clean up hard drives for the reason above.

-Chris

_________________
-Chris


Top
 Profile Personal album Singer's Showcase Profile 
 
PostPosted: Sat Jan 25, 2014 12:26 pm 
Offline
Super Duper Poster
Super Duper Poster

Joined: Wed May 18, 2005 10:03 pm
Posts: 2674
Location: Jersey
Been Liked: 160 times
If I wanted to pare down my library for a particular hard drive, I would just put the major brands on it like SC, DK, PHM, Legends and Pocket Songs. That would be my core. There would still be some duplicates but you would have most people covered. To that I would add the KV tracks that I have purchased because I probably don't have any other versions of those songs. There would be certain songs that my regulars sang on a steady basis so I would also add those songs. Then I would keep track of any song that a customer requested that wasn't on my internal laptop hard drive and add those songs as they were requested. If someone asked to sing a song from a Monster Hits disc, I would add that entire disc to my main drive unless I was running out of space. After a short period of time, you probably wouldn't be accessing your external drive for many songs at all and you probably wouldn't even have to hook it up on most nights. Seems to me that it would take a lot less time than checking through many thousands of individual files and deleting them one at a time.


Top
 Profile Singer's Showcase Profile 
 
PostPosted: Sat Jan 25, 2014 1:54 pm 
Offline
Super Poster
Super Poster

Joined: Tue Jan 24, 2006 7:34 pm
Posts: 1227
Location: Completely Lost
Been Liked: 15 times
Are all the songs in the same folder?

_________________
Okay, who took my pants?


Top
 Profile Singer's Showcase Profile 
 
PostPosted: Sat Feb 15, 2014 4:11 am 
Offline
newbie
newbie
User avatar

Joined: Sat Feb 15, 2014 3:46 am
Posts: 1
Been Liked: 0 time
I used DuplicateFilesDeleter and I was extremely pleased with the result.


Top
 Profile Singer's Showcase Profile 
 
PostPosted: Mon Feb 17, 2014 12:33 am 
Offline
Major Poster
Major Poster
User avatar

Joined: Tue Feb 20, 2007 9:38 am
Posts: 77
Location: Fort Wayne, IN
Been Liked: 3 times
Song List Generator from Karaosoft could help. Import all of the karaoke tracks, toggle the show all versions button, The Versions column will list all the different brands you have. You can export a list as a csv or xls file.

Ironically, it does not show you if you have more than one of the same brand. :roll:

_________________
Swing Time Karaoke


Top
 Profile Singer's Showcase Profile 
 
PostPosted: Mon Feb 17, 2014 4:29 am 
Offline
Extreme Poster
Extreme Poster
User avatar

Joined: Thu Jul 10, 2008 5:13 pm
Posts: 3801
Images: 1
Location: Florida
Been Liked: 1612 times
I've said it earlier, but here it is again.
1) Put all your songs in one folder and tag them with proper disc, artist and song info.
2) Use KJFileManager (Karaosoft) and strip all data from the disc id except for the Manu code (SC CB MM, etc)
3) When done all the dupes will return a message "cannot perform request when the file already exists, what would you like to do with those unprocessed files" (this is the jest of the quote)
4) Move those files that still have ALL the disc ID info intact to a "dupe 1" folder
5) Go back to the files that have the stripped disc ID and read the tag and perform "tag to file" operation
6) You now have only 1 of every manufacturer in the list and if you are trying to maximize your songs for multiple rigs this would work very well (Chris).

It is very simple and only requires waiting for the results and no tedious hand sorting. You can perform the same task with your "dupe 1" folder and remove the remaining dupes to "dupe 2" folder ...... rinse and repeat.

I have around 4500 unique cd's and a massive amount of dupes within 1 manufacturer, so this dramatically reduced the searched field.


Top
 Profile Personal album Singer's Showcase Profile 
 
PostPosted: Fri Nov 21, 2014 1:53 am 
Offline
newbie
newbie
User avatar

Joined: Fri Nov 21, 2014 1:51 am
Posts: 1
Been Liked: 0 time
DuplicateFilesDeleter works excellent to find duplicate files and delete them quickly than other software. You may also try it and I m sure you will be happy too. Thank you!


Top
 Profile Singer's Showcase Profile 
 
PostPosted: Tue Nov 25, 2014 1:45 pm 
Offline
Novice Poster
Novice Poster

Joined: Thu Sep 04, 2014 8:06 pm
Posts: 38
Been Liked: 10 times
For me, I use KJFile manager to edit the songs to the format DISCID - Artist - Title. I pull a list of my current collection with SongListGenerator and filter out the duplicates. Anything I add to my collection is checked to ensure the title and artist is matched precisely for all manufacturers. The benefit to doing this is SongListGenerator will spit out a song list with no duplicates, but can easily show the different manufacturer versions


Top
 Profile Singer's Showcase Profile 
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 57 posts ]  Go to page 1, 2, 3  Next

All times are UTC - 8 hours


Who is online

Users browsing this forum: No registered users and 456 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Powered by phpBB® Forum Software © phpBB Group

Privacy Policy | Anti-Spam Policy | Acceptable Use Policy Copyright © Karaoke Scene Magazine
design & hosting by Cross Web Tech