vaeth/bookmarkdupes

Sometimes too many bookmarks are removed?

vaeth opened this issue · 12 comments

vaeth commented

This bug is opened as a reaction on a review on mozilla.

For some not yet clarified reason, with many duplicate bookmarks and "automatic" selection more than only duplicates have been reported to be removed.

It was conjectured that the number of bookmarks is related, but I tested with huge numbers; therefore, it is unlikely that the number alone is the reason.

I consider it also very unlikely that bookmark dupes instructed firefox to remove a bookmark which was not marked. This leaves the following possible causes for the bug:

  1. Too many bookmarks have been marked. I consider it very likely that the code for marking is correct. So very likely this could happen only if none-duplicates had been displayed in one group. Possible reasons.

    • Perhaps "similiar dupes" were selected by accident instead of "exact dupes"?
    • A bug in the code for calculating or displaying the groups of duplicates.
      So far, I have not been able to detect such a bug. If somebody finds an example please inform me (e.g. by emailing me a bookmark file where the problem occurs).

  2. The firefox code for removing a certain bookmark id removes might be buggy and remove the wrong (or more than 1) bookmark (e.g. the bookmark id is not unique as it should be). Possible, but I consider this also quite unlikely.

Edit: For testing 1 (b), it might be useful to use bookmark-dupes-2.4, because this version displays the number of marked bookmarks (you do not have to actually remove bookmarks for counting...).

This version is currently only in the master branch (and you have to load it with about:debugging), because I will wait some days for the spanish and chinese translation before publishing it on mozilla.

vaeth commented

bookmark-dupes-2.4 is now released on mozilla

yorksille commented

Almost 7000 bookmarks and over 100 folders, subfolders, subsub folders, etc. which FF screwed up long ago with tons of dups -- bookmarks AND folders. Finally trying your extension to see if I can clean all this up. Some dupes in different folders are intentional.

Anyway, I tried BD on one folder. It deleted almost everything. Most were not dups.

Here's one example. It deleted both of them along with most of the rest of the bookmarks in this folder. (I restored from backup.)

image

vaeth commented
  1. Did you enable expert mode? (I wonder how you get the few with the common url...)
  2. Are you sure that there are really no two bookmarks with the same name and ip adress in the same folder?
  3. Do you see the same list when you press the "Dupes" button again?

I never have seen the exactly same bookmark reported twice in a window when it was not really present twice.
If this is really the case (and the answers to the question are N - Y - Y), I have only the explanation that firefox has completely messed up your stored bookmarks and reports them in a loop or something similar. As I am still not able to reproduce the problem, I do not know how to help.

yorksille commented
vaeth commented

After you mention xmarks and how your duplicates arose, it seems now more likely to me that your bookmarks file is seriously hosed (and e.g. doing something inconsistent when reading and writing).

We can try of course, though I will have time for non-trivial experiments only on the weekend, and I am not optimistic that I can find a solution if my above conjecture turns out to be correct. If you are willing to share the file, please send it by pm (briefly before the weekend if you do not want to risk that it is on my disk until then.)

yorksille commented
vaeth commented

What I would try is to copy the latest bookmark backup file from the folder. Then start up with a completely clean profile (in particular, not yet switching on syncing) and restore the backup file: If you have good luck, the backup file itself is not broken, and then tools like bookmarkdupes might work (or might not even be necessary anymore). Alternatively, instead of the backup files, you can try to export/import the bookmarks are HTML though that will be more lossy.
A problem is: Once you switch on sync, the issue might return as the broken data might be on the server...

yorksille commented
vaeth commented

I realized that there is also a misunderstanding:

It deleted both of them along with most of the rest of the bookmarks in this folder.

From each group of duplicates all are displayed. If you selected all of them for deletion, of course all of them are deleted, and none will be left. You must leave at least one of them in each group.

yorksille commented

#2 (above) was unchecked as with all the others unchecked bookmarks that were deleted. I saw one of each dup (of those I looked at) left unchecked before I deleted what was checked. (I only copied this one example.) As you said, apparently a problem with the FF file.

yorksille commented

I exported the bookmarks to HTML. It's a mess. All kinds of folders and URLs duplicated that aren't showing that way in the bookmarks, old folders and URLs deleted long ago that are there, etc. I'll try importing to Chrome and see what I get and if that doesn't work will try your suggestion. Thanks for the ideas on how to fix this!