Enable synchronisation of favourites
frobijn opened this issue · 73 comments
(This request resembles #1613 but the export/import from that issue would not be sufficient)
This request: please make it possible to synchronize favorites among devices. This is not a request to implement the synchronization itself - I can understand that it is impossible for Osmand to support all possible solutions (external SD card, DropBox, OneDrive, Google Drive...). But the way Osmand handles favorites at the moment makes it very cumbersome to perform any form of synchronization, even with a third-party app. The request is to modify the Osmand behaviour to allow for easy synchronisation, and preferably allow automated synchronisation (e.g., via a third-party app).
Some background: I'm a huge fan of Osmand and I'm using it extensively on holidays. It's great to have an offline map/navigation app. I use the favourites a lot to mark the location of hotels, train stations and other points that I want to visit, have to change trains or buses etc. Especially in countries where you cannot read or understand the local language, having a map that tells you whether you are close to your target is very practical. So while traveling I tend to add, move, remove and rename favorites a lot.
I'm using Osmand on two or three devices (tablet, new phone for safe areas, old phone as backup and for less secure areas), more or less at the same time. The tablet is great for planning itineraries, the phones for navigation and ad-hoc route planning, and the tablet is also a backup in case the battery of the phone is empty. But then it should be easy to distribute changes to the favourites made on one device to the other two.
With the current Osmand version that is impossible, or at least it is a nightmare.
Exporting the favourites is possible. Go to the list of favourites and use the share option to export the file. It is possible to save the favourites.gpx to Dropbox, or to save the file to an external location (memory card/USB stick that is connected via USB). (It is not possible (on Android 5.x) to save the favourites to a file on the internal/external SD card, probably because of security issues).
The main problem is importing the favourites, because Osmand always prefers the data on favourites that is stored in its internal database. There are several ways to import the favourites.gpx:
- Double-click the .gpx file. Osmand is one of the options to open the file, the select the option to import the file. This will import all favourites from the file, but the favourites already present will not be deleted or modified. You've got most of the favourites twice; only new or renamed favourites are present only onces, as are the favourites you wanted to get rid of. This is not the solution.
- You can delete all favourites first. That's a bit of a nightmare, as you have to be very sure that you do not throw away any modifications that were not yet present on the other device. Then double click the .gpx file and import it. All favourites are present in the correct categories, but the colors associated with the categories are lost. This is not the solution.
- Copy the .gpx file in the files directory of Osmand, force Osmand to stop, restart Osmand. Now all new favourites are present, but the existing ones are still present and they do not get updated as Osmand ignores the data for those in the .gpx file.This is not the solution.
- Delete all favourites, save the .gpx file in the Osmand files directory, force Osmand to stop restart Osmand. This works!
The last one works, but it is quite cumbersome and there's plenty of room for error.
Personally, I would prefer a third-party app solution: create an app that synchronizes two versions of a favourites.gpx (allowing for changes in both versions), and updating the collection of favourites in Osmand. (I could make such an app, I've built similar software.) Such an app could automate the process, but... Osmand won't allow that. If only all favourites were writen to the favourites.gpx, then such an app could read the data without user intervention. If only Osmand would (have an option to) use the data from the external (favourites.gpx) file if it exists instead of from the internal data file, then modifications could be communicated automatically to Osmand. If only there was a way (e.g., via the Osmand api) to reinitialise the favourites from the external data, then such an app could do its work without the user having to force Osmand to stop.
OsmAnd doesn't delete missing favorites on purpose in order to not loose anything wanted. Full synchronization of favorites will require mechanism how user can undelete favorites and this is not discussed how to implement yet. Today it is possible to recompile osmand https://github.com/osmandapp/Osmand/blob/master/OsmAnd/src/net/osmand/plus/FavouritesDbHelper.java#L76
Another Workaround (OsmAnd stores 2 gpx files for favorites 1 on external sd card another in internal memory):
A) Delete internal GPX file
B) Synchronize internal GPX file as well
I had already looked at the source code to figure out what's happening. Your suggestion for another workaround is indeed possible, but then I have to root all devices as an app or user can't access the internal file without root privileges.
Agree, we need to enable somekind of developer settings which will be easy to set in the app. One of that setting should be use only external storage for favorites, so it will be easy to synchronize.
+1 for me, would be a great feature
+1 for me too - I plan on the tablet and bike with the phone
Friendly bump here (I also use it for both car driving and biking, so that would be really useful) +1
+1 for me also, have it on three devices
there is 1 other navi app i'm aware off that offers such a feature, but for various reasons i like osmand better.
osmand provides a nice grouping feature as well. but even if you are willing to edit/merge/convert favourites witha gpx editor, handling of this information is quite cumbersome (at least i didn't find a good editor). another idea was to convert the GPX to chrome bookmarks, do cleansing there and convert back, hoping that i could convert bookmark folder to osmand favourite groups...
anyhow, getting a better synchronization would be more than "nice to have" for me.
Maybe the following can be helpful:
See the latest new feature on https://overpass-turbo.eu ...
You can store now your queries in an API by openstreetmap.org!
That means: log in with your OSM account and have some storing space for small data sets.
Maybe it is possible to store the favourites.gpx file there as well?
Would be really interested to have this feature. Having the ability to move preferences/settings/favorites between devices will save users lots of effort with new or multiple devices.
+1, this is really a missing feature.
This is the weakest link in Osmand, an otherwise amazing project. A lot of us are building huge GPX datasets and rarely backing them up, let alone syncing, because of this problem. Not only "nice to have" but needed urgently.
I agree, it is something that has been missing for a really long time.
Optional sync of user data with cloud is almost the standard now and GPX are exactly the kind of data users need to sync and back up. It is accumulated sporadically and over long time, edited on different devices, not audited often and you usually can't predict when you will need to find that place you marked two years ago. Countless times I saved a POI just in case, long time passed, then I needed it - and of course it was on another device, lost due to device reset and/or my failure to back it up.
I would see it as optionally automatic Dropbox/Google Drive synchronization. Those who want to be extra careful about not overwriting existing GPX from the cloud can leave it off and keep doing it manually. Casual users will just turn it on and forget about it. Some people will combine the approach - setting up sync but periodically exporting GPX manually after important edits. I don't think undelete functionality is really necessary to implement before cloud sync.
+1, absolutely necessary for travelling
+1 would like to sync as well...
+1 absolutely necessary
+1 transitioning between phones is a pain, and loss of trek data hurts
+1 without this "favorites" are effectively tied to the device and are therefore temporary.
Very important for me, too. I have over 300 favourites which I use and change constantly and there is no way to sync them on different devices and no save place to store them constantly.
Favourites are of course a kind of private layer above the public informations but I think it is a very important layer. Historically OSM comes from public sharing, but not every favourite is worth being published and valid to be shared with everybody.
+1 from me
PSA: Please avoid "+1" messages on GitHub issues, that's what the reactions (the "+☺" above comments) are for. Otherwise subscribers end up with a lot of pointless notifications.
I tried to ask for this in #2636, but was misunderstood / didn't find the right words. All it'd take is to have seperate files for each favourite. Because these files are so small it could just be added and the favourites.gpx could be left as is. An additional folder "favourites" with subfolders of groups that would be in sync with the content of favourites.gpx and prioritized if there are conflicts. Could be an add-on "Favourite Sync Folder" or an option in the developer add-on.
P.S.: I can sync favourites.gpx itself, but this doesn't handle conflicts, if both devices add a favourite without being online.
any updates on this request?
please!
Yes, it is planned for 2019
Please make synchronization.
Hey! Nextcloud Maps development is currently in progress. There already is a REST API to manipulate favourites. It would be great to create an OsmAnd plugin able to sync favourites with Nextcloud Maps. Being able to sync with a privacy-friendly solution as Nextcloud Maps would be a killer feature IMHO.
This API has been discussed in nextcloud/maps#30
If I find time and motivation, I could do it in a few months. If someone wants to do it, feel free to ask for some changes in Maps favourites API if necessary.
I also have multiple devices running OSMand and keeping the favorites synced is very labor intensive. This would be a very nice feature to have.
Yes, it is planned for 2019
Any update on this, or a roadmap we can look at? (sorry to start 2020 with a push!)
Yes, it is in roadmap for 2020, for sure. We are looking to synchronize with profiles and everything.
@vshcherb
Please make also a export / import or sync for private custom search filters. Until now they are the only one without any workaround for share / im-ex / sync among devices.
Thanks a lot !
Peter
Sync with OsmAnd would be great!
As an easy workaround you can sync the favourites.gps file with a tool like syncthing. Syncthing allows you to sync folders in your android's filesystem to your PC. and there are probably many other tools which do a similar job.
Perhaps there is a misunderstanding here. As OP said:
This is not a request to implement the synchronization itself - I can understand that it is impossible for Osmand to support all possible solutions
And as @biiiep just said:
As an easy workaround you can sync the favourites.gps file with a tool like syncthing.
Really, that is not a workaround, that is the solution. Syncing should be left to a separate sync tool, not reinvented by Osmand. Favorites have a standard file format for data exchange: GPX. And already Osmand uses GPX for both favorites and markers. The favorites.gpx file is kept updated by Osmand, making it fine for one-way sync already. (Markers unfortunately still require manual export.) But OP seems to suggest that Osmand does not read the file, preferring an internal database.
So what is needed is simply for Osmand to keep the favories.gpx file fully synced with its internal database. Then users will be able to sync the favorites.gpx file using whatever sync tool they prefer.
Correct, @vshcherb ?
Really, that is not a workaround, that is the solution.
It's neither a workaround nor the solution. That's what I'm doing until now, but it has too many limitations. Favourites are saved in one file. So all syncing conflicts will result in one of two changes being lost. Changing that is the whole point of this issue. To make it easily syncable, we'd need e. g. folders for favourites groups and one file for each favourite. That's what I was asking for in #2636.
(Optionally) storing these not in the app folder, but in a separate folder "Locations" on internal storage would even make it possible for other navigation apps to implement it the same way, so we could have consistent location favourites even across apps.
Sorry, I can't understand the problem.
I synchronize twoway the files and folders I want from OSMAND with "OneSync" to my OneDrive.
All Android, iPhone and PC are every time the same.
I also don't see a real problem. Only in three cases there is some manual work necessary
- delete of one or more points
- change of a point name which leads to "semi duplicates"
- move of a point to a different group
Theese cases are not easy to handle programmatically, because it is not really clear, how the user wants this to be solved.
Sorry, I can't understand the problem.
I synchronize twoway the files and folders I want from OSMAND with "OneSync" to my OneDrive.
All Android, iPhone and PC are every time the same.
Have two devices offline. Add an individual favourite on each. Go online with both.
...
Another Workaround (OsmAnd stores 2 gpx files for favorites 1 on external sd card another in internal memory):
A) Delete internal GPX file
B) Synchronize internal GPX file as well
Where is this internal file? Even on my rooted device (Android 6.0.1, osmand+) I can't find it. This would help as a workaround, as I don't have much hope any solution is going to be implemented any time soon.
Hey! Nextcloud Maps development is currently in progress. There already is a REST API to manipulate favourites. It would be great to create an OsmAnd plugin able to sync favourites with Nextcloud Maps. Being able to sync with a privacy-friendly solution as Nextcloud Maps would be a killer feature IMHO.
This API has been discussed in nextcloud/maps#30
If I find time and motivation, I could do it in a few months. If someone wants to do it, feel free to ask for some changes in Maps favourites API if necessary.
Is it possible to integrate the favorit sync as an own app? I'm unshure about the OSMAnd-API https://github.com/osmandapp/osmand-api-demo. There are a bunch of methods discribed to add and modifify favorits. This would be enough to get the Nextcloud maps favorits into OSMAnd, but I'm missing functions to get the favorits or get notified if favorits change to upload them to nextcloud.
Any suggestions?
It is possible to add these API methods, I think there are methods to get favorites or probably we can introduce 2 methods - "Last time favorites has changed" + "Access to raw Favorites.gpx.gz"
How diffecult would a
registerForFavoritsUpdates
be? Would be more efficient otherwise the app would need to pull with a certain intevall, or?
if app is running obviously it is eaiser and should be done though 2nd periodic check on start still need to happen
I don't know if you are aware of it, but if/when OSMAnd will have this kind of sincronization, maybe it could be listed also in the Nextcloud appstore: https://apps.nextcloud.com/
This thaks to nextcloud/appstore#95
This would not mean that the users could install OSMAnd via the Nextcloud appstore, yet for sure it would be another way to gain new users via the awareness of OSMAnd existance and capabilities.
Just a thought ... but in the interim whilst this is hopefully being sorted -
Would it be easy / relatively simple for an upcoming version of OsmAnd to include a "Select All" button / icon somewhere in the "My Places" section, AFTER clicking on the "Bin" icon (next to the Bin at the top)?
That way, we wouldn't have to click on each individual Favorite group before deleting, prior to importing the newer favorites.gpx. I've ~ 50+ groups to delete individually and on occasions, I accidentally miss the tick-box and cause the list to pop open, making deletion even more frustrating.
Just a thought 🥴
👍
I have the same issue. The internal database should update itself with respect to the changes in favourites.gpx.
Might I propose a solution based on DecSync? I uses the filesystem to synchronize the favorites, like others suggested. But it is designed in such a way that there are no file conflicts.
Disclaimer: I made it myself.
I have now actually made a proof of concept of OsmAnd DecSync, which is an OsmAnd plugin that uses DecSync to synchronize the favorites! Note that it is a proof of concept, as it requires root and heavy manual intervention, as there is currently no way for a plugin to access the favorites.
To make it work correctly, this access has to be added. It is sufficient to add a method to get the time of the latest update and a way to access favourites.gpx, as was proposed previously.
Furthermore, the plugin makes use of ids to identify favorites and categories, but OsmAnd cannot store them. Is is possible to add this possibility? Currently, the plugin guesses based on the name,color etc., but storing it directly would make it simpler.
@vshcherb Any update on the introduction of the methods "Last time favorites has changed" and "Access to raw Favorites.gpx.gz"?
@39aldo39 -- it is actually possible to configure OsmAnd to write its data to shared memory which is accessible by other apps without requiring root access. Settings -> OsmAnd settings > Data storage folder > Shared memory. this is what i use in order to do synchronization with syncthing.
@39aldo39 -- it is actually possible to configure OsmAnd to write its data to shared memory which is accessible by other apps without requiring root access.
Settings -> OsmAnd settings > Data storage folder > Shared memory. this is what i use in order to do synchronization with syncthing.
Indeed, this provides one-way synchronization. The favourites.gpx file is updated every time a favourite is added/modified/deleted, so the OsmAnd favourites can be shared with other applications via this file. The other direction (updating favourites in OsmAnd without user interaction) is still not possible in an automated way.
@39aldo39 -- it is actually possible to configure OsmAnd to write its data to shared memory which is accessible by other apps without requiring root access.
Settings -> OsmAnd settings > Data storage folder > Shared memory. this is what i use in order to do synchronization with syncthing.
Thanks for the information! I would still prefer it if it wasn't necessary, but it is still acceptable. I updated the plugin to work using the shared memory functionality! The setup is a bit more complicated, but after that it should work automatically. I have even published a v0.2 release with an APK to make it easier to try out!
Same problem. I wrote a node.js utility that converts my Google Earth favorites into GPX, then pushes to /files/favourites.gpx and /files/tracks/, but indeed, in order to simply update icon color you have to first remove existing favorite.
PROPOSED SOLUTIONS:
- expose favorites DB, so we could just remove the file (either manually, with
adb, or other tool) - add support for
/files/commands.txt- upon app load commands would be executed, e.g.WIPE_FAVORITES_OUTwould clear your favs - add another tag support to
<extensions>, e.g.
<extensions>
<osmand:background>circle</osmand:background>
<osmand:action>REMOVE</osmand:action> // or UPDATE, whatever
</extensions>@DamianoPantani what if you move them to a category called trash and keep it disabled? yes, we also could treat special coordinates to delete points like lat/lon=0.0001/0.0001 or tag in extension
@vshcherb that would do the trick but I can only move favorites manually. Can I do it from the favourites.gpx level?
Hm... than I think I'm wrong with that idea and it needs to be implemented first.
Another year, same question:
I read all posts until here - but I still do not understand, if there will be a "real" syncronizing functionality (not just copying the favorites-file from A to B and back) coming? Stuff it in the "pro"-subscription, I would take it. But not having a real sync option is one of the bigger issues osmand has.
Depends what's real and what's not real synchronization, updating / adding points - works but deleting points is not yet.
Meanwhile OsmAnd has a backup/restore feature and OsmAnd Pro has a cloud feature. So technically you can just backup your favorites (or tracks, markers, settings, ...) on one device to the cloud and restore it on another device from the cloud. This is quite similar to a synchronization.
If deletion doesn't work yet this is a pity, though.
Once we implement timestamp tracking / deleting points / incremental synchronization, all possible solutions will benefit from it.
Meanwhile OsmAnd has a backup/restore feature and OsmAnd Pro has a cloud feature. So technically you can just backup your favorites (or tracks, markers, settings, ...) on one device to the cloud and restore it on another device from the cloud. This is quite similar to a synchronization.
"quite similar" is ecactly not "my" definition of "real" sync. For just copying files around I dont need a "pro"-version, using the osmand cloud.
I would like to add that a sync feature would also allow to share favorites with a partner. Are there other people interested in an integration with next cloud?
I would like to add that a sync feature would also allow to share favorites with a partner. Are there other people interested in an integration with next cloud?
I´m interested in every solution that will give me the possibility to sync favourites - and in the best case also tracks - between cloud and my devices!
OP still has the best idea to begin resolving this whole issue:
The favourites.gpx file is updated every time a favourite is added/modified/deleted, so the OsmAnd favourites can be shared with other applications via this file. The other direction (updating favourites in OsmAnd without user interaction) is still not possible in an automated way.
Forget database access, and ADB, and anything Nextcloud-specific, for now.
Just make favourites.gpx and any other tracks writable from outside the app. In other words, link them directly to the state of the DB. Caching where necessary, of course.
Just make
favourites.gpxand any other tracks writable from outside the app. In other words, link them directly to the state of the DB.
There is one large drawback: OsmAnd only adds favorites from this file but never deletes them if they aren't contained in favorites.gpx.
There is one large drawback: OsmAnd only adds favorites from this file but never deletes them if they aren't contained in favorites.gpx.
Yes, that was exactly my point... If favourites.gpx is used to store the state of the POIs, either instead of the DB or in regular sync with it, then what you see on the screen will always reflect what is in the file.
Yes, that was exactly my point... If
favourites.gpxis used to store the state of the POIs, either instead of the DB or in regular sync with it, then what you see on the screen will always reflect what is in the file.
Could be a good start indeed, I will have a look at the favourites.gpx and see what information is possible to parse.
im wait sync too
Hey! Nextcloud Maps development is currently in progress.
If I find time and motivation, I could do it in a few months. If someone wants to do it, feel free to ask for some changes in Maps favourites API if necessary.
@julien-nc I would love that, any way I can support?
Similar discussion is ongoing for Organic Maps, if both implement sync I would be one very happy deGoogler :)
organicmaps/organicmaps#622