Unable to successfully complete profile export data
dredmorbius opened this issue · 15 comments
I'm opening this issue based on multiple reports by other users with several impending Diaspora* pod shutdowns and export / archival of data being necessary.
I am not experiencing this issue myself.
I do not have specifics concerning the mode of failure.
See discussion at, e.g., https://diaspora.glasswings.com/posts/f605e81053c3013ae6b9002590d8e506 (Archive)
Profiles pavi@joindiaspora.com (Pavithran S ) and schestowitz@joindiaspora.com (Dr. Roy Schestowitz) both report that they are unable to successfully complete data export from the pod joindiaspora.com. Dr. Roy Schestowitz claims to have about or over 500,000 posts dating back 10+ years. Based on my own profile history (~2,800 posts, 18 MB archive, beginning in 2012), I would estimate Schestowitz's total export to be several GB of ZIPed data, and roughly twice that unzipped.
The specific reasons or modes of failure are unknown.
The pod administrator, Lukas Matt, has been generally unresponsive to communications either on Diaspora* itself or by email. The pod's support email address has been generally unreachable since at least September 2021.
All that said, ensuring members can recover their data is a core principle of Diaspora*, and steps should be taken to see that this is generally supported.
Possible issues and/or remedies would be:
- Ensuring that large and/or old profiles can successfully complete data export.
- Providing additional estimates as to time to completion, or reasons for failure, of data export requests.
- Providing a mechanism to request partial or segmented data exports, e.g., by year boundaries, or limited to a specific maximum file size.
- Providing mechanism for exports via another designated hub.
My last suggestion might be part of a general account migration feature, in which an OLD account could designate one or more NEW successor accounts, and ensure that both post AND comment data, as well as direct message conversations, are replicated to those new accounts. Data export of the OLD account might then be initiated from the NEW account based on such federation.
This of course carries numerous implications, including security and potential account or content hijacking.
But in the meantime, there's a 1 March 2022 shutdown looming and multiple users who seem unable to export their data.
The podmins of joindiaspora.com need to provide reasons for why this is failing, like log files or error messages, we cannot use crystal balls to imagine what's going wrong. A user account created on 2010 on my pod was able to export their data just fine, so this is not a general bug in the software, but rather something specific to joindiaspora. Therefore, this issue is non-actionable.
The design and implementation of the migration feature has been discussed in detail, and the solution that's currently planned is the best one everyone could come up with in regards to all the "implications" you mentioned. You're several years late to participate in that discussion, but if you feel like it, check out the current implementation thread on discourse, the implementation work that's already done, the specifications work done, and participate there. Discussions are also not actionable issues, which is why the project uses Discourse in the first place. :)
The podmins of joindiaspora.com need to provide reasons for why this is failing...
The podmin has been unresponsive, as noted. They've been specifically invited to comment on this issue.
I opened this issue so that users could post specific details, on the project tree itself (and not on yet a third platform, Discourse), of what their issues are.
My Tardis is in for repairs, and of course, the anticipated completion date is a complete hash. Expecting users to travel several years back in time and saying so baldly on this thread doesn't make that viable. The issue may also affect others in future.
I'm well aware that a solution prior to shutdown is a long shot. It's somewhat less a long shot than several other possibilities being discussed, and seems to most advisable course of action.
I'm requesting the issue be reopened as informational so that the input from both users and podmins can occur.
Thank you.
Even if they had a crystal ball to find out what the problem is, if the podmin is unresponsive then how are you going to fix it?. I mean you need the credentials to log in to the server and fix it otherwise it is useless to have a solution. Try to get in touch with the podmin in other ways...
@P3lUZa One possiblity, provided by having an open issue, is that the podmin (Lukas Matt) might, as he's been invited to do, provide that information on this issue.
Please reopen the issue.
Tag it as informational or needs information or whatever. But don't close it.
I'd be willing to accept closure after 1 March 2022, or the final closure of JoindiasporaCom, after which time the specific information in this case will no longer be available.
Can we accept this as a viable compromise?
As soon as there is an actionable issue, as in something that we can look at and fix, this issue will be reopened. As of right now, this is not the case: there is nothing for us to do here.
If there's a Discourse thread or threads relevant to this discussion could they please be linked here?
Thanks.
There is no Discourse thread on the state of joindiaspora, no. Running pods is not the project team's responsibility, so I don't see why we should be discussing this.
And just to add some words for other people finding this issue: yeah, it sucks, I know. Joindiaspora being broken is nothing new -- I personally offered to take over maintenance of that pod in 2019, which never was reacted to. Since then, a couple of incidents happened there that resulted in an unknown amount of profile pictures and post media being lost, as well as an incident that resulted in an unknown amount of users being unable to sign into their account or reset their passwords.
While everyone in the project team can understand your pain, there is absolutely nothing we can do here. The decision to hand over joindiaspora to the folks who're maintaining it now was made a long time ago. At that time, this made sense because having a large pod run by the same people who build the software somewhat defeats the whole idea of decentralization. In hindsight, this might not have been the smartest move, and we probably should have kept the pod under our maintenance, but there time travel has not been invented yet.
No matter how much we want to do something, we can't. We do not have access to the server and/or any user data. We are unable to export your data, and we are also unable to figure out why exports fail in the first place. Unless someone with admin access to joindiaspora provides usable information, for example error messages, we cannot do anything here, and starting new threads and issues about this is not productive.
@denschub Not to put too fine a point on it, setting up expectations, failing them, and declaring the topic irrelevant to devs is precisely the sort of bad look Diaspora*'s had for a long time which despite my own long-time association with the network makes me question whether or not that's a productive use of my time.
Contrast the viewpont of the Debian Project, with which I've likewise been a long-time user and some-time contributor:
Our priorities are our users and free software
We will be guided by the needs of our users and the free software community. We will place their interests first in our priorities. We will support the needs of our users for operation in many different kinds of computing environments.
https://www.debian.org/social_contract
If there is in fact an instrumentation, configuration, diagnostic, or other issue that's contributing to the problem here, then I feel strongly that it is the project team's responsibility to identify and correct that.
Again, holding this issue open during the window in which information which might prove useful could be contributed would be helpful in that regard.
@denschub Regarding Discourse, you wrote above:
You're several years late to participate in that discussion, but if you feel like it, check out the current implementation thread on discourse, the implementation work that's already done, the specifications work done, and participate there.
Where is that discussion, please? Because as someone not familiar with Discourse or Diaspora*'s use of it, finding that is nontrivial.
Thanks.
I feel strongly that it is the project team's responsibility to identify and correct that.
And how exactly do you think we could be able to achieve that without, you know, access to the installation that's causing problems?
Where is that discussion, please?
Literally the first result that comes up when you search for "migration". 214 messages, with links to all relevant other discussions/documents.
@denschub To be clear, this issue is not about recovering data from JoindiasporaCom specifically.
It is about identifying any possible issues which are contributing to failures of data export, and ensuring that those 1) don't occur in future and 2) if possible can be remedied in the case of JoindiasporaCom.
And for which specific experiences on that pod or others on which similar issues are being experienced. I'm aware of at least two pods facing pending shutdown, I don't know if others have similar issues, though that would not be surprising.
That last of course does rely on action and cooperation from JoindiasporaCom's admin. Which as has been noted by several participants in this discussion, seems less than likely.
But addressing bugs, instrumentation, configuration, and diagnosis do fall under the Diaspora* project's purview.
And how exactly do you think we could be able to achieve that without, you know, access to the installation that's causing problems?
Addressing bugs and requires us knowledge about a) what bugs exist, and b) what's causing this. Which is not the case here - and you appear to be unwilling to understand that.
I have locked this discussion because it's an endless spiral of unproductive arguments. See the last unhidden comment for an overview over why we can't do anything here.
If someone is able to provide actual reasons for failing exports (i.e. either steps-to-reproduce that can be replicated in a lab environment, or error messages with stack traces from production pods with failing exports), please either open a new issue about the specific error, or email team@diasporafoundation.org - thank you.