Migrate Accounts To Another Instance & Import/Export

[seniorm0ment]

Migrate Account To Another Instance
Suggesting the ability to migrate a BookWyrm account, and all information associated with it (lists, reviews, etc.) to a new instance.

Describe alternatives you've considered
Cloning/syncing instead of simply migrating. This would allow a user to have a backup account on another instance in the case one goes down. The user could input the second instance information on the other account and have it sync once a day automatically.
The user could always delete the account on the main instance if they want to migrate, so this is migration but with optional sync functionality.

Export/Import
Suggesting the ability to export/import a profile (via file) to store locally, as well as shelves/lists/reading progress.
This could potentially also be offered in a .json format or similar if users would like to utilize it in other programs.
But it can also be used to import their content into other instances, or have a local copy.

[arkhi]

(related to #75)

[formerlytomato]

Copying my text from #1470 for future reference:

I honestly would go as far as to say this is IMO the biggest priority for any federated platform. People avoid smaller instances because once they're on there, they're stuck there. Their followers, posts, and profile are tied to that instance and have to deal with the risk of an instance shutting down or not getting interaction. People concentrate on flagship and larger instances, furthering the centralization problem and defeating the whole strength of the fediverse.
With Bookwyrm, you spend a lot more time setting up your profile which will make starting over much more disheartening but also a lot of this content is stuff that seems like there'd be less complications with forwarding between instances.

• Quotes
• Reviews
• Book lists

I might be missing something, but all of this stuff is stuff that's tied to the profile alone and not really time or interaction-dependent like a Mastodon post or DM. And its all stuff which people spend hours setting up when considered over the long-term. Being able to migrate this stuff from instance to instance would be a very big deal and make it a lot more appealing to try out (and by extend create) other instances.
Also, I'm not sure how useful referencing Mastodon's migration system would be here but it currently is somehow also capable of handling the redirecting of followers to the new account.

[loppear]

I'm looking at running a personal/small-group instance, and moving my primary account there. Same issue for encouraging friends to join here or there, confidence . Concurring with this issue and @TomatDividedBy0 above, unlike migrating in Mastodon I would expect to move some set of my associated data, although would be comfortable leaving some activity-oriented data behind.

I'm anticipating being either ok leaving the prior account around to preserve comments, notify followers OR fully delete/leave. I suppose that boils down to "Just Export/Import Books (and associated)" or "Move Accounts", and could live with a reason that only the latter was a good idea.

It is possible this issue should split between

1. Features to allow round-trip Import/Export of specific various data (Book Metadata, Book shelving and reading activity, Reviews, Quotes, Lists, Following).
2. Features to ease migration (helpful UI flow/combining of import/export process, de-duplication or data deletion, account aliasing/redirection)

My additional thoughts:

• Shelved Books.
  • probably implies Quotes, Reviews at a minimum
  • and ideally Reading Activity (though multi-valued - side question of whether to focus on CSV import/export or more structured).
  • Can this be more complete than GR imports lookups, a way to seed an instance with the additional metadata/cover-imports already in BW data? Perhaps Add ability to create books from import failures #1820 will make this partially moot.
  • I presume there's a real issue here with federation, in terms of duplicated reviews and quotes if/whether the original account is not deleted.
    • If removing the prior account's data is the only low-impact answer, I could live with that for me but maybe not as an admin.
    • Not having looked at the specifics of review federation, it's also possible that migration imports could skip federating - duplication is still an issue for migrating to a server that has already received your old instance reviews then.
• Book Lists
  • I haven't looked at if/how these federate today though and is probably a separate or less central point for me.
  • but if I used them for private/unlisted organizing shelves, I might expect the ability to duplicate them with this feature.
• Following/Followers
  • I do not know if the mastodon auto-redirect-follow feature (account aliasing & redirection?) is a reusable part of activitypub, if so I'd want to eventually support it, but I'm guessing the effort is high.
  • At a minimum I would want my following list to export/import.
• Comments and other interaction activities
  • I would expect these not to migrate.

[skobkin]

I think it's important to point out that such migration should allow to export/import also all books which were only available on the old instance to the new instance.

For example I've added some books which weren't available even in public catalogues to the bookwyrm.social and while moving to my own instance, I'd prefer to move (copy) them with me so I still have them on my shelves.
It may require more complicated export format support (like ZIP archive with JSON data and images), but it's very important for users who actively contribute to the book list.

[mirenbz]

Any progress on this? Currently (for what I've seen) the CSV export does not even carry read activity or shelf classifications, so even "manual" moving becomes tedious...

[hughrun]

I have pinned this issue because it is increasingly important that we make progress on this, and also so that people can see that it has been 'reported'. For a healthy Bookwyrm ecosystem we need user account portability and many users assume we have it.

I think we can probably work on this incrementally, so nobody needs to feel that they can't make a contribution without having to include every single possible use case up front.

Related:

• Import should give users control over how books are shelved #449
• CSV Export does not include imported reviews #2156
• Generic CSV import #2411
• Create an option to import BookWyrm CSVs into a new BookWyrm account #2666
• Allow Imports To Specific Shelf #2669
• Exported CSV file does not seem to contain information on what shelf my books are on #2846
• Export user book data as csv #1556

[hughrun]

I am expecting to take an initial look at this over the coming weekend.

I think @skobkin is correct that a csv is not going to cut it for this, we'll need to look at something like JSON or perhaps a better choice would be TOML, since we're probably looking at up/downloading files rather than needing to transfer data over http. Maybe .zip or .gz or similar if we want to package up book covers etc.

Part of the delay on this has, I think, come from anticipation of #2170 but that's going to be a complex and longer term project.

If anyone else is currently working on user migration please let me know so I don't double up on work.

[skobkin]

@hughrun Sounds awesome!

That's what was stopping me from researching about setting up my own instance, I'd still have a lot of trouble moving my content anyway.

If it'd be implemented, it'd make much easier for people to distribute across instances more evenly. It should be useful for all of us considering how slow main instance could be sometimes. And Fediverse is also about spreading the load, so it's natural to have this feature available.

Not to mention how useful it is in cases when instance is about to close.

P.S. Just an idea from the top of my head. If it wouldn't make you work much harder, maybe it would be a good idea to consider the potential re-usage of such feature in move activity implementation. AFAIK, Bookwyrm doesn't currently support account migration like Mastodon or Pleroma.
But if it'll make it harder then pay it no mind, because we still could migrate the data manually if it's implemented. Walking in small steps could be better in that case.

[formerlytomato]

I am expecting to take an initial look at this over the coming weekend.

I think @skobkin is correct that a csv is not going to cut it for this, we'll need to look at something like JSON or perhaps a better choice would be TOML, since we're probably looking at up/downloading files rather than needing to transfer data over http. Maybe .zip or .gz or similar if we want to package up book covers etc.

Part of the delay on this has, I think, come from anticipation of #2170 but that's going to be a complex and longer term project.

If anyone else is currently working on user migration please let me know so I don't double up on work.

Social Coding is currently doing a test-run of an organizational model for free-software development known as guilds.

The TLDR is that we currently have an informal task force which just voted on dedicating the next two months to tackling this issue. The plan we're going with has been outlined here.

Just wanted to post this to notify those involved that there is more hands that can be put towards this and if anyone has any input or advice, to feel free to communicate with us.