After spending two hours to understand the tables and relations, wrote a script to migrate the categories and boards. The current output is like this:
The script also maintains the order of the categories and the boards inside them.
[quote]I got the dump imported into MySQL.
[/quote]
That was my thinking when I started inspecting the data, to load as many tables as possible into MySQL and go from there. The tools I have for inspection are not so hot: e.g., Notepad/Wordpad! Advice on a better way to inspect the dump would be appreciated.
[quote]After spending two hours to understand the tables and relations, wrote a script to migrate the categories and boards. The current output is like this:
[/quote]
Is this the view from within Flarum? (Is there a consensus that Flarum is a good target? Or a good intermediate step to something else?)
Looks good! I can see that the GUI should work well on small display screens.
The only thing I’m wondering about is the sub-categories at the bottom and how they relate to larger blocks.
I did peruse the dump file a bit, but was slow on the trigger to download MySQL, and haven’t downloaded Flarum yet either. I’ll do so, if we are going in this direction.
For this large data, phpMyAdmin and other web based tools will trigger a post error, you need to work from the command line. After you create the database, do this from cmd:
$ mysql -uuname -p
Enter password: *****
mysql> use dbname
mysql> source path/to/dumpFile.sql
That will do the trick of importing. It took me 30 seconds for that to complete.
This is really awesome. Congratulations. What do you see as the next steps?
Hey @Riven, I started migrating the users and found that there are a ton of users who have activated the accounts but haven’t logged in even once. The lastLogin field for some is set to 0. Should I migrate them too? Or can I optimize the script to remove them?
First step is to port the users. Once that is done, I could look into porting the meta data, like the profile picture etc., Right now however profile pictures cannot be ported because it appears to me that Flarum caches them in other locations, and I cannot access the server of JGO to download the images.
After this is done, I can continue to port the posts. The good thing about Flarum is that it supports posts in both BBCode and also Markdown. However, I think I have to modify them a bit.
Porting of users is done, but however they are created with no password, and hence they cannot log in. They are required to click on forget password link and create new password for the accounts. Members visible in the admin panel:
And here is a ported user:
Profile pictures are a bit off, but I don’t think I can fix that. Still had to search where Flarum stores them. And of course, mods and groups aren’t ported yet.
Great progress dudes!
Once you get 10% of data pumped over, it’s typically a breeze to do the remaining 90%, especially on such a simple data-model.
So by all means, feel free to reduce your dataset, if it makes porting easier/speedier. In the final stages of the migration you would take the whole dataset.
Ofcourse this is one of the first snags of a data-migration: how do we get application X to understand the password-encryption (salted-hash) from application Y. Typically this involves a bit of custom code in the new application. I could PM you your own SMF credentials (as hash+salt) and you could test your new implementation with that.
Having to reset your password seems like a pretty reasonable thing when switching to a new site.
A quick change is that I added error reporting and also a progress notifier in the tool. This migration script is CLI based, as you can see it:
This should be more better as I will get to know if the script was hanged or not. I had to admit it is stressing out the MySQL server, as I can see the disk access at 99% rate. I have also added recognition of user groups, and currently this script translates everyone as Members, Mods and Admins.
By the way, did I mention that I’m committing this to GitHub? It’s at https://github.com/sriharshachilakapati/JGO-Flarum-Migration
Thanks! That will keep me excited to do, it was boring a bit to stare at the screen doing nothing while the migration is done. Currently for 11513 users, it is taking me ~12 minutes to do the port, that is with the reduced data which excludes people who have never logged in after activating their accounts and also non-activated accounts.
Yep I agree. I had looked into the sources of Flarum and found that it uses the same hashing used by Laravel framework, which I guess is bcrypt. I’d like to experiment with that, please do send me my own credentials in the PM.
True that, it would be fine with most of the users as they log-in with social media anyways, but wouldn’t hurt more for experimenting right?
Keep it simple… just require everyone to change their password on first login, move on.
Cas 
It could stress out JGO servers as requiring to change the password means that the server has to send a lot of mails with links.
You underestimate just how, er, popular JGO isn’t.
Cas
this is also always a good practise from time to time, tough
Typically data-migration is fastest if you let the database do the bulk of the work:
SELECT ... some data-transformations in the select-statement
INTO dst_table
FROM src_table
JOIN ... potentially some more JOINs to be able to properly fill dst_table
Migrating 11K users should take a split second, with this approach.
How are things going SHC? Is there somewhere you are hosting it that we can have a play with?
I’m having a little bit of trouble with posts breaking because of tables. Flarum’s BBCode plugin has limited tags, so I’m now in the process of writing an extension for it. Also, the icode tag breaks as well.
However the markdown support is good, so in the end, I might be just using the text formatter library and translate from BBCode to markdown. Just now I’m reading up the internal representation of posts. It appears that SMF stores the posts as raw BBCode. Flarum however stores it in a internal XML format that is easier to render.
By the way, I’m not going to host this anywhere without the permission of @Riven. In the meanwhile you can set up a local Flarum instance and run the migrator yourself.
You have that permission, SHC. The (textual) content in the dump is slightly mangled, anybody using this as the foundation of a fork would have to be nuts
How are you liking the data-migration so far?
How easy is it to extend the Flarum UI? For example, adding a sidebar like on JGO.
Do you consider Flarum as a community-centric platform, or more of a Q&A platform?
When looking at the Flarum UI it feels to me like the focus is on Q&A. Maybe it’s me
Thanks for that Riven, but I think that I’m going to do that only once I believe it got to a point to show others. I still have to write code to migrate posts.
However I’m taking some time, as I now have to move to a city that is ~700 km away because of job. I might not be available for two weeks because of house hunting, packing and moving things etc.
I’ll be getting back to this from 11 or 12 December. The progress is smooth enough.
Regarding the UI of Flarum, I beleive that it is community centric, as is Discourse. To me they both appear to be same. However, I don’t think you can get a sidebar into it. You can, by fiddling with adding a component to Mithril, but I don’t think you can integrate it without spoiling the look. That’s my opinion though.
See ya soon once I settle down there.