Non-destructive migration from Mantis to Redmine

Posted on 2010-04-16, 22:55, by uk, under Perl, Tools.

Once in a while i have to have new software. On a recent research about using VCS on NAS devices i came across Redmine (cause they talk about GIT and Mercurial a lot). We have used Mantis for at least two years and were never really satisfied with it – don’t get me wrong, it is a great piece of software, but kind of slow to handle. However, i installed Redmine for a recent project of us and it worked out really well.

Now i had to transfer our whole mantis database of the last years into Redmine. There is an official migration guide/script, but it does work destructive (kills all your Redmine data) and has also problems with the latest Mantis version (1.2.0) – or at least i assume it’s only with the latest. Being a Perl programmer has it advantages (besides being a Perl programmer), so i wrote a migration script myself, based on the rake-file from the Redmine source.

About

So here is what it does in short:

  • Migrate existing Mantis database non-destructive (keeps your Redmine data).
  • Able to map Mantis data to coresponding Redmine data (eg users, projects and so on).
  • Fully user interactive (suggest what to map to what, but gives you the capability to change.. assume you have a highly configured Mantis installation).

Things you should keep in mind before migration:

  • Custom fields of the type multi list are not supported in Redmine.
  • Custom fields of the type checkbox with multiple values will be translated to simple lists.
  • Migrates only attachment files which are stored in the database (default).
  • Tested only with Mantis 1.2.0 (current) and Redmine 0.9.3 (debian current).
  • Make a backup of your Redmine database BEFORE you run the script (eg mysqldump..)!

And another thing: this is a “work in progress”, no warranties, guaranties, whats-o-ever are given.

Install

Download (below) the script, put it in some directory (it will create multiple small YAML files and save all attachment files from the Mantis database to disk – size depends on your data).

Make it executable:

chmod +x mantis2redmine.pl

You need perl up and running. Also install prerequisite perl modules (YAML, DBIx::Simple). On debian that would be:

apt-get install libyaml-perl libdbix-simple-perl

Maybe create a configuration file. All command line parameters could be written in the file. Example:

mantis_db_host = localhost
mantis_db_name = mantis
mantis_db_login = mantis_user
mantis_db_pass = mantis_password

redmine_db_host = localhost
redmine_db_name = redmine
redmine_db_login = redmine_user
redmine_db_pass = redmine_password

You can run the script with ‘–help’ to see all possible parameters.

Usage

I suggest you run the script in dry-run-mode and setup the Mantis->Redmine mappings as you like (script will guide you through):

./mantis2redmine.pl -c config.file --dry_run

Even in dry-run-mode all your mappings will be stored in files in the current folder (store-<name>.map). If you want re-configure any mapping, simply remove the store-file and run the script again. Now make a backup of your Redmine database and run the script again:

./mantis2redmine.pl -c config.file --load_maps

This could take some time, depending on the size of your original Mantis database.

After the script finishes successfully check your Redmine installation. Probably you have to assign the imported (or existing) users to the projects (because Redmine supports groups the script doesn’t do that).

The last step is to copy all exported attachment files (default in ‘./attachments/’ folder) in your Redmine file directory (in a “normal” Redmine installation this would be ‘/files’ in your Redmine base dir; if you installed via debian apt-get it is in ‘/var/lib/redmine/default/files’).

Done.

Download

Here it is: latest (0.3.1)

Github: https://github.com/ukautz/mantis2redmine

mantis_user
Comment (RSS)  |  Trackback

28 Comments

  1. vladimir prieto says:
    2010-07-06 at 00:29

    on ubuntu 10.04 this script gave me errors:

    ~/rails$ ./mantis2redmine.pl --help
-bash: ./mantis2redmine.pl: /bin/env: intérprete erróneo: No existe el fichero ó directorio

    is on spanish and means: file or folder doesn’t exist.

    i did install libraries YAML, DBIx::Simple as you apt-get command said.

    what can i do to make it work?

    thanks!

  2. uk says:
    2010-07-06 at 00:38

    Seems i’ve got linked /usr/bin/env to /bin/env for some reason and assumed this the correct path. Is fixed now. Download again, or change first line in the file to

    #!/usr/bin/env perl
  3. vladimir prieto says:
    2010-07-06 at 14:26

    i’m beginning to walk, but a few steps after a fall down :)

    this is the command i used

    ./mantis2redmine.pl -c mantis2redmine.conf --attachment_dir=attachment --dry_run

    had to do it with attachment_dir option cuz it force me to (didn’t create it by default or automaticly).

    running it i began to see errors:

     *** Projects ***

Use of uninitialized value $first_id in hash element at ./mantis2redmine.pl line 366,  line 3.
Use of uninitialized value within @_ in list assignment at ./mantis2redmine.pl line 976,  line 3.
Mantis: Project
  micontrol - csystems :   1
  micontrol - aguasanandres :   2
  micontrol - rexhelmez :   3

Redmine: Project

Project translation
    Mantis               ->     Redmine
  1:micontrol - csystems ->  -1:new
  2:micontrol - aguasanandres ->  -1:new
  3:micontrol - rexhelmez ->  -1:new                 

Use of uninitialized value in concatenation (.) or string at ./mantis2redmine.pl line 1015,  line 3.
Use of uninitialized value in concatenation (.) or string at ./mantis2redmine.pl line 1015,  line 3.
Type 'ok' if you confirm or num:num  or num:-1 for create as new to change assignment
  eg 3: to change Project of 'micontrol - rexhelmez' to ''
  type 'print' to show the Redmine/Mantis tabels again
(ok/num:num/print) > ok

 *** Versions ***

Use of uninitialized value $first_id in hash element at ./mantis2redmine.pl line 404,  line 4.
Use of uninitialized value within @_ in list assignment at ./mantis2redmine.pl line 976,  line 4.
Mantis: Version

Redmine: Version

Version translation
    Mantis               ->     Redmine             

Use of uninitialized value $last_mantis in concatenation (.) or string at ./mantis2redmine.pl line 1015,  line 4.
Use of uninitialized value in concatenation (.) or string at ./mantis2redmine.pl line 1015,  line 4.
Use of uninitialized value $last_mantis in hash element at ./mantis2redmine.pl line 1015,  line 4.
Use of uninitialized value in concatenation (.) or string at ./mantis2redmine.pl line 1015,  line 4.
Use of uninitialized value in concatenation (.) or string at ./mantis2redmine.pl line 1015,  line 4.
Type 'ok' if you confirm or num:num  or num:-1 for create as new to change assignment
  eg : to change Version of '' to ''
  type 'print' to show the Redmine/Mantis tabels again
(ok/num:num/print) > ok
Use of uninitialized value in numeric eq (==) at ./mantis2redmine.pl line 943,  line 5.
Can't use an undefined value as a HASH reference at ./mantis2redmine.pl line 943,  line 5.

    after that it creates:

    $ ls -l
total 68
drwxr-xr-x 2 micontrol micontrol  4096 2010-07-06 09:12 attachment
-rw-r--r-- 1 micontrol micontrol   234 2010-07-05 19:21 mantis2redmine.conf
-rwxr-xr-x 1 micontrol micontrol 35656 2010-07-06 09:07 mantis2redmine.pl
-rw-r--r-- 1 micontrol micontrol    78 2010-07-06 09:14 store-custom_fields.map
-rw-r--r-- 1 micontrol micontrol   219 2010-07-06 09:14 store-priorities.map
-rw-r--r-- 1 micontrol micontrol   139 2010-07-06 09:16 store-projects.map
-rw-r--r-- 1 micontrol micontrol    65 2010-07-06 09:14 store-relations.map
-rw-r--r-- 1 micontrol micontrol   175 2010-07-06 09:14 store-roles.map
-rw-r--r-- 1 micontrol micontrol   203 2010-07-06 09:14 store-stati.map

    with attachment dir empty. obviously there are misssing maps…doesn’t it?

  4. vladimir prieto says:
    2010-07-07 at 15:19

    any ideas?

  5. uk says:
    2010-07-07 at 17:35

    Sorry, got swamped with daily business. Ok, i’ve looked in the code. As it seems, those errors indicate, that your redmine is kind of empty and the script does not handle this very well. Is your redmine db empty (important for further debugging)?

  6. vladimir prieto says:
    2010-07-07 at 18:33

    just the default_data from a fresh install

  7. uk says:
    2010-07-07 at 19:14

    Alright. I’ve writte it to migrate mantis databases into existing redmine databases with already existing content. I think the default importer from mantis works perfectly with fresh databases. However, i’ve made some small adjustments and it should work now also with empty redmine databases. Download above. Thanks for reporting.

  8. freightcar says:
    2010-09-10 at 05:44

    I am trying to migrate and getting the following error:

    DBD::mysql::st execute failed: Column ‘author_id’ cannot be null at ./mantis2redmine.pl line 766

    any words of wisdom?

    Thanks
    FC

  9. freightcar says:
    2010-09-10 at 07:13

    I think I fixed it.

    There were mantis issues assigned to non-existent users (ie deleted user account after the issue has been closed)

    I changed value in reporter_id field in mantis_bug_table and mantis_bugnote_table to “1″ (admin) for all nonexistent users.

    FC

  10. Sergey says:
    2010-12-20 at 15:04

    Type ‘ok’ if you confirm or num:num or num:-1 for create as new to change assignment
    eg 93:1 to change admin of ‘admin’ to ‘admin’
    type ‘print’ to show the Redmine/Mantis tabels again
    (ok/num:num/print) > ok

    Import Users
    .Can’t locate SQL/Abstract.pm in @INC (@INC contains: /etc/perl /usr/local/lib/perl/5.10.0 /usr/local/share/perl/5.10.0 /usr/lib/perl5 /usr/share/perl5 /usr/lib/perl/5.10 /usr/share/perl/5.10 /usr/local/lib/site_perl .) at /usr/share/perl5/DBIx/Simple.pm line 75, line 9.

    Where i can get Abstract.pm?

  11. uk says:
    2010-12-22 at 13:51

    You can get it either from cpan.org (cpan install ..) or from your distri (eg debian: aptitude install libsql-abstract-perl)

  12. Luis Davim says:
    2011-01-25 at 19:48

    Wat if I use Postgres as redmine DB? is there a easy way to make this script work with a postgres redmine db?

  13. uk says:
    2011-01-26 at 01:27

    You need to change the login credentials. They are in L168-177:

    my $dbix_mantis = DBIx::Simple->connect(
    'DBI:mysql:database='. $opt{ mantis_db_name }. ';host='. $opt{ mantis_db_host },
    $opt{ mantis_db_login }, $opt{ mantis_db_pass },
    { RaiseError => 1 }
);
my $dbix_redmine = DBIx::Simple->connect(
    'DBI:mysql:database='. $opt{ redmine_db_name }. ';host='. $opt{ redmine_db_host },
    $opt{ redmine_db_login }, $opt{ redmine_db_pass },
    { RaiseError => 1 }
);

    A simple change of the driver (‘DBI:mysql’ -> ‘DBI:pg’) might be sufficient. However, postgres authentication might require more settings (eg setting shema and so on)

  14. Philipp Schüle says:
    2011-02-04 at 14:37

    hi uk,

    first of all: noice script. the redmine migration script did not work at all for me and yours did much better … ;)

    i pimped your script a little bit to do the following:
    - use mantis categories as redmine issue_categories
    - allow to set redmine tracker depending on severity of mantis issues
    - set status and date for project versions
    - skip attachments if attachment files are in the mantis database

    would you put the script on github? then we could improve it together …

    thanks,
    philipp

  15. uk says:
    2011-02-04 at 16:18

    Hi Phillip

    Great idea to put it on github. Please, pimp it :)
    https://github.com/ukautz/mantis2redmine

    Greets Ulrich

  16. Philipp Schüle says:
    2011-02-04 at 18:01

    thanks :)

    i’ll merge my stuff in the next days … but now: weekend!

  17. Vincent says:
    2011-03-19 at 18:54

    Excellent work! This helped me a lot. I had to make some changes to the script and do a few additional steps, so I’ve uploaded my changes to GitHub and will share my experience here. My fork on GitHub is at https://github.com/vincentm8/mantis2redmine. This is my situation:

    - MantisBT version 1.1.6, attachments saved on disk in a location outside webroot
    - Redmine version 1.1.2, new install with default data
    - MySQL version 5.1
    - Perl version 5.12

    I created an admin user for myself in Redmine. Then I backed up the Mantis database and the newly-installed Redmine database so I could easly roll back and start over.

    I ran into trouble with the categories in Mantis, and got an error “Categories: DBD::mysql::st execute failed: Table ‘mantisbt.mantis_category_table’ doesn’t exist”. Mantis version 1.2 which the script was written for probably stores categories differently. Since I only needed to migrate once, I decided to modify the Mantis database to match the script. The steps I took are:

    - create categories table with the expected structure
    - copy categories into new table
    - add a category_id column to the bug table
    - update bugs to category_id based on the category name (truncated for this post)

    This allowed the script to run, and the categories seemed to come through OK into Redmine. However I eventually realized that categories didn’t get mapped to the right issues, and some issues ended up with no categories assigned. I only realized this after work had been done in Redmine so it was too late to go back.

    The other problems I ran into I fixed by modifying the script.

    - Removed FROM_UNIXTIME from all SQL queries.

    - Added default settings for newly-created Redmine projects to enable the Issue Tracking module and default trackers.

    - Fixed migration of attachments saved on disk. I added a configuration option attachment_disk to specify the path to the Mantis filestore (as seen by the script).

    - Redmine now requires the fields root_id, lft, rgt to be set for each item. If they’re not set you’ll get a 500 Server Error when trying to view the issue list in Redmine. The suggested fix came from http://www.redmine.org/boards/2/topics/21832?r=21847.

    Some final manual tasks after the migration were to copy across news items and project files. I didn’t try to script this because there weren’t that many, and not all of them needed to be migrated.

  18. Vincent says:
    2011-03-21 at 00:59

    Also, one possible “gotcha”: Redmine’s installation script doesn’t specify a character set for its database tables. Make sure your MySQL server’s default character set is set to utf8. If you’re not able to change the default, after installing Redmine you can alter each table to change the character set to utf8 (double-check each field too). You can save yourself trouble if you do this before you start populating tables with data.

  19. wmiadmin says:
    2011-04-04 at 13:32

    .DBD::mysql::st execute failed: Column ‘status_id’ cannot be null at /usr/local/share/perl/5.10.0/DBIx/Simple.pm line 172, line 8.
    at ./mantis2redmine.pl line 898

  20. wmiadmin says:
    2011-04-04 at 14:23

    Vincent, your script

    Import Issues (. = Issue, – = Journal, + = Attachment)
    .-DBD::mysql::st execute failed: Column ‘created_on’ cannot be null at /usr/local/share/perl/5.10.0/DBIx/Simple.pm line 172, line 8.
    at ./mantis2redmine.pl line 948

  21. Bastian says:
    2011-05-03 at 15:24

    Hi,

    thanks for the script.

  22. David says:
    2011-06-28 at 08:24

    Hi,

    It seems to fail loading DBIx, i’m using the bitnami stack for redmine, and i’ve already imported perl lib as described.

    Here the stack trace :
    Can’t locate DBIx/Simple.pm in @INC (@INC contains: /opt/bitnami/git/lib/site_perl/5.8.8/i686-linux /opt/bitnami/git/lib/site_perl/5.8.8 /opt/bitnami/perl/lib/site_perl/5.8.8/i686-linux /opt/bitnami/perl/lib/5.8.8/i686-linux /opt/bitnami/perl/lib/5.8.8 /opt/bitnami/perl/lib/5.8.8/i686-linux /bitnami/rubystack-linux/output/perl/lib/5.8.8/i686-linux /bitnami/rubystack-linux/output/perl/lib/5.8.8 /bitnami/rubystack-linux/output/perl/lib/site_perl/5.8.8/i686-linux /bitnami/rubystack-linux/output/perl/lib/site_perl/5.8.8 /bitnami/rubystack-linux/output/perl/lib/site_perl .) at ./mantis2redmine.pl line 86.
    BEGIN failed–compilation aborted at ./mantis2redmine.pl line 86.

    Any tips ?

    Thx
    David

  23. uk says:
    2011-06-28 at 10:56

    Hi David

    You need to install DBIx::Simple from CPAN. I am not sure what bitnami is, but assuming your include paths, have a look whether there is a /opt/bitnami/perl/bin/cpan executable. If so, run /opt/bitnami/perl/bin/cpan i DBIx::Simple.

    Greets, Ulrich

  24. ayong says:
    2011-07-19 at 08:39

    How can I do if I want use utf8 charset?

    I found default is latin1 not utf8.

  25. Bobby says:
    2011-08-18 at 11:02

    .DBD::mysql::st execute failed: Column ‘status_id’ cannot be null at /usr/local/share/perl/5.10.0/DBIx/Simple.pm line 172, line 8.
    at ./mantis2redmine.pl line 898

    I have the same problems “wmiadmin”, have anybody a idea?

    Greets Bobby

  26. ivana says:
    2011-09-23 at 13:37

    hi, im trying to migrate my mantis db to redmine, but im getting this message any tip? i`ve just started using redmine..

    /home/luz/Descargas/redmine/migrcion mantis/mantis2redmine# ./mantis2redmine.pl
    Can’t locate DBIx/Simple.pm in @INC (@INC contains: /etc/perl /usr/local/lib/perl/5.10.1 /usr/local/share/perl/5.10.1 /usr/lib/perl5 /usr/share/perl5 /usr/lib/perl/5.10 /usr/share/perl/5.10 /usr/local/lib/site_perl .) at ./mantis2redmine.pl line 86.
    BEGIN failed–compilation aborted at ./mantis2redmine.pl line 86.

  27. Peter says:
    2011-11-02 at 11:51

    Hi,

    This is a great script :-)

    I’m testing it on a huge Mantis database and found:

    - Database value lft,rgt can’t be NULL so
    UPDATE issues SET lft = 0 WHERE lft IS NULL
    UPDATE issues SET rgt = 0 WHERE rgt IS NULL
    - Databse value root_id must point to someone or it self (not NULL)
    UPDATE issues SET root_id = id WHERE root_id = NULL

    You can use it with Bitnami if the standard linux console is used and
    the host is set for 127.0.0.1 (not localhost)

    Regards Peter

  28. foo says:
    2011-11-20 at 19:12

    @wmiadmin

    I was having the same problem. It looks like some of @Vincent’s changes were Mantis 1.1.x-specific. I reverted his DATE_FORMAT changes, since Mantis 1.2.x seems to use UNIX timestamps, and the “‘created_on’ cannot be null” error went away. Yay!

    https://github.com/tsnfoo/mantis2redmine

Leave a Reply

CAPTCHA image

«
»