3394: Problem with upgrade from 1.7.22 to 2.0.1. Django evolution fails

rafal.*******@gmai***** (Google Code) (Is this you? Claim this profile.)
June 2, 2014
What version are you running?
1.7.22

What's the URL of the page containing the problem?
N/A

What steps will reproduce the problem?
1. Install RB 2.0.1
2. Run rb-site upgrade /path/to/my/board
3. Run once again rb-site upgrade /path/to/my/board

What is the expected output? What do you see instead?

After the first run of the upgrade command, rb-site upgrade /path/ I get:
Rebuilding directory structure
Updating database. This may take a while.

The log output below, including warnings and errors,
can be ignored unless upgrade fails.

------------------ <begin log output> ------------------
Creating tables ...
Upgrading Review Board from 1.7.22 to 2.0.1
There are unapplied evolutions for auth.
There are unapplied evolutions for contenttypes.
There are unapplied evolutions for accounts.
There are unapplied evolutions for attachments.
There are unapplied evolutions for changedescs.
There are unapplied evolutions for diffviewer.
There are unapplied evolutions for hostingsvcs.
There are unapplied evolutions for reviews.
There are unapplied evolutions for scmtools.
Adding baseline version for new models
Project signature has changed - an evolution is required
Installing custom SQL ...
Installing indexes ...
Installed 0 object(s) from 0 fixture(s)
Evolution successful.
------------------- <end log output> -------------------

Resetting in-database caches.

Upgrade complete!


If the command is run the second time, it ends with error message:

Rebuilding directory structure
Updating database. This may take a while.

The log output below, including warnings and errors,
can be ignored unless upgrade fails.

------------------ <begin log output> ------------------
Creating tables ...
Project signature has changed - an evolution is required
Installing custom SQL ...
Installing indexes ...
Installed 0 object(s) from 0 fixture(s)
The stored evolutions do not completely resolve all model changes.

Run `./manage.py evolve --hint` to see a suggestion for the changes required.

The following are the changes that could not be resolved:
The application markup has been deleted
In model changedescs.ChangeDescription:
    Field 'rich_text' has been added
In model attachments.FileAttachment:
    Field 'added_in_filediff' has been added
    Field 'repository' has been added
    Field 'repo_revision' has been added
    Field 'repo_path' has been added
In model auth.Permission:
    Meta property 'unique_together' has changed
In model reviews.Comment:
    Field 'rich_text' has been added
    Field 'extra_data' has been added
In model reviews.Group:
    Field 'extra_data' has been added
    In field 'mailing_list':
        Property 'max_length' has changed
    Meta property 'unique_together' has changed
In model reviews.FileAttachmentComment:
    Field 'diff_against_file_attachment' has been added
    Field 'rich_text' has been added
In model reviews.Review:
    Field 'rich_text' has been added
    Field 'extra_data' has been added
In model reviews.ReviewRequestDraft:
    Field 'commit_id' has been added
    Field 'extra_data' has been added
    Field 'rich_text' has been added
In model reviews.ReviewRequest:
    Field 'issue_dropped_count' has been added
    Field 'extra_data' has been added
    Field 'commit_id' has been added
    Field 'issue_resolved_count' has been added
    Field 'issue_open_count' has been added
    Field 'rich_text' has been added
    Meta property 'unique_together' has changed
In model reviews.ScreenshotComment:
    Field 'rich_text' has been added
    Field 'extra_data' has been added
In model contenttypes.ContentType:
    Meta property 'unique_together' has changed
In model accounts.Profile:
    Field 'show_closed' has been added
    Field 'show_submitted' has been deleted
In model accounts.ReviewRequestVisit:
    Meta property 'unique_together' has changed
In model accounts.LocalSiteProfile:
    Meta property 'unique_together' has changed
In model scmtools.Repository:
    Meta property 'unique_together' has changed
In model diffviewer.DiffSet:
    Field 'extra_data' has been added
In model diffviewer.FileDiffData:
    Field 'extra_data' has been added
In model diffviewer.DiffSetHistory:
    Field 'extra_data' has been added
In model diffviewer.FileDiff:
    Field 'extra_data' has been added
CommandError: Your models contain changes that Django Evolution cannot resolve automatically.


What operating system are you using? What browser?
Ubuntu 12.04

Please provide any additional information below.
Installed version of django evolution is 0.7.1. RB 2.0.1 is installed with easy_install -U ReviewBoard. I tried with 2.0.0 installed from egg and the result is the same.

rb-site manage /var/www/sackboard/ evolve -- --hint
#----- Evolution for auth
from django_evolution.mutations import ChangeMeta


MUTATIONS = [
    ChangeMeta('Permission', 'unique_together', ((u'content_type', u'codename'),))
]
#----------------------
#----- Evolution for contenttypes
from django_evolution.mutations import ChangeMeta


MUTATIONS = [
    ChangeMeta('ContentType', 'unique_together', (('app_label', 'model'),))
]
#----------------------
#----- Evolution for accounts
from django_evolution.mutations import AddField, ChangeMeta, DeleteField
from django.db import models


MUTATIONS = [
    AddField('Profile', 'show_closed', models.BooleanField, initial=True),
    DeleteField('Profile', 'show_submitted'),
    ChangeMeta('ReviewRequestVisit', 'unique_together', ((u'user', u'review_request'),)),
    ChangeMeta('LocalSiteProfile', 'unique_together', ((u'user', u'local_site'), (u'profile', u'local_site')))
]
#----------------------
#----- Evolution for attachments
from django_evolution.mutations import AddField
from django.db import models


MUTATIONS = [
    AddField('FileAttachment', 'added_in_filediff', models.ForeignKey, null=True, related_model='diffviewer.FileDiff'),
    AddField('FileAttachment', 'repository', models.ForeignKey, null=True, related_model='scmtools.Repository'),
    AddField('FileAttachment', 'repo_revision', models.CharField, max_length=64, null=True, db_index=True),
    AddField('FileAttachment', 'repo_path', models.CharField, max_length=1024, null=True)
]
#----------------------
#----- Evolution for changedescs
from django_evolution.mutations import AddField
from django.db import models


MUTATIONS = [
    AddField('ChangeDescription', 'rich_text', models.BooleanField, initial=True)
]
#----------------------
#----- Evolution for diffviewer
from django_evolution.mutations import AddField
from djblets.db.fields import JSONField


MUTATIONS = [
    AddField('DiffSet', 'extra_data', JSONField, null=True),
    AddField('FileDiffData', 'extra_data', JSONField, null=True),
    AddField('DiffSetHistory', 'extra_data', JSONField, null=True),
    AddField('FileDiff', 'extra_data', JSONField, null=True)
]
#----------------------
#----- Evolution for reviews
from django_evolution.mutations import AddField, ChangeField, ChangeMeta
from django.db import models
from djblets.db.fields import CounterField
from djblets.db.fields import JSONField


MUTATIONS = [
    AddField('Comment', 'rich_text', models.BooleanField, initial=False),
    AddField('Comment', 'extra_data', JSONField, null=True),
    AddField('Group', 'extra_data', JSONField, null=True),
    ChangeField('Group', 'mailing_list', initial=None, max_length=254),
    ChangeMeta('Group', 'unique_together', ((u'name', u'local_site'),)),
    AddField('FileAttachmentComment', 'diff_against_file_attachment', models.ForeignKey, null=True, related_model='attachments.FileAttachment'),
    AddField('FileAttachmentComment', 'rich_text', models.BooleanField, initial=False),
    AddField('Review', 'rich_text', models.BooleanField, initial=False),
    AddField('Review', 'extra_data', JSONField, null=True),
    AddField('ReviewRequestDraft', 'commit_id', models.CharField, max_length=64, null=True, db_index=True),
    AddField('ReviewRequestDraft', 'extra_data', JSONField, null=True),
    AddField('ReviewRequestDraft', 'rich_text', models.BooleanField, initial=False),
    AddField('ReviewRequest', 'issue_dropped_count', CounterField, null=True),
    AddField('ReviewRequest', 'extra_data', JSONField, null=True),
    AddField('ReviewRequest', 'commit_id', models.CharField, max_length=64, null=True, db_index=True),
    AddField('ReviewRequest', 'issue_resolved_count', CounterField, null=True),
    AddField('ReviewRequest', 'issue_open_count', CounterField, null=True),
    AddField('ReviewRequest', 'rich_text', models.BooleanField, initial=False),
    ChangeMeta('ReviewRequest', 'unique_together', ((u'commit_id', u'repository'), (u'changenum', u'repository'), (u'local_site', u'local_id'))),
    AddField('ScreenshotComment', 'rich_text', models.BooleanField, initial=False),
    AddField('ScreenshotComment', 'extra_data', JSONField, null=True)
]
#----------------------
#----- Evolution for scmtools
from django_evolution.mutations import ChangeMeta


MUTATIONS = [
    ChangeMeta('Repository', 'unique_together', ((u'name', u'local_site'), (u'path', u'local_site')))
]
#----------------------
Trial evolution successful.




mysqldump -u root -p<removed> --databases <removed>--tables django_evolution
-- MySQL dump 10.13  Distrib 5.5.34, for debian-linux-gnu (x86_64)
--
-- Host: <removed> Database: <removed>
-- ------------------------------------------------------
-- Server version	5.5.34-0ubuntu0.12.04.1

/*!40101 SET @OLD_CHARACTER_SET_CLIENT=@@CHARACTER_SET_CLIENT */;
/*!40101 SET @OLD_CHARACTER_SET_RESULTS=@@CHARACTER_SET_RESULTS */;
/*!40101 SET @OLD_COLLATION_CONNECTION=@@COLLATION_CONNECTION */;
/*!40101 SET NAMES utf8 */;
/*!40103 SET @OLD_TIME_ZONE=@@TIME_ZONE */;
/*!40103 SET TIME_ZONE='+00:00' */;
/*!40014 SET @OLD_UNIQUE_CHECKS=@@UNIQUE_CHECKS, UNIQUE_CHECKS=0 */;
/*!40014 SET @OLD_FOREIGN_KEY_CHECKS=@@FOREIGN_KEY_CHECKS, FOREIGN_KEY_CHECKS=0 */;
/*!40101 SET @OLD_SQL_MODE=@@SQL_MODE, SQL_MODE='NO_AUTO_VALUE_ON_ZERO' */;
/*!40111 SET @OLD_SQL_NOTES=@@SQL_NOTES, SQL_NOTES=0 */;

--
-- Table structure for table `django_evolution`
--

DROP TABLE IF EXISTS `django_evolution`;
/*!40101 SET @saved_cs_client     = @@character_set_client */;
/*!40101 SET character_set_client = utf8 */;
CREATE TABLE `django_evolution` (
  `id` int(11) NOT NULL AUTO_INCREMENT,
  `version_id` int(11) NOT NULL,
  `app_label` varchar(200) NOT NULL,
  `label` varchar(100) NOT NULL,
  PRIMARY KEY (`id`),
  KEY `django_evolution_fef0b09d` (`version_id`),
  CONSTRAINT `version_id_refs_id_62f3fe80` FOREIGN KEY (`version_id`) REFERENCES `django_project_version` (`id`)
) ENGINE=InnoDB AUTO_INCREMENT=68 DEFAULT CHARSET=latin1;
/*!40101 SET character_set_client = @saved_cs_client */;

--
-- Dumping data for table `django_evolution`
--

LOCK TABLES `django_evolution` WRITE;
/*!40000 ALTER TABLE `django_evolution` DISABLE KEYS */;
INSERT INTO `django_evolution` VALUES (1,1,'auth','auth_delete_message'),(2,1,'sessions','session_expire_date_db_index'),(3,1,'accounts','is_private'),(4,1,'accounts','timezone'),(5,1,'accounts','open_an_issue'),(6,1,'changedescs','fields_changed_longtext'),(7,1,'diffviewer','add_parent_diffs'),(8,1,'diffviewer','filediff_filenames_1024_chars'),(9,1,'diffviewer','diffset_basedir'),(10,1,'diffviewer','filediff_status'),(11,1,'diffviewer','add_diff_hash'),(12,1,'diffviewer','diffsethistory_diff_updated'),(13,1,'reviews','change_descriptions'),(14,1,'reviews','last_review_timestamp'),(15,1,'reviews','shipit_count'),(16,1,'reviews','default_reviewer_repositories'),(17,1,'reviews','null_repository'),(18,1,'reviews','localsite'),(19,1,'reviews','group_incoming_request_count'),(20,1,'reviews','group_invite_only'),(21,1,'reviews','group_visible'),(22,1,'reviews','default_reviewer_local_site'),(23,1,'reviews','add_issues_to_comments'),(24,1,'reviews','file_attachments'),(25,1,'reviews','file_attachment_comment_extra_data'),(26,1,'reviews','review_request_last_review_activity_timestamp'),(27,1,'scmtools','bugzilla_url_charfield'),(28,1,'scmtools','repository_raw_file_url'),(29,1,'scmtools','repository_visible'),(30,1,'scmtools','repository_path_length_255'),(31,1,'scmtools','localsite'),(32,1,'scmtools','repository_access_control'),(33,1,'scmtools','group_site'),(34,1,'scmtools','repository_hosting_accounts'),(35,1,'scmtools','repository_extra_data_null'),(36,2,'accounts','extra_data'),(37,2,'accounts','timezone_length_30'),(38,2,'attachments','file_attachment_orig_filename'),(39,2,'hostingsvcs','account_hosting_url'),(40,2,'hostingsvcs','account_hosting_url_max_length_255'),(41,2,'reviews','review_request_depends_on'),(42,3,'attachments','file_attachment_file_max_length_512'),(43,3,'diffviewer','diffset_base_commit_id'),(44,4,'accounts','localsiteprofile_permissions'),(45,5,'site','localsite_public'),(46,7,'auth','auth_unique_together_baseline'),(47,7,'contenttypes','contenttypes_unique_together_baseline'),(48,7,'accounts','unique_together_baseline'),(49,7,'accounts','profile_show_closed'),(50,7,'attachments','file_attachment_repo_info'),(51,7,'attachments','file_attachment_repo_path_no_index'),(52,7,'attachments','file_attachment_repo_revision_max_length_64'),(53,7,'changedescs','rich_text'),(54,7,'diffviewer','filediffdata_line_counts'),(55,7,'diffviewer','filediffdata_extra_data'),(56,7,'diffviewer','all_extra_data'),(57,7,'hostingsvcs','account_unique_together_baseline'),(58,7,'reviews','commit_id'),(59,7,'reviews','file_attachment_comment_diff_id'),(60,7,'reviews','rich_text'),(61,7,'reviews','base_comment_extra_data'),(62,7,'reviews','unique_together_baseline'),(63,7,'reviews','extra_data'),(64,7,'reviews','review_request_issue_counts'),(65,7,'reviews','group_mailing_list_charfield'),(66,7,'reviews','review_request_draft_commit_id'),(67,7,'scmtools','unique_together_baseline');
/*!40000 ALTER TABLE `django_evolution` ENABLE KEYS */;
UNLOCK TABLES;
/*!40103 SET TIME_ZONE=@OLD_TIME_ZONE */;

/*!40101 SET SQL_MODE=@OLD_SQL_MODE */;
/*!40014 SET FOREIGN_KEY_CHECKS=@OLD_FOREIGN_KEY_CHECKS */;
/*!40014 SET UNIQUE_CHECKS=@OLD_UNIQUE_CHECKS */;
/*!40101 SET CHARACTER_SET_CLIENT=@OLD_CHARACTER_SET_CLIENT */;
/*!40101 SET CHARACTER_SET_RESULTS=@OLD_CHARACTER_SET_RESULTS */;
/*!40101 SET COLLATION_CONNECTION=@OLD_COLLATION_CONNECTION */;
/*!40111 SET SQL_NOTES=@OLD_SQL_NOTES */;

-- Dump completed on 2014-06-02  8:57:22
chipx86
#1 chipx86
This is almost always due to upgrading, then downgrading, then upgrading again. Or, a previously failed upgrade attempt due to some other error (often a MyISAM vs. InnoDB table difference).

Can you show me the output of:

    rb-site manage /path/to/site list-evolutions

Also, was this the very first upgrade attempt?
  • +NeedInfo
#2 rafal.*******@gmai***** (Google Code) (Is this you? Claim this profile.)
It is the very first attempt to upgrade this 1.7.22 version to 2.0.1. I am trying to test new 2.0.1 in testing environment separated from my 1.7.22 production instance. I can migrate the database from production with mysqldump or restore the whole environment from backup but the error stops me every time and I do not know what could be wrong. This production instance was upgraded many times but never downgraded. 


Applied evolutions for 'auth':
    auth_delete_message
    auth_unique_together_baseline

Applied evolutions for 'contenttypes':
    contenttypes_unique_together_baseline

Applied evolutions for 'sessions':
    session_expire_date_db_index

Applied evolutions for 'accounts':
    is_private
    timezone
    open_an_issue
    extra_data
    timezone_length_30
    localsiteprofile_permissions
    unique_together_baseline
    profile_show_closed

Applied evolutions for 'attachments':
    file_attachment_orig_filename
    file_attachment_file_max_length_512
    file_attachment_repo_info
    file_attachment_repo_path_no_index
    file_attachment_repo_revision_max_length_64

Applied evolutions for 'changedescs':
    fields_changed_longtext
    rich_text

Applied evolutions for 'diffviewer':
    add_parent_diffs
    filediff_filenames_1024_chars
    diffset_basedir
    filediff_status
    add_diff_hash
    diffsethistory_diff_updated
    diffset_base_commit_id
    filediffdata_line_counts
    filediffdata_extra_data
    all_extra_data

Applied evolutions for 'hostingsvcs':
    account_hosting_url
    account_hosting_url_max_length_255
    account_unique_together_baseline

Applied evolutions for 'reviews':
    change_descriptions
    last_review_timestamp
    shipit_count
    default_reviewer_repositories
    null_repository
    localsite
    group_incoming_request_count
    group_invite_only
    group_visible
    default_reviewer_local_site
    add_issues_to_comments
    file_attachments
    file_attachment_comment_extra_data
    review_request_last_review_activity_timestamp
    review_request_depends_on
    commit_id
    file_attachment_comment_diff_id
    rich_text
    base_comment_extra_data
    unique_together_baseline
    extra_data
    review_request_issue_counts
    group_mailing_list_charfield
    review_request_draft_commit_id

Applied evolutions for 'scmtools':
    bugzilla_url_charfield
    repository_raw_file_url
    repository_visible
    repository_path_length_255
    localsite
    repository_access_control
    group_site
    repository_hosting_accounts
    repository_extra_data_null
    unique_together_baseline

Applied evolutions for 'site':
    localsite_public

#3 rafal.*******@gmai***** (Google Code) (Is this you? Claim this profile.)
After few more tries I managed to finish rb-site upgrade without any errors :) Unfortunately I cannot tell what I did wrong and what solved the problem, sorry. 

At the moment I have another problem with formatting in review request window (please see attached image). Text areas seem to be badly formatted and too wide. It occurs only for "migrated" review requests. Newly created review requests look well, as it should.

I confirmed the formatting problem in chromium-browser, firefox and IE8.

Could it be upgrading/evolution related issue?
chipx86
#4 chipx86
Can you please file a separate bug for the text areas? We keep bug reports to one incident per report.
  • -NeedInfo
    +SetupIssue