3486: Upgrade 2.0 to 2.0.3 fails

drz****@gmai***** (Google Code) (Is this you? Claim this profile.)
July 28, 2014 
What version are you running?
2.0

What's the URL of the page containing the problem?
All, unable to access after upgrade. See log below.

What steps will reproduce the problem?
1. easy_install --index-url=https://pypi.mycopany.com -U ReviewBoard
2.  rb-site upgrade /srv/reviewboard-env/reviewboard.mycompany/
3.

What is the expected output? What do you see instead?
Expect: no error.

Actual: output of running rb-site upgrade:
Rebuilding directory structure
Updating database. This may take a while.

The log output below, including warnings and errors,
can be ignored unless upgrade fails.

------------------ <begin log output> ------------------
Creating tables ...
Upgrading Review Board from 2.0 to 2.0.3
There are unapplied evolutions for accounts.
There are unapplied evolutions for scmtools.
Project signature has changed - an evolution is required
Installing custom SQL ...
Installing indexes ...
Installed 0 object(s) from 0 fixture(s)
CommandError: Error applying evolution: constraint "scmtools_repository_df00fa5c" of relation "scmtools_repository" does not exist


When Accessing review board I get this:
Traceback (most recent call last):

  File "/usr/lib/python2.6/site-packages/Django-1.6.5-py2.6.egg/django/core/handlers/base.py", line 88, in get_response
    response = middleware_method(request)

  File "/srv/reviewboard-env/lib/python2.6/site-packages/ReviewBoard-2.0.3-py2.6.egg/reviewboard/accounts/middleware.py", line 14, in process_request
    user = request.user.get_profile()

  File "/srv/reviewboard-env/lib/python2.6/site-packages/ReviewBoard-2.0.3-py2.6.egg/reviewboard/accounts/models.py", line 284, in _get_profile
    self._profile = Profile.objects.get(user=self)

  File "/usr/lib/python2.6/site-packages/Django-1.6.5-py2.6.egg/django/db/models/manager.py", line 151, in get
    return self.get_queryset().get(*args, **kwargs)

  File "/usr/lib/python2.6/site-packages/Django-1.6.5-py2.6.egg/django/db/models/query.py", line 304, in get
    num = len(clone)

  File "/usr/lib/python2.6/site-packages/Django-1.6.5-py2.6.egg/django/db/models/query.py", line 77, in __len__
    self._fetch_all()

  File "/usr/lib/python2.6/site-packages/Django-1.6.5-py2.6.egg/django/db/models/query.py", line 857, in _fetch_all
    self._result_cache = list(self.iterator())

  File "/usr/lib/python2.6/site-packages/Django-1.6.5-py2.6.egg/django/db/models/query.py", line 220, in iterator
    for row in compiler.results_iter():

  File "/usr/lib/python2.6/site-packages/Django-1.6.5-py2.6.egg/django/db/models/sql/compiler.py", line 713, in results_iter
    for rows in self.execute_sql(MULTI):

  File "/usr/lib/python2.6/site-packages/Django-1.6.5-py2.6.egg/django/db/models/sql/compiler.py", line 786, in execute_sql
    cursor.execute(sql, params)

  File "/usr/lib/python2.6/site-packages/Django-1.6.5-py2.6.egg/django/db/backends/util.py", line 53, in execute
    return self.cursor.execute(sql, params)

  File "/usr/lib/python2.6/site-packages/Django-1.6.5-py2.6.egg/django/db/utils.py", line 99, in __exit__
    six.reraise(dj_exc_type, dj_exc_value, traceback)

  File "/usr/lib/python2.6/site-packages/Django-1.6.5-py2.6.egg/django/db/backends/util.py", line 53, in execute
    return self.cursor.execute(sql, params)

ProgrammingError: column accounts_profile.should_send_email does not exist
LINE 1: ..._id", "accounts_profile"."first_time_setup_done", "accounts_...

What operating system are you using? What browser?


Please provide any additional information below.
Some info from our reviewdatabase. The  scmtools_repository_df00fa5c constrain is an index, and it seems to exist:


reviewboard=> \dS+ scmtools_repository
                                                           Table "public.scmtools_repository"
       Column       |          Type          |                            Modifiers                             | Storage  | Stats target | Description
--------------------+------------------------+------------------------------------------------------------------+----------+--------------+-------------
 id                 | integer                | not null default nextval('scmtools_repository_id_seq'::regclass) | plain    |              |
 name               | character varying(64)  | not null                                                         | extended |              |
 path               | character varying(255) | not null                                                         | extended |              |
 mirror_path        | character varying(255) | not null                                                         | extended |              |
 raw_file_url       | character varying(255) | not null                                                         | extended |              |
 username           | character varying(32)  | not null                                                         | extended |              |
 password           | character varying(128) | not null                                                         | extended |              |
 tool_id            | integer                | not null                                                         | plain    |              |
 bug_tracker        | character varying(256) | not null                                                         | extended |              |
 encoding           | character varying(32)  | not null                                                         | extended |              |
 visible            | boolean                | not null                                                         | plain    |              |
 local_site_id      | integer                |                                                                  | plain    |              |
 public             | boolean                | not null                                                         | plain    |              |
 extra_data         | text                   |                                                                  | extended |              |
 hosting_account_id | integer                |                                                                  | plain    |              |
Indexes:
    "scmtools_repository_pkey" PRIMARY KEY, btree (id)
    "scmtools_repository_97c59b73" UNIQUE, btree (name, local_site_id)
    "scmtools_repository_df00fa5c" UNIQUE, btree (path, local_site_id)
    "scmtools_repository_hosting_account_id" btree (hosting_account_id)
    "scmtools_repository_local_site_id" btree (local_site_id)
    "scmtools_repository_tool_id" btree (tool_id)
Foreign-key constraints:
    "scmtools_repository_hosting_account_id_fkey" FOREIGN KEY (hosting_account_id) REFERENCES hostingsvcs_hostingserviceaccount(id) DEFERRABLE INITIALLY DEFERRED
    "scmtools_repository_local_site_id_fkey" FOREIGN KEY (local_site_id) REFERENCES site_localsite(id) DEFERRABLE INITIALLY DEFERRED
    "scmtools_repository_tool_id_fkey" FOREIGN KEY (tool_id) REFERENCES scmtools_tool(id) DEFERRABLE INITIALLY DEFERRED
Referenced by:
    TABLE "attachments_fileattachment" CONSTRAINT "attachments_fileattachment_repository_id_fkey" FOREIGN KEY (repository_id) REFERENCES scmtools_repository(id) DEFERRABLE INITIALLY DEFERRED
    TABLE "scmtools_repository_users" CONSTRAINT "repository_id_refs_id_2600c334" FOREIGN KEY (repository_id) REFERENCES scmtools_repository(id) DEFERRABLE INITIALLY DEFERRED
    TABLE "diffviewer_diffset" CONSTRAINT "repository_id_refs_id_7acd9d8c" FOREIGN KEY (repository_id) REFERENCES scmtools_repository(id) DEFERRABLE INITIALLY DEFERRED
    TABLE "reviews_defaultreviewer_repository" CONSTRAINT "repository_id_refs_id_8161d2ae" FOREIGN KEY (repository_id) REFERENCES scmtools_repository(id) DEFERRABLE INITIALLY DEFERRED
    TABLE "reviews_reviewrequest" CONSTRAINT "repository_id_refs_id_a7582d83" FOREIGN KEY (repository_id) REFERENCES scmtools_repository(id) DEFERRABLE INITIALLY DEFERRED
    TABLE "scmtools_repository_review_groups" CONSTRAINT "repository_id_refs_id_c6cc1927" FOREIGN KEY (repository_id) REFERENCES scmtools_repository(id) DEFERRABLE INITIALLY DEFERRED
Has OIDs: no
#1 abhishek*********@gmai***** (Google Code) (Is this you? Claim this profile.) 
I had this issue as well. I don't quite get what django evolutions is up to; it seems like it's trying to drop a constraint instead of an index. I'm not honestly sure what the difference is in terms of postgresql since one has to be backed by the other anyways. I was able to get around this with the following psqls, followed by running the upgrade again. Not my fault if this doesn't work; just sayin':

reviewboard=> BEGIN;
BEGIN
reviewboard=> DROP INDEX scmtools_repository_df00fa5c;
DROP INDEX
reviewboard=> ALTER TABLE scmtools_repository ADD CONSTRAINT scmtools_repository_df00fa5c UNIQUE (path, local_site_id);
NOTICE:  ALTER TABLE / ADD UNIQUE will create implicit index "scmtools_repository_df00fa5c" for table "scmtools_repository"
ALTER TABLE
reviewboard=> \d scmtools_repository;
                                       Table "public.scmtools_repository"
       Column       |          Type          |                            Modifiers                             
--------------------+------------------------+------------------------------------------------------------------
 id                 | integer                | not null default nextval('scmtools_repository_id_seq'::regclass)
 name               | character varying(64)  | not null
 path               | character varying(255) | not null
 mirror_path        | character varying(255) | not null
 raw_file_url       | character varying(255) | not null
 username           | character varying(32)  | not null
 password           | character varying(128) | not null
 tool_id            | integer                | not null
 bug_tracker        | character varying(256) | not null
 encoding           | character varying(32)  | not null
 visible            | boolean                | not null
 local_site_id      | integer                | 
 public             | boolean                | not null
 extra_data         | text                   | 
 hosting_account_id | integer                | 
Indexes:
    "scmtools_repository_pkey" PRIMARY KEY, btree (id)
    "scmtools_repository_97c59b73" UNIQUE, btree (name, local_site_id)
    "scmtools_repository_df00fa5c" UNIQUE, btree (path, local_site_id)
    "scmtools_repository_hosting_account_id" btree (hosting_account_id)
    "scmtools_repository_local_site_id" btree (local_site_id)
    "scmtools_repository_tool_id" btree (tool_id)
Foreign-key constraints:
    "scmtools_repository_hosting_account_id_fkey" FOREIGN KEY (hosting_account_id) REFERENCES hostingsvcs_hostingserviceaccount(id) DEFERRABLE INITIALLY DEFERRED
    "scmtools_repository_local_site_id_fkey" FOREIGN KEY (local_site_id) REFERENCES site_localsite(id) DEFERRABLE INITIALLY DEFERRED
    "scmtools_repository_tool_id_fkey" FOREIGN KEY (tool_id) REFERENCES scmtools_tool(id) DEFERRABLE INITIALLY DEFERRED

reviewboard=> ALTER TABLE "scmtools_repository" DROP CONSTRAINT scmtools_repository_df00fa5c;
ALTER TABLE
reviewboard=> ROLLBACK;
ROLLBACK
reviewboard=> BEGIN;
BEGIN
reviewboard=> DROP INDEX scmtools_repository_df00fa5c;
DROP INDEX
reviewboard=> ALTER TABLE scmtools_repository ADD CONSTRAINT scmtools_repository_df00fa5c UNIQUE (path, local_site_id);
NOTICE:  ALTER TABLE / ADD UNIQUE will create implicit index "scmtools_repository_df00fa5c" for table "scmtools_repository"
ALTER TABLE
reviewboard=> COMMIT;
#2 abhishek*********@gmai***** (Google Code) (Is this you? Claim this profile.) 
bah, wrong copy and paste. You only need the last 4 commands. Oh well, doesn't hurt to run the whole set and see what you're getting :). Maybe a good reason to start bundling the evolutions when db changes are made instead of depending on --hint?
chipx86
#3 chipx86 
Wait, why are you ever running --hint? We bundle the evolutions.

That said, there is a bug here. Looking into it.
  • +NeedInfo
#4 abhishek*********@gmai***** (Google Code) (Is this you? Claim this profile.) 
I'm not running hint, it looked like that's what rb-site upgrade was doing. I could be wrong though, not entirely sure how this code works
#5 abhishek*********@gmai***** (Google Code) (Is this you? Claim this profile.) 
This, from the sounds of it, might have been fixed by 2.0.5? The release notes suggest "django-evolutions" changes. not sure if that encompasses this or not.
david
#6 david 
Yes, this should be fixed by 2.0.5
  • -NeedInfo
    +Fixed
#7 ric****@gmai***** (Google Code) (Is this you? Claim this profile.) 
Got the same issue when upgrading from 2.0.1 to 2.0.5. Manually fixed with Abhishek's commands.
chipx86
#8 chipx86 
What database?

Happen to have a dump of your 2.0.1 available still?
#9 robottomon***********@gmai***** (Google Code) (Is this you? Claim this profile.) 
We had the same issue going from 2.0 to 2.0.5 using postgres as the database.  We followed this workaround and it worked.
#10 vzei****@gmai***** (Google Code) (Is this you? Claim this profile.) 
Just got the same issue upgrading from 2.0.2 to 2.0.11 so this is definitely not fixed yet.

Thanks a lot to Abhishek for the workaround!