4286: RBTools return 1 with an HTTP 500 telling there was an error when posting a review with unicode.

ffogell

What version are you running?

RBTools 0.7.5

What steps will reproduce the problem?

  1. Post a review from perforce with é ĉ ç or any extended ascii char in changelist

What is the expected output? What do you see instead?

The review is actually created but not the diff

What operating system are you using?

Win7

Attach the debug out from the command.

Warning at /api/review-requests/
Incorrect string value: '\xE9\xE9 \xE7\xE7\xE7...' for column 'summary' at row 1
Request Method: POST
Request URL: https://tg-reviewboard01.ubisoft.org/api/review-requests/
Django Version: 1.4.10
Exception Type: Warning
Exception Value:
Incorrect string value: '\xE9\xE9 \xE7\xE7\xE7...' for column 'summary' at row 1
Exception Location: /usr/lib64/python2.6/site-packages/MySQLdb/cursors.py in _warning_check, line 89
Python Executable: /usr/bin/python
Python Version: 2.6.6
Python Path:
['/var/www/html/tg.reviewboard01.ubisoft.org/conf',
'/usr/lib/python2.6/site-packages/python_dateutil-1.5-py2.6.egg',
'/usr/lib/python2.6/site-packages/Pygments-1.6rc1-py2.6.egg',
'/usr/lib/python2.6/site-packages/paramiko-1.9.0-py2.6.egg',
'/usr/lib/python2.6/site-packages/pycrypto-2.6-py2.6-linux-x86_64.egg',
'/usr/lib/python2.6/site-packages/mimeparse-0.1.3-py2.6.egg',
'/usr/lib/python2.6/site-packages/Markdown-2.2.1-py2.6.egg',
'/usr/lib/python2.6/site-packages/docutils-0.10-py2.6.egg',
'/usr/lib/python2.6/site-packages/django_pipeline-1.2.24-py2.6.egg',
'/usr/lib/python2.6/site-packages/feedparser-5.1.3-py2.6.egg',
'/usr/lib/python2.6/site-packages/pytz-2012j-py2.6.egg',
'/usr/lib/python2.6/site-packages/ReviewBoard-1.7.21-py2.6.egg',
'/usr/lib/python2.6/site-packages/Djblets-0.7.28-py2.6.egg',
'/usr/lib/python2.6/site-packages/Django-1.4.10-py2.6.egg',
'/usr/lib64/python26.zip',
'/usr/lib64/python2.6',
'/usr/lib64/python2.6/plat-linux2',
'/usr/lib64/python2.6/lib-tk',
'/usr/lib64/python2.6/lib-old',
'/usr/lib64/python2.6/lib-dynload',
'/usr/lib64/python2.6/site-packages',
'/usr/lib64/python2.6/site-packages/PIL',
'/usr/lib/python2.6/site-packages',
'/usr/lib/python2.6/site-packages/setuptools-0.6c11-py2.6.egg-info']
Server time: Wed, 2 Mar 2016 14:50:39 +0000

Environment:

Request Method: POST
Request URL: https://tg-reviewboard01.xxxx/api/review-requests/

Django Version: 1.4.10
Python Version: 2.6.6
Installed Applications:
['django.contrib.admin',
'django.contrib.auth',
'django.contrib.contenttypes',
'django.contrib.markup',
'django.contrib.sites',
'django.contrib.sessions',
'django.contrib.staticfiles',
'djblets.datagrid',
'djblets.extensions',
'djblets.feedview',
'djblets.gravatars',
'djblets.log',
'djblets.pipeline',
'djblets.siteconfig',
'djblets.util',
'djblets.webapi',
'pipeline',
'reviewboard.accounts',
'reviewboard.admin',
'reviewboard.attachments',
'reviewboard.changedescs',
'reviewboard.diffviewer',
'reviewboard.extensions',
'reviewboard.hostingsvcs',
'reviewboard.notifications',
'reviewboard.reviews',
'reviewboard.reviews.ui',
'reviewboard.scmtools',
'reviewboard.site',
'reviewboard.ssh',
'reviewboard.webapi',
'django_evolution']
Installed Middleware:
['django.middleware.gzip.GZipMiddleware',
'reviewboard.admin.middleware.InitReviewBoardMiddleware',
'django.middleware.clickjacking.XFrameOptionsMiddleware',
'django.middleware.common.CommonMiddleware',
'django.middleware.doc.XViewMiddleware',
'django.middleware.http.ConditionalGetMiddleware',
'django.middleware.locale.LocaleMiddleware',
'django.contrib.sessions.middleware.SessionMiddleware',
'django.contrib.auth.middleware.AuthenticationMiddleware',
'django.contrib.messages.middleware.MessageMiddleware',
'djblets.siteconfig.middleware.SettingsMiddleware',
'reviewboard.admin.middleware.LoadSettingsMiddleware',
'djblets.extensions.middleware.ExtensionsMiddleware',
'djblets.log.middleware.LoggingMiddleware',
'reviewboard.accounts.middleware.TimezoneMiddleware',
'reviewboard.admin.middleware.CheckUpdatesRequiredMiddleware',
'reviewboard.admin.middleware.X509AuthMiddleware',
'reviewboard.site.middleware.LocalSiteMiddleware',
'djblets.extensions.middleware.ExtensionsMiddlewareRunner']

Traceback:
File "/usr/lib/python2.6/site-packages/Django-1.4.10-py2.6.egg/django/core/handlers/base.py" in get_response
111. response = callback(request, callback_args, callback_kwargs)
File "/usr/lib/python2.6/site-packages/Django-1.4.10-py2.6.egg/django/views/decorators/cache.py" in _wrapped_view_func
89. response = view_func(request, *args,
kwargs)
File "/usr/lib/python2.6/site-packages/Django-1.4.10-py2.6.egg/django/views/decorators/vary.py" in inner_func
19. response = func(
args, kwargs)
File "/usr/lib/python2.6/site-packages/Djblets-0.7.28-py2.6.egg/djblets/webapi/resources.py" in call
482. result = view(request, api_format=api_format, *args,
kwargs)
File "/usr/lib/python2.6/site-packages/Djblets-0.7.28-py2.6.egg/djblets/webapi/resources.py" in post
692. return self.create(args, kwargs)
File "/usr/lib/python2.6/site-packages/ReviewBoard-1.7.21-py2.6.egg/reviewboard/webapi/decorators.py" in _check
127. return view_func(*args,
kwargs)
File "/usr/lib/python2.6/site-packages/Djblets-0.7.28-py2.6.egg/djblets/webapi/decorators.py" in _call
113. return view_func(
args, kwargs)
File "/usr/lib/python2.6/site-packages/Djblets-0.7.28-py2.6.egg/djblets/webapi/decorators.py" in _checklogin
134. return view_func(*args,
kwargs)
File "/usr/lib/python2.6/site-packages/Djblets-0.7.28-py2.6.egg/djblets/webapi/decorators.py" in _call
113. return view_func(args, kwargs)
File "/usr/lib/python2.6/site-packages/Djblets-0.7.28-py2.6.egg/djblets/webapi/decorators.py" in _call
113. return view_func(*args,
kwargs)
File "/usr/lib/python2.6/site-packages/Djblets-0.7.28-py2.6.egg/djblets/webapi/decorators.py" in _validate
283. return view_func(
args, new_kwargs)
File "/usr/lib/python2.6/site-packages/ReviewBoard-1.7.21-py2.6.egg/reviewboard/webapi/resources.py" in create
7135. changenum, local_site)
File "/usr/lib/python2.6/site-packages/ReviewBoard-1.7.21-py2.6.egg/reviewboard/reviews/managers.py" in create
124. review_request.save()
File "/usr/lib/python2.6/site-packages/ReviewBoard-1.7.21-py2.6.egg/reviewboard/reviews/models.py" in save
775. super(ReviewRequest, self).save(
kwargs)
File "/usr/lib/python2.6/site-packages/ReviewBoard-1.7.21-py2.6.egg/reviewboard/reviews/models.py" in save
420. super(BaseReviewRequestDetails, self).save(**kwargs)
File "/usr/lib/python2.6/site-packages/Django-1.4.10-py2.6.egg/django/db/models/base.py" in save
463. self.save_base(using=using, force_insert=force_insert, force_update=force_update)
File "/usr/lib/python2.6/site-packages/Django-1.4.10-py2.6.egg/django/db/models/base.py" in save_base
529. rows = manager.using(using).filter(pk=pk_val)._update(values)
File "/usr/lib/python2.6/site-packages/Django-1.4.10-py2.6.egg/django/db/models/query.py" in _update
560. return query.get_compiler(self.db).execute_sql(None)
File "/usr/lib/python2.6/site-packages/Django-1.4.10-py2.6.egg/django/db/models/sql/compiler.py" in execute_sql
988. cursor = super(SQLUpdateCompiler, self).execute_sql(result_type)
File "/usr/lib/python2.6/site-packages/Django-1.4.10-py2.6.egg/django/db/models/sql/compiler.py" in execute_sql
818. cursor.execute(sql, params)
File "/usr/lib/python2.6/site-packages/Djblets-0.7.28-py2.6.egg/djblets/log/middleware.py" in execute
49. return self.cursor.execute(sql, params)
File "/usr/lib/python2.6/site-packages/Django-1.4.10-py2.6.egg/django/db/backends/mysql/base.py" in execute
114. return self.cursor.execute(query, args)
File "/usr/lib64/python2.6/site-packages/MySQLdb/cursors.py" in execute
175. if not self._defer_warnings: self._warning_check()
File "/usr/lib64/python2.6/site-packages/MySQLdb/cursors.py" in _warning_check
89. warn(w[-1], self.Warning, 3)

Exception Type: Warning at /api/review-requests/
Exception Value: Incorrect string value: '\xE9\xE9 \xE7\xE7\xE7...' for column 'summary' at row 1

$ rbt <command> --debug <args>
<debug output>

Please provide any additional information below.

Like I said the review request is created but the diff isnt.
I seriously needs help because been 2 days I trying to work on this.

So the problem seem to go with the diff part when posting
Is there a way to make Reviewboard work with unicode ?
Would it be possible to improve debug information because in my case I have zero idea on what the problem is.

#1 ffogell

I added the full log.

  • +
    +		Logs	{>>> RBTools 0.7.5 alpha 0 (dev)
    >>> Python 2.7.6 (default, Nov 10 2013, 19:24:18) [MSC v.1500 32 bit (Intel)]
    >>> Running on Windows-7-6.1.7601-SP1
    >>> Home = C:\Users\fjean\AppData\Roaming
    >>> Current directory = F:\p4\tg1716\productivity\team\dev\output\submitassistant\SubmitAssistant_UI\Debug
    >>> Checking for a Subversion repository...
    >>> Unable to execute "svn help": skipping SVN
    >>> Checking for a Git repository...
    >>> Unable to execute "git --help" or "git.cmd --help": skipping Git
    >>> Checking for a Mercurial repository...
    >>> Unable to execute "hg --help": skipping Mercurial
    >>> Checking for a CVS repository...
    >>> Unable to execute "cvs": skipping CVS
    >>> Checking for a Perforce repository...
    >>> Running: p4 -c MTL-WKS-BC807 -p p4-techgroup.mtl.toto.org:1716 info
    >>> Running: diff --version
    >>> repository info: Path: [u'p4-techgroup.toto.org:1716', u'p4-techgroup.mtl.toto.org:1716'], Base path: None, Supports changesets: True
    >>> Making HTTP GET requ
#2 ffogell

It seem it is realted to the database. But the problem is i get zero information.

From the server :
None - guest - /api/review-requests/40337/ - Internal Server Error: /api/review-requests/40337/
Traceback (most recent call last):
File "/usr/lib/python2.6/site-packages/Django-1.4.10-py2.6.egg/django/core/handlers/base.py", line 111, in get_response
response = callback(request, callback_args, callback_kwargs)
File "/usr/lib/python2.6/site-packages/Django-1.4.10-py2.6.egg/django/views/decorators/cache.py", line 89, in _wrapped_view_func
response = view_func(request, *args,
kwargs)
File "/usr/lib/python2.6/site-packages/Django-1.4.10-py2.6.egg/django/views/decorators/vary.py", line 19, in inner_func
response = func(
args, kwargs)
File "/usr/lib/python2.6/site-packages/Djblets-0.7.28-py2.6.egg/djblets/webapi/resources.py", line 482, in call
result = view(request, api_format=api_format, *args,
kwargs)
File "/usr/lib/python2.6/site-packages/Djblets-0.7.28-py2.6.egg/djblets/webapi/resources.py", line 707, in put
return self.update(request, args, kwargs)
File "/usr/lib/python2.6/site-packages/ReviewBoard-1.7.21-py2.6.egg/reviewboard/webapi/decorators.py", line 127, in _check
return view_func(*args,
kwargs)
File "/usr/lib/python2.6/site-packages/Djblets-0.7.28-py2.6.egg/djblets/webapi/decorators.py", line 113, in _call
return view_func(
args, kwargs)
File "/usr/lib/python2.6/site-packages/Djblets-0.7.28-py2.6.egg/djblets/webapi/decorators.py", line 134, in _checklogin
return view_func(*args,
kwargs)
File "/usr/lib/python2.6/site-packages/Djblets-0.7.28-py2.6.egg/djblets/webapi/decorators.py", line 113, in _call
return view_func(args, kwargs)
File "/usr/lib/python2.6/site-packages/Djblets-0.7.28-py2.6.egg/djblets/webapi/decorators.py", line 113, in _call
return view_func(*args,
kwargs)
File "/usr/lib/python2.6/site-packages/Djblets-0.7.28-py2.6.egg/djblets/webapi/decorators.py", line 283, in _validate
return view_func(
args, new_kwargs)
File "/usr/lib/python2.6/site-packages/ReviewBoard-1.7.21-py2.6.egg/reviewboard/webapi/resources.py", line 7253, in update
draft.save()
File "/usr/lib/python2.6/site-packages/ReviewBoard-1.7.21-py2.6.egg/reviewboard/reviews/models.py", line 420, in save
super(BaseReviewRequestDetails, self).save(
kwargs)
File "/usr/lib/python2.6/site-packages/Django-1.4.10-py2.6.egg/django/db/models/base.py", line 463, in save
self.save_base(using=using, force_insert=force_insert, force_update=force_update)
File "/usr/lib/python2.6/site-packages/Django-1.4.10-py2.6.egg/django/db/models/base.py", line 529, in save_base
rows = manager.using(using).filter(pk=pk_val)._update(values)
File "/usr/lib/python2.6/site-packages/Django-1.4.10-py2.6.egg/django/db/models/query.py", line 560, in _update
return query.get_compiler(self.db).execute_sql(None)
File "/usr/lib/python2.6/site-packages/Django-1.4.10-py2.6.egg/django/db/models/sql/compiler.py", line 988, in execute_sql
cursor = super(SQLUpdateCompiler, self).execute_sql(result_type)
File "/usr/lib/python2.6/site-packages/Django-1.4.10-py2.6.egg/django/db/models/sql/compiler.py", line 818, in execute_sql
cursor.execute(sql, params)
File "/usr/lib/python2.6/site-packages/Djblets-0.7.28-py2.6.egg/djblets/log/middleware.py", line 49, in execute
return self.cursor.execute(sql, params)
File "/usr/lib/python2.6/site-packages/Django-1.4.10-py2.6.egg/django/db/backends/mysql/base.py", line 114, in execute
return self.cursor.execute(query, args)
File "/usr/lib64/python2.6/site-packages/MySQLdb/cursors.py", line 175, in execute
if not self._defer_warnings: self._warning_check()
File "/usr/lib64/python2.6/site-packages/MySQLdb/cursors.py", line 89, in _warning_check
warn(w[-1], self.Warning, 3)
Warning: Incorrect string value: '\xE9\xE9\xE9\xE9\xE9 ...' for column 'summary' at row 1

#3 ffogell

So everything seem related with database ?

david
#4 david

This is a problem where the database was not created with the correct column encoding. There's a fair amount of documentation for mysql on how to set or change the text encoding.

  • -New
    +SetupIssue