3803: Error publishing review due to unicode characters

msunde
What version are you running?
2.0.14

What's the URL of the page containing the problem?
http://review.XXXX/r/991/

What steps will reproduce the problem?
1. Create a review request with french characters.
2. Publish the review request
3. There is an error in the log.

What is the expected output? What do you see instead?
Review should be properly published.

What operating system are you using? What browser?
Amazon Linux

Please provide any additional information below.

This is similar to bug #3796

From the reviewboard log file:

2015-03-11 17:57:33,975 - ERROR -  - Exception thrown for user XXXX at http://review.XXXX/api/review-requests/991/draft/?api_format=json&force-text-type=html&include-text-types=raw&expand=depends_on%2Ctarget_people%2Ctarget_groups

'ascii' codec can't decode byte 0xc3 in position 22: ordinal not in range(128)
Traceback (most recent call last):
  File "/usr/lib/python2.6/site-packages/Django-1.6.10-py2.6.egg/django/core/handlers/base.py", line 112, in get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/usr/lib/python2.6/site-packages/Django-1.6.10-py2.6.egg/django/views/decorators/cache.py", line 52, in _wrapped_view_func
    response = view_func(request, *args, **kwargs)
  File "/usr/lib/python2.6/site-packages/Django-1.6.10-py2.6.egg/django/views/decorators/vary.py", line 19, in inner_func
    response = func(*args, **kwargs)
  File "/usr/lib/python2.6/site-packages/Djblets-0.8.16-py2.6.egg/djblets/webapi/resources.py", line 498, in __call__
    request, method, view, api_format=api_format, *args, **kwargs)
  File "/usr/lib/python2.6/site-packages/Djblets-0.8.16-py2.6.egg/djblets/webapi/resources.py", line 569, in call_method_view
    return view(request, *args, **kwargs)
  File "/usr/lib/python2.6/site-packages/Djblets-0.8.16-py2.6.egg/djblets/webapi/decorators.py", line 117, in _call
    return view_func(*args, **kwargs)
  File "/usr/lib/python2.6/site-packages/ReviewBoard-2.0.14-py2.6.egg/reviewboard/webapi/decorators.py", line 110, in _check
    return view_func(*args, **kwargs)
  File "/usr/lib/python2.6/site-packages/Djblets-0.8.16-py2.6.egg/djblets/webapi/decorators.py", line 117, in _call
    return view_func(*args, **kwargs)
  File "/usr/lib/python2.6/site-packages/Djblets-0.8.16-py2.6.egg/djblets/webapi/decorators.py", line 138, in _checklogin
    return view_func(*args, **kwargs)
  File "/usr/lib/python2.6/site-packages/Djblets-0.8.16-py2.6.egg/djblets/util/decorators.py", line 78, in _call
    f = augmented_func(*args, **kwargs)
  File "/usr/lib/python2.6/site-packages/Djblets-0.8.16-py2.6.egg/djblets/webapi/decorators.py", line 117, in _call
    return view_func(*args, **kwargs)
  File "/usr/lib/python2.6/site-packages/ReviewBoard-2.0.14-py2.6.egg/reviewboard/webapi/decorators.py", line 31, in _check
    return webapi_login_required(view_func)(*args, **kwargs)
  File "/usr/lib/python2.6/site-packages/Djblets-0.8.16-py2.6.egg/djblets/webapi/decorators.py", line 117, in _call
    return view_func(*args, **kwargs)
  File "/usr/lib/python2.6/site-packages/Djblets-0.8.16-py2.6.egg/djblets/webapi/decorators.py", line 138, in _checklogin
    return view_func(*args, **kwargs)
  File "/usr/lib/python2.6/site-packages/Djblets-0.8.16-py2.6.egg/djblets/webapi/decorators.py", line 117, in _call
    return view_func(*args, **kwargs)
  File "/usr/lib/python2.6/site-packages/ReviewBoard-2.0.14-py2.6.egg/reviewboard/webapi/decorators.py", line 110, in _check
    return view_func(*args, **kwargs)
  File "/usr/lib/python2.6/site-packages/Djblets-0.8.16-py2.6.egg/djblets/util/decorators.py", line 78, in _call
    f = augmented_func(*args, **kwargs)
  File "/usr/lib/python2.6/site-packages/Djblets-0.8.16-py2.6.egg/djblets/webapi/decorators.py", line 117, in _call
    return view_func(*args, **kwargs)
  File "/usr/lib/python2.6/site-packages/Djblets-0.8.16-py2.6.egg/djblets/webapi/resources.py", line 790, in get
    etag = self.get_etag(request, obj)
  File "/usr/lib/python2.6/site-packages/Djblets-0.8.16-py2.6.egg/djblets/webapi/resources.py", line 1275, in get_etag
    encode_etag=False)
  File "/usr/lib/python2.6/site-packages/Djblets-0.8.16-py2.6.egg/djblets/webapi/resources.py", line 1305, in generate_etag
    etag = repr(self.serialize_object(obj, request=request, *kwargs))
  File "/usr/lib/python2.6/site-packages/ReviewBoard-2.0.14-py2.6.egg/reviewboard/webapi/mixins.py", line 60, in serialize_object
    obj, *args, **kwargs)
  File "/usr/lib/python2.6/site-packages/Djblets-0.8.16-py2.6.egg/djblets/webapi/resources.py", line 1057, in serialize_object
    'title': six.text_type(value),
  File "/usr/lib/python2.6/site-packages/ReviewBoard-2.0.14-py2.6.egg/reviewboard/reviews/models/base_review_request_details.py", line 283, in __str__
    return six.text_type(self.summary)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 22: ordinal not in range(128)
#1 msunde
Any update on this one? The problem is still occurring with the update to reviewboard 2.0.18:


2015-07-28 21:36:50,643 - ERROR -  - Exception thrown for user XXXX at http://review.XXXXX/api/review-requests/3111/draft/?api_format=json&force-text-type=html&include-text-types=raw%20&exp
and=depends_on%2Ctarget_people%2Ctarget_groups

'ascii' codec can't decode byte 0xc3 in position 40: ordinal not in range(128)
Traceback (most recent call last):
  File "/usr/local/lib/python2.6/site-packages/Django-1.6.11-py2.6.egg/django/core/handlers/base.py", line 112, in get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/usr/local/lib/python2.6/site-packages/Django-1.6.11-py2.6.egg/django/views/decorators/cache.py", line 52, in _wrapped_view_func
    response = view_func(request, *args, **kwargs)
  File "/usr/local/lib/python2.6/site-packages/Django-1.6.11-py2.6.egg/django/views/decorators/vary.py", line 19, in inner_func
    response = func(*args, **kwargs)
  File "/usr/local/lib/python2.6/site-packages/Djblets-0.8.21-py2.6.egg/djblets/webapi/resources.py", line 497, in __call__
    request, method, view, api_format=api_format, *args, **kwargs)
  File "/usr/local/lib/python2.6/site-packages/Djblets-0.8.21-py2.6.egg/djblets/webapi/resources.py", line 568, in call_method_view
    return view(request, *args, **kwargs)
  File "/usr/local/lib/python2.6/site-packages/Djblets-0.8.21-py2.6.egg/djblets/webapi/decorators.py", line 117, in _call
    return view_func(*args, **kwargs)
  File "/usr/local/lib/python2.6/site-packages/ReviewBoard-2.0.18-py2.6.egg/reviewboard/webapi/decorators.py", line 110, in _check
    return view_func(*args, **kwargs)
  File "/usr/local/lib/python2.6/site-packages/Djblets-0.8.21-py2.6.egg/djblets/webapi/decorators.py", line 117, in _call
    return view_func(*args, **kwargs)
  File "/usr/local/lib/python2.6/site-packages/Djblets-0.8.21-py2.6.egg/djblets/webapi/decorators.py", line 138, in _checklogin
    return view_func(*args, **kwargs)
  File "/usr/local/lib/python2.6/site-packages/Djblets-0.8.21-py2.6.egg/djblets/util/decorators.py", line 78, in _call
    f = augmented_func(*args, **kwargs)
  File "/usr/local/lib/python2.6/site-packages/Djblets-0.8.21-py2.6.egg/djblets/webapi/decorators.py", line 117, in _call
    return view_func(*args, **kwargs)
  File "/usr/local/lib/python2.6/site-packages/ReviewBoard-2.0.18-py2.6.egg/reviewboard/webapi/decorators.py", line 31, in _check
    return webapi_login_required(view_func)(*args, **kwargs)
  File "/usr/local/lib/python2.6/site-packages/Djblets-0.8.21-py2.6.egg/djblets/webapi/decorators.py", line 117, in _call
    return view_func(*args, **kwargs)
  File "/usr/local/lib/python2.6/site-packages/Djblets-0.8.21-py2.6.egg/djblets/webapi/decorators.py", line 138, in _checklogin
    return view_func(*args, **kwargs)
  File "/usr/local/lib/python2.6/site-packages/Djblets-0.8.21-py2.6.egg/djblets/webapi/decorators.py", line 117, in _call
    return view_func(*args, **kwargs)
  File "/usr/local/lib/python2.6/site-packages/ReviewBoard-2.0.18-py2.6.egg/reviewboard/webapi/decorators.py", line 110, in _check
    return view_func(*args, **kwargs)
  File "/usr/local/lib/python2.6/site-packages/Djblets-0.8.21-py2.6.egg/djblets/util/decorators.py", line 78, in _call
    f = augmented_func(*args, **kwargs)
  File "/usr/local/lib/python2.6/site-packages/Djblets-0.8.21-py2.6.egg/djblets/webapi/decorators.py", line 117, in _call
    return view_func(*args, **kwargs)
  File "/usr/local/lib/python2.6/site-packages/Djblets-0.8.21-py2.6.egg/djblets/webapi/resources.py", line 789, in get
    etag = self.get_etag(request, obj, **kwargs)
  File "/usr/local/lib/python2.6/site-packages/Djblets-0.8.21-py2.6.egg/djblets/webapi/resources.py", line 1290, in get_etag
    encode_etag=False, **kwargs)
  File "/usr/local/lib/python2.6/site-packages/Djblets-0.8.21-py2.6.egg/djblets/webapi/resources.py", line 1318, in generate_etag
    etag = repr(self.serialize_object(obj, request=request, **kwargs))
  File "/usr/local/lib/python2.6/site-packages/ReviewBoard-2.0.18-py2.6.egg/reviewboard/webapi/mixins.py", line 60, in serialize_object
    obj, *args, **kwargs)
  File "/usr/local/lib/python2.6/site-packages/Djblets-0.8.21-py2.6.egg/djblets/webapi/resources.py", line 1067, in serialize_object
    'title': six.text_type(value),
  File "/usr/local/lib/python2.6/site-packages/ReviewBoard-2.0.18-py2.6.egg/reviewboard/reviews/models/base_review_request_details.py", line 276, in __str__
    return six.text_type(self.summary)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 40: ordinal not in range(128)
#2 viney

Hi, Any plans of taking it up soon? I hit this error quite frequently.

chipx86
#3 chipx86

Review Board fully supports Unicode characters, but the database may not be configured to store the data properly as UTF-8. Are you the administrator? Can you check on the encoding of the database and tables? If they're storing as UTF-8, then this should work as expected.

  • -New
    +NeedInfo
  • +Unicode
#4 viney

The encoding for DB and tables is latin1_swedish_ci. Will change it to UTF-8 and post an update on how it behaves.
Thanks.

david
#5 david
  • -NeedInfo
    +Incomplete