3870: UnicodeDecodeError: 'ascii' codec can't decode byte 0xd0 in position 1: ordinal not in range(128)

paff*****@gmai***** (Google Code) (Is this you? Claim this profile.)
david
 david
May 17, 2015 
What version are you running?
0.7.3alpha0

What's the URL of the page containing the problem?
Our local svn server.

What steps will reproduce the problem?
I am creating a remote svn post-commit rbt post hook.

What is the expected output? What do you see instead?
I am expecting my diff to get uploaded to the review board server.

What operating system are you using? What browser?
CentOS with kernel 2.6.32-279.el6.x86_64

Please provide any additional information below.
It fails on most files, but if I create a simple text document with few changes in it - it works.

Here's a part of the debug log:

>>> Running: svn diff --non-interactive --diff-cmd=diff --notice-ancestry http://xxxxxxxxxxxxxxxxxx@206 http://xxxxxxxxxxxxxxxxxx@207 --username xxxxxx --password xxxxxx
Traceback (most recent call last):
  File "/usr/bin/rbt", line 9, in <module>
    load_entry_point('RBTools==0.7.3alpha0.dev', 'console_scripts', 'rbt')()
  File "/usr/lib/python2.6/site-packages/RBTools-0.7.3alpha0.dev-py2.6.egg/rbtools/commands/main.py", line 133, in main
    command.run_from_argv([RB_MAIN, command_name] + args)
  File "/usr/lib/python2.6/site-packages/RBTools-0.7.3alpha0.dev-py2.6.egg/rbtools/commands/__init__.py", line 580, in run_from_argv
    exit_code = self.main(*args) or 0
  File "/usr/lib/python2.6/site-packages/RBTools-0.7.3alpha0.dev-py2.6.egg/rbtools/commands/post.py", line 640, in main
    extra_args=extra_args)
  File "/usr/lib/python2.6/site-packages/RBTools-0.7.3alpha0.dev-py2.6.egg/rbtools/clients/svn.py", line 369, in diff
    'diff': b''.join(diff),
UnicodeDecodeError: 'ascii' codec can't decode byte 0xd0 in position 47: ordinal not in range(128)
david
#1 david
  • +PendingReview
  • +Component-RBTools
  • +david
david
#2 david 
Fixed in release-0.7.x (8d4c064). This will ship in 0.7.3. Thanks!
  • -PendingReview
    +Fixed
#4 thibaul********@gmai***** (Google Code) (Is this you? Claim this profile.) 
I have the same problem here
I'm trying to setup a post-commit hook based on contrib/tools/svn-hook-postcommit-review (updated to last RBTools syntax) 

And the same error is raised on 'diff': b''.join(diff)

Here are some version details
 - Debian Linux 3.2.0-4-amd64 #1 SMP Debian 3.2.63-2
 - apache 2.4.10 (Debian)
 - subversion 1.8.10 (r1615264)
 - python 2.7.9
 - python-svn 1.7.8-1
And
 - ReviewBoard 2.0.18
 - RBTools 0.7.4

Command line invoked
rbt post --repository=REP --username=USR --password=PWD --submit-as=ORG --summary=SUM --description=DSC --debug revN:revM

Traceback (most recent call last):
  File "/usr/local/bin/rbt", line 9, in <module>
    load_entry_point('RBTools==0.7.4', 'console_scripts', 'rbt')()
  File "/usr/local/lib/python2.7/dist-packages/RBTools-0.7.4-py2.7.egg/rbtools/commands/main.py", line 133, in main
    command.run_from_argv([RB_MAIN, command_name] + args)
  File "/usr/local/lib/python2.7/dist-packages/RBTools-0.7.4-py2.7.egg/rbtools/commands/__init__.py", line 612, in run_from_argv
    exit_code = self.main(*args) or 0
  File "/usr/local/lib/python2.7/dist-packages/RBTools-0.7.4-py2.7.egg/rbtools/commands/post.py", line 691, in main
    extra_args=extra_args)
  File "/usr/local/lib/python2.7/dist-packages/RBTools-0.7.4-py2.7.egg/rbtools/clients/svn.py", line 365, in diff
    'diff': b''.join(diff),
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe9 in position 8: ordinal not in range(128)
#5 thibaul********@gmai***** (Google Code) (Is this you? Claim this profile.) 
Found a quick fix

I replaced 
   'diff': b''.join(diff),
By
   'diff': b''.join("%s" % (diff_item.decode('cp1252')) for diff_item in diff)

in rbtools/clients/svn.py

This works for me because my source files are written with Windows IDE, in France ;). Maybe there is a more elegant (and international solution) but this fix works for me.
david
#6 david 
You can add additional encodings in the "Encoding" field under "Advanced Settings" for each repository.
Stilor
#7 Stilor 
The setting is described as "The encoding used for files in this repository". I am not sure that a single "encoding" for the repository is sufficient: for example, our repository has multiple third-party modules; most of them are in UTF-8 but some use more legacy encodings like, apparently, ISO-8859-1. It would be nice to be able to configure multiple encodings as fallbacks (e.g. "first try utf-8, then iso-8859-1, then cp1252") or configure the encoding on portions of the repository (e.g. by setting some SVN property - reviewboard:encoding for example - on a directory to indicate the module's encoding).
Stilor
#8 Stilor 
For the record, I am seeing this with RBTools 1.0.1:


Traceback (most recent call last):
  File "/usr/local/bin/rbt", line 11, in <module>
    sys.exit(main())
  File "/usr/local/lib/python2.7/dist-packages/rbtools/commands/main.py", line 120, in main
    command.run_from_argv([RB_MAIN, command_name] + args)
  File "/usr/local/lib/python2.7/dist-packages/rbtools/commands/__init__.py", line 732, in run_from_argv
    exit_code = self.main(*args) or 0
  File "/usr/local/lib/python2.7/dist-packages/rbtools/commands/post.py", line 806, in main
    extra_args=extra_args)
  File "/usr/local/lib/python2.7/dist-packages/rbtools/clients/svn.py", line 578, in diff
    'diff': b''.join(diff),
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 29: ordinal not in range(128)
Stilor
#9 Stilor 
Would also be nice if rbt pinpointed the exact file that's giving it such trouble - pinpointing the culprit among ~500 changed files is no fun :(
david
#10 david 
Alexey, what version of Python are you running?
  • -Fixed
    +NeedInfo
  • -david
    +david
david
#11 david
  • -NeedInfo
    +Fixed