3376: RBTools 0.6 - UnicodeEncodeError with ISO-8859-15

Nicolas*******@gmai***** (Google Code) (Is this you? Claim this profile.)
July 11, 2014
What version are you running?
ReviewBoard 2.0 / RBTools 0.6

What's the URL of the page containing the problem?
Not applicable as it's a RBTools issue.
What steps will reproduce the problem?
1. Create a new review with RBTools using ISO-8859-15
2. See RBTools log
Expected output: A review should be created with the corresponding diff.
Actual Result:
>>> Got API Error 105 (HTTP code 400): One or more fields had errors 
>>> Error data: {u'fields': {u'path': [u"Unable to parse diff revision 
>>> header '(r\xc3\xa9vision 0)'"]}, u'stat': u'fail', u'err': {u'msg': 
>>> u'One or more fields had errors', u'code': 105}}
Traceback (most recent call last):
  File "/usr/local/bin/rbt", line 9, in <module>
    load_entry_point('RBTools==0.6', 'console_scripts', 'rbt')()
  File "/usr/local/lib/python2.6/dist-packages/RBTools-0.6-py2.6.egg/rbtools/commands/main.py", line 134, in main
    command.run_from_argv([RB_MAIN, command_name] + args)
  File "/usr/local/lib/python2.6/dist-packages/RBTools-0.6-py2.6.egg/rbtools/commands/__init__.py", line 422, in run_from_argv
    exit_code = self.main(*args) or 0
  File "/usr/local/lib/python2.6/dist-packages/RBTools-0.6-py2.6.egg/rbtools/commands/post.py", line 769, in main
  File "/usr/local/lib/python2.6/dist-packages/RBTools-0.6-py2.6.egg/rbtools/commands/post.py", line 539, in post_request
    error_msg.append(str(e) + '\n')
UnicodeEncodeError: 'ascii' codec can't encode characters in position 107-108: ordinal not in range(128)
Operating system: Windows 7-64
Browsers: Internet Explorer 8 / Google Chrome


Format error has been corrected in some python files for Review Board, but it seems that this issue is present since several RBTools versions.
It works well for text files with UTF-8 format, but when we want to upload diff with ISO-8859-15, we have the error describe in the "actual result" part.

In ISO-8859-15, there are characters such as "éèùàçê".

Thanks and Best regards.
#1 david
There are multiple issues going on here. One is a failure in rbtools to show a good error. The other is a failure to parse the diff.

The fix for the error message is easy, but I need some more data about the reason why there's an error to begin with. Can you detail the exact steps that you performed (not just "Create a new review with RBTools using ISO-8859-15" but the commands that you ran and what version control system you're using), and maybe even attach the diff here/email it to me privately?
  • +NeedInfo
#2 david
RBTools error fixed in git as of 608bae8
#3 Nicolas*******@gmai***** (Google Code) (Is this you? Claim this profile.)

Thanks for the correction in git.

About requested information, we use the following command:

rbt post \
--server=http://<myserver>/reviews \
--publish \
--disable-proxy \
--submit-as="$AUTHOR" \
--description="$DESCRIPTION" \
--summary="$SUMMARY" \
--branch="$BRANCH" \
--testing-done="$TEST" \
--review-request-id="$RB" \
--diff-filename="SVN.diff" \
--username="<username>" \
--password="<password>" \

We use the same command for a "french" project (ISO-8859-15) and for an "english" project (UTF-8).
- For the first project, "rbt post" fails with the "UnicodeEncodeError" error.
- For the second one, it works well.

Subversion version is 1.7.16.

As requested, I attached an "ISO-8859-15" version of the diff, so you can check the content of the file.
Data as been anonymize, but it does not change the content of the file.

If you need more information, don't hesitate to ask.

Thanks and regards.
  • +
    Index: components/xxxxx/branches/x.y.z.a/xxxxx_purge_donnees_CUF.sql
    --- components/xxxxx/branches/x.y.z.a/xxxxx_purge_donnees_CUF.sql	(revision 11997)
    +++ components/xxxxx/branches/x.y.z.a/xxxxx_purge_donnees_CUF.sql	(revision 11998)
    @@ -1,9 +1,6 @@
     /* ------------------------------------------------------------------------------------ */
     /*                                                                                      */
     /* NOM PROGRAMME    : xxxxx_purge_donnees_CUF.sql 						            */
    -/* AUTEUR           : xxx                                                               */
    -/* VERSION          : xxxxxx                                                         */
    -/* DATE             : 22/05/2014                                                        */
     /* DESCRIPTION      : Suppression de donn�es pour la xxxxx                            */
     /* $Revision$                                                
#4 david
Does this particular .diff file cause the exact same error? It doesn't seem to contain the header "(r\xc3\xa9vision 0)" which is causing problems.
#5 Nicolas*******@gmai***** (Google Code) (Is this you? Claim this profile.)

No, the issue is not causing only by this diff file. We have several others which generate the same error.

As you can see, in this file, there are some "\xc3\xa9" characters in the file (not in the header). Corresponding world in the file are: données, supprimées, inutilisés.

#6 david
Does this particular .diff file produce the exact same text in the "Error data" part of the traceback?
#7 david
  • -NeedInfo