2054: RBTools-0.3.2 + Perforce - failed to upload diff when source file is a new file

fogy****@gmai***** (Google Code) (Is this you? Claim this profile.)
Aug. 23, 2013 
Version: RevewBoard 1.5.2 + RBTools-0.3.2-py2.6
Client OS: Windows XP SP3
p4v-version: Perforce Visual Client/NTX86/2009.1/205670
Review-Type: Pre-submit review

Procedure:
1. add a new file to p4 workspace
2. create a new changelist
3. post-review change-num
4. error as below:

---------------------------------------------------------------
>>> RBTools 0.3.2
>>> Home = C:\Documents and Settings\fogy_wang\Application Data
>>> svn info
>>> git rev-parse --git-dir
>>> p4 info
>>> repository info: Path: perforce.my-rb-host.org:1666, Base path: None, Supports changesets: True
>>> HTTP GETting api/
>>> HTTP GETting http://review.my-rb-host.org/api/info/
>>> Using the new web API
>>> Generating diff for changenum 748607
>>> p4 describe -s 748607
>>> Processing add of //Core/TRS/DEV/ID_Reputation/src/ARQ/common/test.cxx
>>> diff -urNp c:\docume~1\fogy_w~1\locals~1\temp\tmp_l6zi6 C:\p4\default.fogy_wang.ID_Reputation-Lead\Core\TRS\DEV\ID_Reputation\src\ARQ\common\test.cxx
>>> Using changenums on Review Board 1.5.2. Falling back to the deprecated 1.0 API
==> Review Board Login Required
Enter username and password for Review Board at http://review.my-rb-host.org/
>>> Logging in with username "fogy_wang"
>>> HTTP POSTing to http://review.my-rb-host.org/api/json/accounts/login/: {'username': 'fogy_wang', 'password': '**************'}
>>> Logged in.
>>> Attempting to create review request on perforce.my-rb-host.org:1666 for 748607
>>> HTTP POSTing to http://review.my-rb-host.org/api/json/reviewrequests/new/: {'changenum': '748607', 'repository_path': 'perforce.my-rb-host.org:1666'}
>>> Review request created
>>> Uploading diff, size: 313
>>> HTTP POSTing to http://review.my-rb-host.org/api/json/reviewrequests/449/diff/new/: {}
>>> Got API Error 105 (HTTP code 200): One or more fields had errors
>>> Error data: {u'fields': {u'path': [u'[P4#run] Warnings during command execution( "p4 files //Core/TRS/DEV/ID_Reputation/src/ARQ/common/test.cxx#1" )\n\n\t[Warning]: //Core/TRS/DEV/ID_Reputation/src/ARQ/common/test.cxx#1 - no such file(s).\n\n']}, u'stat': u'fail', u'err': {u'msg': u'One or more fields had errors', u'code': 105}, u'deprecated': {u'in_version': u'1.5'}}
Your review request still exists, but the diff is not attached.


Error uploading diff

The generated diff file was empty. This usually means no files were
modified in this change.

Try running with --output-diff and --debug for more information.
---------------------------------------------------------------


Additional Description:
This error ONLY appears when the uploaded file is a NEW file (i.e. p4 server has no previous revisions); if the uploaded file already exists on perforce server (i.e. at least one previous revision exists), the review could be successfully posted.
I can understand the warning by 'p4 file [file-repository-path]' because there isn't a file on p4 server but shouldn't RB uses the uploaded diff (i.e. size 313) instead when the file is 'new'?
I used to use 'RBTools-0.2rc2.dev_20100308-py2.6.bak' with 'Review Board 1.0.5.1' the issue didn't exist.
#1 carm*****@gmai***** (Google Code) (Is this you? Claim this profile.) 
I just saw a very similar issue - maybe the same one - where I saw this error only when the entire diff consisted of newly added EMPTY files. It could be one empty file or more than one. But if one of the files wasn't empty things seemed to work correctly.
#2 benb****@gmai***** (Google Code) (Is this you? Claim this profile.) 
Running Windows 7 64bit client with Perforce for SCM, RBTools 0.3.3, ReviewBoard 1.6 hosted on Xubuntu 10.04 LTS VM (an unofficial VM being used for evaluation purposes) I get a similar error when running post-review on a changelist that is mostly new files.

Output of post-review --debug 206994

>>> RBTools 0.3.3
>>> Home = C:\Users\ben\AppData\Roaming
>>> p4 info
>>> repository info: Path: lenel-sourceold.windows.lenel.com:1666, Base path: None, Supports changesets: True
>>> p4 counters
>>> HTTP GETting api/
>>> HTTP GETting http://10.112.8.55/api/info/
>>> Using the new web API
>>> Generating diff for changenum 206994
>>> p4 describe -s 206994
>>> Processing edit of //Development/Path/To/My/src/Solution.sln
>>> Writing "//Development/Path/To/My/src/Solution.sln#10" to "c:\users\ben\appdata\local\temp\tmp5eiypf"
>>> p4 print -o c:\users\ben\appdata\local\temp\tmp5eiypf -q //Development/Path/To/My/src/Solution.sln#10
>>> diff -urNp c:\users\ben\appdata\local\temp\tmp5eiypf D:/Development\Path\To\My\src\Solution.sln
>>> Processing add of //Development/OnGuard/Branches/Titan/DotNet/src/TwoFlower/app.config
>>> diff -urNp c:\users\ben\appdata\local\temp\tmporoe81 D:/Development\Path\To\My\src\TwoFlower\app.config
>>> Processing add of //Development/Path/To/My/src/TwoFlower/PrincipalExt.cs
>>> diff -urNp c:\users\ben\appdata\local\temp\tmporoe81 D:/Development\Path\To\My\src\TwoFlower\Principal.cs
>>> Processing add of //Development/Path/To/My/src/TwoFlower/Program.cs
>>> diff -urNp c:\users\ben\appdata\local\temp\tmporoe81 D:/Development\Path\To\My\src\TwoFlower\Program.cs
>>> Processing add of //Development/Path/To/My/src/TwoFlower/Properties/AssemblyInfo.cs
>>> diff -urNp c:\users\ben\appdata\local\temp\tmporoe81 D:/Development\Path\To\My\src\TwoFlower\Properties\AssemblyInfo.cs
>>> Processing add of //Development/Path/To/My/src/TwoFlower/TwoFlower.csproj
>>> diff -urNp c:\users\ben\appdata\local\temp\tmporoe81 D:/Development\Path\To\My\src\TwoFlower\TwoFlower.csproj
>>> Processing add of //Development/Path/To/My/src/TwoFlower/WrappedServer.cs
>>> diff -urNp c:\users\ben\appdata\local\temp\tmporoe81 D:/Development\Path\To\My\src\TwoFlower\WrappedServer.cs
>>> Attempting to create review request on lenel-sourceold.windows.lenel.com:1666 for 206994
>>> HTTP POSTing to http://10.112.8.55/api/review-requests/: {'changenum': '206994', 'repository': 'lenel-sourceold.windows.lenel.com:1666'}
>>> Got API Error 204 (HTTP code 409): The change number specified has already been used
>>> Error data: {u'stat': u'fail', u'review_request': {u'status': u'pending', u'last_updated': u'2011-09-13 17:28:43', u'description': u'Test rig for using two embedded application servers\n', u'links': {u'diffs': {u'href': u'http://10.112.8.55/api/review-requests/5/diffs/', u'method': u'GET'}, u'repository': {u'href': u'http://10.112.8.55/api/repositories/2/', u'method': u'GET', u'title': u'Old OnGuard Repository'}, u'changes': {u'href': u'http://10.112.8.55/api/review-requests/5/changes/', u'method': u'GET'}, u'self': {u'href': u'http://10.112.8.55/api/review-requests/5/', u'method': u'GET'}, u'update': {u'href': u'http://10.112.8.55/api/review-requests/5/', u'method': u'PUT'}, u'last_update': {u'href': u'http://10.112.8.55/api/review-requests/5/last-update/', u'method': u'GET'}, u'reviews': {u'href': u'http://10.112.8.55/api/review-requests/5/reviews/', u'method': u'GET'}, u'draft': {u'href': u'http://10.112.8.55/api/review-requests/5/draft/',u'method': u'GET'}, u'file_attachments': {u'href': u'http://10.112.8.55/api/review-requests/5/file-attachments/', u'method': u'GET'}, u'submitter': {u'href': u'http://10.112.8.55/api/users/ben/', u'method': u'GET', u'title': u'ben'},u'screenshots': {u'href': u'http://10.112.8.55/api/review-requests/5/screenshots/', u'method': u'GET'}, u'delete': {u'href': u'http://10.112.8.55/api/review-requests/5/', u'method': u'DELETE'}}, u'target_groups': [], u'bugs_closed': [], u'changenum': 206994, u'target_people': [], u'testing_done': u'', u'branch': u'', u'id': 5, u'time_added': u'2011-09-13 17:28:43', u'summary': u'Test rig for using two embedded application servers', u'public': False}, u'err': {u'msg': u'The change number specified has already been used', u'code': 204}}
>>> Review request already exists. Updating it...
>>> HTTP PUTting to http://10.112.8.55/api/review-requests/5/: {'changenum': 206994}
>>> Uploading diff, size: 16111
>>> HTTP POSTing to http://10.112.8.55/api/review-requests/5/diffs/: {}
Traceback (most recent call last):
  File "c:\Python27\Scripts\post-review-script.py", line 8, in <module>
    load_entry_point('rbtools==0.3.3', 'console_scripts', 'post-review')()
  File "c:\Python27\lib\site-packages\rbtools-0.3.3-py2.7.egg\rbtools\postreview.py", line 4004, in main
  File "c:\Python27\lib\site-packages\rbtools-0.3.3-py2.7.egg\rbtools\postreview.py", line 3663, in tempt_fate
  File "c:\Python27\lib\site-packages\rbtools-0.3.3-py2.7.egg\rbtools\postreview.py", line 852, in upload_diff
  File "c:\Python27\lib\site-packages\rbtools-0.3.3-py2.7.egg\rbtools\postreview.py", line 1065, in api_post
  File "c:\Python27\lib\site-packages\rbtools-0.3.3-py2.7.egg\rbtools\postreview.py", line 986, in http_post
  File "c:\Python27\lib\urllib2.py", line 126, in urlopen
    return _opener.open(url, data, timeout)
  File "c:\Python27\lib\urllib2.py", line 394, in open
    response = self._open(req, data)
  File "c:\Python27\lib\urllib2.py", line 412, in _open
    '_open', req)
  File "c:\Python27\lib\urllib2.py", line 372, in _call_chain
    result = func(*args)
  File "c:\Python27\lib\urllib2.py", line 1199, in http_open
    return self.do_open(httplib.HTTPConnection, req)
  File "c:\Python27\lib\urllib2.py", line 1168, in do_open
    h.request(req.get_method(), req.get_selector(), req.data, headers)
  File "c:\Python27\lib\httplib.py", line 955, in request
    self._send_request(method, url, body, headers)
  File "c:\Python27\lib\httplib.py", line 989, in _send_request
    self.endheaders(body)
  File "c:\Python27\lib\httplib.py", line 951, in endheaders
    self._send_output(message_body)
  File "c:\Python27\lib\httplib.py", line 809, in _send_output
    msg += message_body
UnicodeDecodeError: 'ascii' codec can't decode byte 0xef in position 2843: ordinal not in range(128)
#3 roas*****@gmai***** (Google Code) (Is this you? Claim this profile.) 
Referring to the bug description above by benbelly@gmail.com:

File "c:\Python27\lib\httplib.py", line 809, in _send_output
    msg += message_body

The problem was that one string is ASCII and the other is Unicode.
The solution is to create a sitecustomize.py file that says:

    #!/usr/local/bin/python
    # -*- coding: UTF-8 -*-

    import sys
    sys.setdefaultencoding('utf-8')

and store the file at %PYTHONHOME%/Lib/site-packages.
This step says that all strings should be treated as Unicode.
The bug is then fixed.
david
#4 david
  • +Component-RBTools
david
#5 david 
Both the original bug and the unicode misconfiguration haven't been problems in quite some time.
  • +Fixed
#6 3080*****@qq.*** (Google Code) (Is this you? Claim this profile.) 
Thank you for #3 roasti...@gmail.com.
In Ubuntu ,the path is /usr/lib/python27
And if your os language is Chinese,you should
sys.setdefaultencoding('GB18030')

Here is my error msg:

File "/usr/local/lib/python2.7/dist-packages/ReviewBoard-1.7.25-py2.7.egg/reviewboard/diffviewer/managers.py", line 200, in _process_files
    filename = os.path.join(basedir, f2).replace("\\", "/")
File "/usr/lib/python2.7/posixpath.py", line 71, in join
    path += '/' + b
#7 ranja*****@gmai***** (Google Code) (Is this you? Claim this profile.) 
Thank you #3 roasti...@gmail.com , was scratching my head for so long, you saved my day :)