4888: reviewbot returns "internal error" code review for empty files

almt
chipx86
 chipx86 
What version of Review Board are you running?


3.0.18


What version of Review Bot are you running?


1.0.1


What tool(s) is Review Bot running?


flake8


What steps will reproduce the problem?


    

  1. Create a code review with an empty init.py
    2. 

  2. 

    

    

    3. 

  3. 

    

    

    4. 





What is the expected output? What do you see instead?


Bot should run cleanly. Instead, the code review returns something like "flake8 internal error"


Please provide any additional information below.


The main issue is in reviewboard/diffviewer/diffutils:patch. This call returns a failure=1 when there's an empty file


    with controlled_subprocess('patch', process) as p:
        stdout, stderr = p.communicate(diff)
        failure = p.returncode



A Q&D workaround is for empty files, skip this block of code and copy the file from old to new
david
#1 david 
Is there an actual error printed out from the reviewbot worker? You may have to run the worker manually so you can see any console output.
  • -New
    +NeedInfo
#2 almt 
An example where I created a tools/__init__.py empty file


On the worker side:




[2020-09-30 16:00:56,316: ERROR/Worker-2] reviewbot.tasks.RunTool: There was an error fetching a source file. (HTTP 500, API Error 221) (server=https://reviews.com/, review_request_id=110143, diff_revision=1)
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/dist-packages/rbtools/api/request.py", line 685, in make_request
    request.url, body, headers, request.method))
  File "/usr/local/lib/python3.6/dist-packages/rbtools/api/cache.py", line 270, in make_request
    response = HTTPResponse(self.urlopen(request))
  File "/usr/lib/python3.6/urllib/request.py", line 223, in urlopen
    return opener.open(url, data, timeout)
  File "/usr/lib/python3.6/urllib/request.py", line 532, in open
    response = meth(req, response)
  File "/usr/local/lib/python3.6/dist-packages/rbtools/api/request.py", line 240, in http_response
    response.info())
  File "/usr/lib/python3.6/urllib/request.py", line 570, in error
    return self._call_chain(*args)
  File "/usr/lib/python3.6/urllib/request.py", line 504, in _call_chain
    result = func(*args)
  File "/usr/lib/python3.6/urllib/request.py", line 650, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 500: INTERNAL SERVER ERROR

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.6/dist-packages/reviewbot/tasks.py", line 185, in RunTool
    base_commit_id=base_commit_id)
  File "/usr/local/lib/python3.6/dist-packages/reviewbot/tools/__init__.py", line 62, in execute
    self.handle_files(review.files, settings)
  File "/usr/local/lib/python3.6/dist-packages/reviewbot/tools/__init__.py", line 78, in handle_files
    self.handle_file(f, settings)
  File "/usr/local/lib/python3.6/dist-packages/reviewbot/tools/pylint.py", line 121, in handle_file
    path = f.get_patched_file_path()
  File "/usr/local/lib/python3.6/dist-packages/reviewbot/processing/review.py", line 89, in get_patched_file_path
    contents = self.patched_file_contents
  File "/usr/local/lib/python3.6/dist-packages/reviewbot/processing/review.py", line 60, in patched_file_contents
    patched_file = self._api_filediff.get_patched_file()
  File "/usr/local/lib/python3.6/dist-packages/rbtools/api/resource.py", line 190, in <lambda>
    self._get_url(url, **kwargs)))
  File "/usr/local/lib/python3.6/dist-packages/rbtools/api/decorators.py", line 27, in request_method
    *args, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/rbtools/api/transport/sync.py", line 77, in execute_request_method
    return self._execute_request(request)
  File "/usr/local/lib/python3.6/dist-packages/rbtools/api/transport/sync.py", line 86, in _execute_request
    rsp = self.server.make_request(request)
  File "/usr/local/lib/python3.6/dist-packages/rbtools/api/request.py", line 687, in make_request
    self.process_error(e.code, e.read())
  File "/usr/local/lib/python3.6/dist-packages/rbtools/api/request.py", line 661, in process_error
    rsp['err']['msg'])
rbtools.api.errors.APIError: There was an error fetching a source file. (HTTP 500, API Error 221)





And on the server side




2020-09-30 23:23:54,265 - ERROR - None - almt - /api/review-requests/110143/diffs/1/files/2089725/patched-file/ - root - PatchedFileResource: Error retrieving patched file for FileDiff 2089725: The patch to "tools/__init__.py" did not apply cleanly.
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/reviewboard/webapi/resources/base_patched_file.py", line 66, in get
    patched_file = get_patched_file(orig_file, filediff, request)
  File "/usr/local/lib/python2.7/dist-packages/reviewboard/diffviewer/diffutils.py", line 308, in get_patched_file
    request=request)
  File "/usr/local/lib/python2.7/dist-packages/reviewboard/diffviewer/diffutils.py", line 218, in patch
    diff, rejects)
PatchError: The patch to "tools/__init__.py" did not apply cleanly.
david
#3 david 
This one is also showing that the root cause of the worker error is that the Review Board API is returning a 500 (which it should never do). Can you look in the Review Board/Apache logs for that exception?
#4 almt 
The last section in comment #2 shows the server reviewboard.log output. The apache output is similar, I reloaded the API request.




 2020-10-01 06:30:26,563 - ERROR - None - almt - /api/review-requests/110143/diffs/1/files/2089725/patched-file/ - root - PatchedFileResource: Error retrieving patched file for FileDiff 2089725: The patch to "tools/__init__.py" did not apply cleanly.
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/reviewboard/webapi/resources/base_patched_file.py", line 66, in get
    patched_file = get_patched_file(orig_file, filediff, request)
  File "/usr/local/lib/python2.7/dist-packages/reviewboard/diffviewer/diffutils.py", line 308, in get_patched_file
    request=request)
  File "/usr/local/lib/python2.7/dist-packages/reviewboard/diffviewer/diffutils.py", line 218, in patch
    diff, rejects)
PatchError: The patch to "tools/__init__.py" did not apply cleanly.
#5 almt 
Sorry, that was still reviewboard.log. Here's the vhosts_access.log




"GET /api/review-requests/110143/diffs/1/files/2089725/patched-file/ HTTP/1.1" 500 782 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/85.0.4183.121 Safari/537.36"
david
#6 david 
How are you creating and posting your review request? Are you using RBTools? What version control system is this?
#7 almt 
Yes, I'm using rbt with git. The error is after I publish and after it starts the reviewbot
#8 almt 
I did a dump of the diff string being passed to diffutils:patch(), when it tries to call p.communicate(diff). This is the diff string:




index 0000000000000000000000000000000000000000..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
chipx86
#9 chipx86 
This should be fixed for the upcoming Review Bot 3.
  • -NeedInfo
    +Fixed
  • +chipx86