4926: Error displaying code review diff with unicode characters

almt

What version are you running?

3.0.17

What's the URL of the page containing the problem?

What steps will reproduce the problem?

Submit a code review where a line change includes a Windows "smart quote", such as

FOO = 1, //Testing Window<92>s smart quote

What is the expected output? What do you see instead?

Instead of seeing the diff, I see this error:

 There was an error displaying this diff.
'utf8' codec can't decode byte 0x92 in position 530: invalid start byte
 This may be a a temporary outage or an issue with the format of your diff.
 Please try again, and if you still have trouble,
 contact support.

What operating system are you using? What browser?

 Ubuntu 18.04

Please provide any additional information below.

Stack trace:

Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/reviewboard/diffviewer/views.py", line 345, in get
response = renderer.render_to_response(request)
File "/usr/local/lib/python2.7/dist-packages/reviewboard/diffviewer/renderers.py", line 58, in render_to_response
return HttpResponse(self.render_to_string(request))
File "/usr/local/lib/python2.7/dist-packages/reviewboard/diffviewer/renderers.py", line 76, in render_to_string
large_data=True)
File "/usr/local/lib/python2.7/dist-packages/djblets/cache/backend.py", line 299, in cache_memoize
compress_large_data))
File "/usr/local/lib/python2.7/dist-packages/djblets/cache/backend.py", line 253, in cache_memoize_iter
items = items_or_callable()
File "/usr/local/lib/python2.7/dist-packages/djblets/cache/backend.py", line 296, in <lambda>
lambda: [lookup_callable()],
File "/usr/local/lib/python2.7/dist-packages/reviewboard/diffviewer/renderers.py", line 75, in <lambda>
lambda: self.render_to_string_uncached(request),
File "/usr/local/lib/python2.7/dist-packages/reviewboard/diffviewer/renderers.py", line 89, in render_to_string_uncached
request=request)
File "/usr/local/lib/python2.7/dist-packages/reviewboard/diffviewer/diffutils.py", line 786, in populate_diff_chunks
chunks = list(generator.get_chunks())
File "/usr/local/lib/python2.7/dist-packages/reviewboard/diffviewer/chunk_generator.py", line 786, in get_chunks
for chunk in super(DiffChunkGenerator, self).get_chunks(cache_key):
File "/usr/local/lib/python2.7/dist-packages/reviewboard/diffviewer/chunk_generator.py", line 107, in get_chunks
large_data=True)
File "/usr/local/lib/python2.7/dist-packages/djblets/cache/backend.py", line 299, in cache_memoize
compress_large_data))
File "/usr/local/lib/python2.7/dist-packages/djblets/cache/backend.py", line 253, in cache_memoize_iter
items = items_or_callable()
File "/usr/local/lib/python2.7/dist-packages/djblets/cache/backend.py", line 296, in <lambda>
lambda: [lookup_callable()],
File "/usr/local/lib/python2.7/dist-packages/reviewboard/diffviewer/chunk_generator.py", line 106, in <lambda>
lambda: list(self.get_chunks_uncached()),
File "/usr/local/lib/python2.7/dist-packages/reviewboard/diffviewer/chunk_generator.py", line 793, in get_chunks_uncached
new = get_patched_file(old, self.filediff, self.request)
File "/usr/local/lib/python2.7/dist-packages/reviewboard/diffviewer/diffutils.py", line 309, in get_patched_file
request=request)
File "/usr/local/lib/python2.7/dist-packages/reviewboard/diffviewer/diffutils.py", line 185, in patch
if empty_diff(diff):
File "/usr/local/lib/python2.7/dist-packages/reviewboard/diffviewer/diffutils.py", line 126, in empty_diff
arr = diff.decode('utf8').split('\n',10)
File "/usr/lib/python2.7/encodings/utf_8.py", line 16, in decode
return codecs.utf_8_decode(input, errors, True)
UnicodeDecodeError: 'utf8' codec can't decode byte 0x92 in position 530: invalid start byte