1541: Multibyte characters in diff fragments are broken with syntax highlighting.

KoRoN.*******@gmai***** (Google Code) (Is this you? Claim this profile.)
chipx86
chipx86
June 14, 2010
I use reviewboard 1.5 beta1.  I manage some repositries which include many
Japanese multibyte characters.  Those characters in diff fragments view are
broken because of Pygment (syntax highlighting library) doesn't know its
input encoding.

I create a patch to fix this problem.  Please check it.
--- reviewboard/diffviewer/diffutils.py.orig	2010-03-11 16:15:21.000000000 +0900
+++ reviewboard/diffviewer/diffutils.py	2010-03-11 16:15:24.000000000 +0900
@@ -477,7 +477,7 @@
         # XXX Guessing is preferable but really slow, especially on XML
         #     files.
         #if filename.endswith(".xml"):
-        lexer = get_lexer_for_filename(filename, stripnl=False)
+        lexer = get_lexer_for_filename(filename, stripnl=False, encoding='utf-8')
         #else:
         #    lexer = guess_lexer_for_filename(filename, data, stripnl=False)
 
chipx86
#1 chipx86
Can you submit this to http://reviews.reviewboard.org/? All patches must go through
our standard code review process.
  • +Confirmed
  • +Component-DiffViewer
    +Milestone-Release1.5
#2 KoRoN.*******@gmai***** (Google Code) (Is this you? Claim this profile.)
I submit a review as http://reviews.reviewboard.org/r/1639/.  But the diff can't be
seen on reviewboard system.  What happen about it?  Is it another bug?
chipx86
#3 chipx86
How did you post it? Did you use post-review?
#4 KoRoN.*******@gmai***** (Google Code) (Is this you? Claim this profile.)
No I didn't.  I had made it by "git diff HEAD".
chipx86
#5 chipx86
Try using post-review. Most likely, it's broken due to your diff using a short SHA1. Long SHA1s are required. post-review will do this for you, or you can pass --full-index to git diff.

There's a bug filed for providing a better error message for this case.
#6 KoRoN.*******@gmai***** (Google Code) (Is this you? Claim this profile.)
Thank you chipx86!

I could post a new diff file which be generated with '--ful-index',
and it seems good in diff viewer.
chipx86
#7 chipx86
Committed to master (2720c7d)
  • -Confirmed
    +Fixed
  • +chipx86