1918: git diffs are not parsed correctly

owen.o******@gmai***** (Google Code) (Is this you? Claim this profile.)
chipx86
chipx86
Feb. 8, 2011
What version are you running? 1.5

What's the URL of the page containing the problem? uploading a diff


What steps will reproduce the problem?
1. create a diff from a git repository with --no-prefix
2. create a review request with the patch

What is the expected output? What do you see instead?

I expect a new request, but I get a 500 server error.

What operating system are you using? What browser?

Mac & Chrome

Please provide any additional information below.

If you create the request first and then upload the diff, you get:

"No valid separator after the filename was found in the diff header"
#1 ke***@me.*** (Google Code) (Is this you? Claim this profile.)
It worked if I touched up the diff to look exactly like a subversion diff.  I think the key was adding "(revision ####)" and "(working copy)" after the filenames.

Obviously, this is not a feasible long-term solution, because it requires way too much manual work for each diff.
chipx86
#2 chipx86
Can you please show me exactly what you use to generate the diff? Please show me your repository configuration too.

We support git diffs without modification. The only important option to pass to 'git diff' is --full-index. You may want to look into using the post-review tool, though, as it simplifies creating/updating review requests in general.
  • +NeedInfo
  • -Type-Defect
    +Type-Support
#3 ke***@me.*** (Google Code) (Is this you? Claim this profile.)
I am actually using git-svn as a client for a subversion repository.  ReviewBoard is hooked up to the svn repo.

I imagine this isn't a common scenario, so I didn't expect to post-review tool to "just work", but I did hope that generating and posting standard diffs would work a little more smoothly.

In this case, I generated a series of diffs with:
$ git format-patch master --no-prefix

These diffs apply cleanly with `patch -p0`, so I hoped that uploading them, and specifying the right path in the svn repo would work.  But, it kept giving the "No valid separator after the filename was found in the diff header" error until I added the "(revision XXXXX)" and "(working copy)" after the filenames.

Is there any other info you need?
chipx86
#4 chipx86
If you're using git-svn then you have to use post-review. It knows how to format compatible diffs.
  • -NeedInfo
    +UserError
  • +chipx86
chipx86
#5 chipx86
  • -UserError
    +NotABug
#6 ivan.ga*******@gmai***** (Google Code) (Is this you? Claim this profile.)
This is obviously a bug. ReviewBoard provides API for uploading diffs, we can't use rbt post to work around bugs in those :(
chipx86
#7 chipx86
Sorry, this is an old bug and much has changed since. You're going to have to provide more information than that. I don't really even know what you mean by the comment?
#8 ivan.ga*******@gmai***** (Google Code) (Is this you? Claim this profile.)
I'm using API to attach git diffs to the ticket. No matter how I generate git diff i'm getting the error:

{
	"fields": {
		"path": ["No valid separator after the filename was found in the diff header"]
	},
	"stat": "fail",
	"err": {
		"msg": "One or more fields had errors",
		"code": 105
	}
}

even for the simplest diff:

diff --git a/src/MimecastServicesForExchange.CoreService.Shared/Application/CalendarSync/Embed/ImageExtractor.cs b/src/MimecastServicesForExchange.CoreService.Shared/Application/CalendarSync/Embed/ImageExtractor.cs
index 5f5680c04ecd40ff1afdf620e5e401983a73ac86..8ef85df206b78ccc8259b4968209f40f774b5872 100644
--- a/src/MimecastServicesForExchange.CoreService.Shared/Application/CalendarSync/Embed/ImageExtractor.cs
+++ b/src/MimecastServicesForExchange.CoreService.Shared/Application/CalendarSync/Embed/ImageExtractor.cs
@@ -172,8 +172,6 @@ namespace Mimecast.Mse.Core.Shared.Application.CalendarSync.Embed
          }
       }
 
-      static private int jajj = 1;
-
       public CalendarEventAttachment GetEmbeddedAttachment(CalendarEventAttachment originalAttachment)
       {
          if(originalAttachment == null) throw new ArgumentNullException("originalAttachment");

I've tried all the permutations of git diff command line parameters and tried to read rbt post source code to simulate the exact same parameter set you use with no luck :( By the way it works well with SVN, the problem only can be reproduced after migrating to git so I'm confident we are using API correctly.

#9 ivan.ga*******@gmai***** (Google Code) (Is this you? Claim this profile.)
I can attach http request/response dump from Charles if that helps.
chipx86
#10 chipx86
Thanks for the info. A few more questions that would help.

If you can attach the raw diff, I can take a deeper look into it.

Can you show me the ways you generated the diff? Also, what version of git?

On the Review Board side, is this configured as a Git repository, or Subversion?

On the client side, is it git or git-svn?
#11 ivan.ga*******@gmai***** (Google Code) (Is this you? Claim this profile.)
Attaching raw diff, raw request and raw response files.

I'm generating diff based on what I found in https://github.com/reviewboard/rbtools/blob/master/rbtools/clients/git.py:

git diff --no-color --full-index --ignore-submodules --no-ext-diff "revision1" "revision2"

git version 1.9.0.msysgit.0 (running on windows vm, but i have no luck generating diff with identical command line on macosx with git version 1.8.5.2 (Apple Git-48). 

On ReviewBoard server side (in house installation) it is configured properly as GIT repository (attaching screenshot) and attaching diff manually from web interface works.

I'm not using git-svn but pure git.

Thank you!

  • +
    diff --git a/src/MimecastServicesForExchange.CoreService.Shared/Application/CalendarSync/Embed/ImageExtractor.cs b/src/MimecastServicesForExchange.CoreService.Shared/Application/CalendarSync/Embed/ImageExtractor.cs
    index 5f5680c04ecd40ff1afdf620e5e401983a73ac86..8ef85df206b78ccc8259b4968209f40f774b5872 100644
    --- a/src/MimecastServicesForExchange.CoreService.Shared/Application/CalendarSync/Embed/ImageExtractor.cs
    +++ b/src/MimecastServicesForExchange.CoreService.Shared/Application/CalendarSync/Embed/ImageExtractor.cs
    @@ -172,8 +172,6 @@ namespace Mimecast.Mse.Core.Shared.Application.CalendarSync.Embed
              }
           }
     
    -      static private int jajj = 1;
    -
           public CalendarEventAttachment GetEmbeddedAttachment(CalendarEventAttachment originalAttachment)
           {
              if(originalAttachment == null) throw new ArgumentNullException("originalAttachment");
    +
    POST /api/review-requests/3800/diffs/ HTTP/1.1
    Accept: application/json, application/xml, text/json, text/x-json, text/javascript, text/xml
    User-Agent: RestSharp 103.4.0.0
    Host: reviews.mimecast.com
    Cookie: rbsessionid=yivlr2xrukpz1wihfw6yyz6ruprsrlhs
    Content-Type: multipart/form-data; boundary=-----------------------------28947758029299
    Content-Length: 1307
    Accept-Encoding: gzip, deflate
    -------------------------------28947758029299
    Content-Disposition: form-data; name="basedir"; filename=""
    Content-Type: application/octet-stream
    ssh://igavryliuk@svn.uk.mimecast.lan/repos/git/MIME/syncengine.git
    -------------------------------28947758029299
    Content-Disposition: form-data; name="path"; filename="diff"
    Content-Type: application/octet-stream
    diff --git a/src/MimecastServicesForExchange.CoreService.Shared/Application/CalendarSync/Embed/ImageExtractor.cs b/src/MimecastServicesForExchange.CoreService.Shared/Application/CalendarSync/Embed/ImageExtractor.cs
    index 5f5680c04ecd40ff1afdf620e5
    +
    HTTP/1.1 400 BAD REQUEST
    Date: Mon, 16 Jun 2014 09:28:10 GMT
    Server: Apache/2.2.22 (Ubuntu)
    Content-Language: en
    X-Content-Type-Options: nosniff
    Expires: Mon, 16 Jun 2014 09:28:10 GMT
    Vary: Accept,Cookie,Accept-Language,Accept-Encoding
    Last-Modified: Mon, 16 Jun 2014 09:28:10 GMT
    Cache-Control: max-age=0
    X-Frame-Options: SAMEORIGIN
    Content-Encoding: gzip
    Connection: close
    Transfer-Encoding: chunked
    Content-Type: text/plain
    {"fields": {"path": ["No valid separator after the filename was found in the diff header"]}, "stat": "fail", "err": {"msg": "One or more fields had errors", "code": 105}}
    +
chipx86
#12 chipx86
What version of Review Board?

I just fed that diff into my instance, and it had no problem parsing it.
#13 ivan.ga*******@gmai***** (Google Code) (Is this you? Claim this profile.)
I'm using 2.0.2
#14 ivan.ga*******@gmai***** (Google Code) (Is this you? Claim this profile.)
It's works when feeding diff manually with web interface but not with API call, maybe the problem is there?
chipx86
#15 chipx86
The web UI actually uses the API call to do the upload. It must have to do with how the request's payload is being formatted.

Though this shouldn't matter in practice, you don't need to send the basedir field unless it's a SVN repository.

Here's where things are confusing.. The error you're getting is from a code path that should never execute for Git, as we use a custom diff parser when posting against Git repositories.

Just to sanity-check, is review request 3800 definitely using the Git repository?
#16 ivan.ga*******@gmai***** (Google Code) (Is this you? Claim this profile.)
You were right, ticket 3800 is created for SVN repository. After we've migrated svn to git the .reviewboardrc setting still pointed to SVN repository name. After changing repository name in .reviewboardrc everything works well. Sorry for wasting your time.
chipx86
#17 chipx86
Not a problem. Glad we got that figured out!

It's nearly 4AM here, so I'm crashing. Have a good day!