3913: Update to RBTools 0.7.4, scripts to interface with reviewboard now error.

jsto****@gmai***** (Google Code) (Is this you? Claim this profile.)
July 15, 2015 
What version are you running?
RBTools 0.7.4 (CentOS package RBTools-0.7.4-1.el7.noarch)

What's the URL of the page containing the problem?
N/A.

What steps will reproduce the problem?
1. post a review to reviewboard explicitly indicating --username and --password.
2.
3.

What is the expected output? What do you see instead?
I expected the post to execute error free so no output. Instead I see:
ERROR: Unexpected API Error: You are not logged in (HTTP 401, API Error 103)



What operating system are you using? What browser?
CentOS for RBTools, Windows/Centos with Mozilla or IE for dashboard access.

Please provide any additional information below.

I posted this information on the reviewboard support fourm:

https://groups.google.com/forum/#!topic/reviewboard/B5SJadJr7WM

Here are the details again:

Hello,

I have been using Review Board version 1.7.16 with great success. I had been using Centos 7 with RBTools 0.7.2 (package RBTools-0.7.2-1.el7.noarch).

I recently updated the Centos 7 install and RBTool was updated to version 0.7.4 (package RBTools-0.7.4-1.el7.noarch).

I use a canned script which posts reviews to a remote server running reviewboard 1.7.16.

The error that I am getting is:

ERROR: Unexpected API Error: You are not logged in (HTTP 401, API Error 103)

The problem with this message is that I know that I am logged in.

This is the script that I run on 0.7.2 which has worked flawlessly. Unfortunately I can't share it all but this is the snippet that runs the rbt stuff.

#--------------------------------------------------------
# Use RBTools if available
if (system(qq(rbt --version 2>/dev/null)) == 0)
{
    # Posting Options:
    my $rbt_options = qq( --publish) unless ( $draft );

    # Review Board Server Options:
    $rbt_options   .= qq( --server "https://{URL Removed, sorry}");
    $rbt_options   .= qq( --username "$username");
    $rbt_options   .= qq( --password "$password");

    # Repository Options:
    $rbt_options   .= qq( --repository "$repository");

    # Review Request Field Options:
    $rbt_options   .= qq( --summary "$title");
    $rbt_options   .= qq( --description "$description") unless (-f "$description");
    $rbt_options   .= qq( --description-file "$description") if (-f "$description");
    $rbt_options   .= qq( --testing-done "$testing") unless (-f "$testing");
    $rbt_options   .= qq( --testing-done-file "$testing") if (-f "$testing");
    $rbt_options   .= qq( --branch "$branch");
    $rbt_options   .= qq( --target-groups "$reviewer_groups");
    $rbt_options   .= qq( --target-people "$reviewers");

    # Diff Generation Options:
    $rbt_options   .= qq( --diff-filename "$diffFile");

    # Create a new review request
    my $output = qx(rbt post $rbt_options);
    my $result = ($? >> 8);
    if ($result == 0)
    {
        (my $url) = $output =~ /(.*\/diff\/)/;
        print "\nYou can watch your review at ".$url."\n\n";

        system("rm -f $diffFile $stdErr");
    }

    exit $result;
}

If I run the rbt command manually with debug on I see the same issue so I know that the script isn't the problem.

~/ws/myproject/code > /usr/bin/rbt post --debug --server "https://review.my.server.com" --username "jstobern" --password "<removed>" --repository "<removed>" --summary "Test cmdline" --branch HEAD --diff-filename review.diff
>>> RBTools 0.7.4
>>> Python 2.7.5 (default, Jun 24 2015, 00:41:19)
[GCC 4.8.3 20140911 (Red Hat 4.8.3-9)]
>>> Running on Linux-3.10.0-229.7.2.el7.x86_64-x86_64-with-centos-7.1.1503-Core
>>> Home = /home/jstobern
>>> Current directory = /home/jstobern/ws/myproject/code
>>> Running: git version
>>> Checking for a Subversion repository...
>>> Running: svn --non-interactive info
>>> Command exited with rc 1: ['svn', '--non-interactive', u'info']
svn: E155007: '/home/jstobern/ws/myproject/code' is not a working copy
---
>>> Checking for a Git repository...
>>> Running: git rev-parse --git-dir
>>> Command exited with rc 128: ['git', 'rev-parse', '--git-dir']
fatal: Not a git repository (or any parent up to mount point /home)
Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).
---
>>> Checking for a Mercurial repository...
>>> Unable to execute "hg --help": skipping Mercurial
>>> Checking for a CVS repository...
>>> repository info: Path: cvspc.my.server.com:/swdev/cvsrep, Base path: None, Supports changesets: False
>>> Making HTTP GET request to https://review.my.server.com/api/
>>> Got API Error 103 (HTTP code 401): You are not logged in
>>> Error data: {u'stat': u'fail', u'err': {u'msg': u'You are not logged in', u'code': 103}}
Traceback (most recent call last):
  File "/usr/bin/rbt", line 9, in <module>
    load_entry_point('RBTools==0.7.4', 'console_scripts', 'rbt')()
  File "/usr/lib/python2.7/site-packages/rbtools/commands/main.py", line 133, in main
    command.run_from_argv([RB_MAIN, command_name] + args)
  File "/usr/lib/python2.7/site-packages/rbtools/commands/__init__.py", line 612, in run_from_argv
    exit_code = self.main(*args) or 0
  File "/usr/lib/python2.7/site-packages/rbtools/commands/post.py", line 643, in main
    api_client, api_root = self.get_api(server_url)
  File "/usr/lib/python2.7/site-packages/rbtools/commands/__init__.py", line 768, in get_api
    raise CommandError('Unexpected API Error: %s' % e)
rbtools.commands.CommandError: Unexpected API Error: You are not logged in (HTTP 401, API Error 103)
~/ws/myproject/code >

I noticed that if I REMOVE the --username option. The rbt command (and hence the script) works correctly. I'm assuming that is because it falls back to using the data in the .rbtools-cookies. This is not how I intended this script to work because the username and password are derived by my script that then passed to rbt via --username and --password.

Is it possible to get the functionality to the way it worked in 0.7.2? Passwords change often in our environment so the ability to use the script to derive the password from other system files was very convenient.

I am wondering if the updates to Bug 3858 might have created this change in behaviour.

Regards,
Jens.
brennie
#1 brennie 
Hi, I'm having trouble reproducing your error with Review Board 1.7.16 / master and RBTools 0.7.4.

I've tried the following to reproduce:

1. Remove ~/.rbtools-cookies and run `rbt status --username username --password password`.
2. Remove ~/.rbtools-cookies and run `rbt status --username username`
3. Change the session ID in ~/.rbtools-cookies and run `rbt status --username username --password password`.
4. Change the session ID in ~/.rbtools-cookies and run `rbt status --username username`.

All of these were successful (provided the username and password combination is correct). I've tried `rbt post` and it seems to work as well. Can you verify that the username:password being used is correct or provide more reproduction steps?

Regards,
Barret
#2 jsto****@gmai***** (Google Code) (Is this you? Claim this profile.) 
Hi Barret,

Thank you for looking into the issue.

I validated the username:password by logging out of the WebGUI and then logged back in with the username:password which I used in the "rbt" command line. The log
in was successful and I could see my dashboard. I'm going to assume that this is a valid test of the username and password.

I performed the same steps as you and here are the results. Based on the results, if I manually enter the password, the action is performed successfully, but if I specify the password on the command line is fails.

Has something changed in how the --password option parses the argument passed in on the command line?

1. Remove ~/.rbtools-cookies and run `rbt status --username username --password password`.  *** FAILED

~/ws/myproject/code > rbt status --debug --server "https://review.my.server.com/" --repository "myRepo" --username "jstobern" --password "mypassword"
>>> RBTools 0.7.4
>>> Python 2.7.5 (default, Jun 24 2015, 00:41:19)
[GCC 4.8.3 20140911 (Red Hat 4.8.3-9)]
>>> Running on Linux-3.10.0-229.7.2.el7.x86_64-x86_64-with-centos-7.1.1503-Core
>>> Home = /home/jstobern
>>> Current directory = /home/jstobern/ws/myproject/code
>>> Running: git version
>>> Checking for a Subversion repository...
>>> Running: svn --non-interactive info
>>> Command exited with rc 1: ['svn', '--non-interactive', u'info']
svn: E155007: '/home/jstobern/ws/myproject/code' is not a working copy
---
>>> Checking for a Git repository...
>>> Running: git rev-parse --git-dir
>>> Command exited with rc 128: ['git', 'rev-parse', '--git-dir']
fatal: Not a git repository (or any parent up to mount point /home)
Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).
---
>>> Checking for a Mercurial repository...
>>> Unable to execute "hg --help": skipping Mercurial
>>> Checking for a CVS repository...
>>> repository info: Path: cvspc.my.server.com:/swdev/cvsrep, Base path: None, Supports changesets: False
>>> Making HTTP GET request to https://review.my.server.com/api/
>>> Got API Error 103 (HTTP code 401): You are not logged in
>>> Error data: {u'stat': u'fail', u'err': {u'msg': u'You are not logged in', u'code': 103}}
Traceback (most recent call last):
  File "/usr/bin/rbt", line 9, in <module>
    load_entry_point('RBTools==0.7.4', 'console_scripts', 'rbt')()
  File "/usr/lib/python2.7/site-packages/rbtools/commands/main.py", line 133, in main
    command.run_from_argv([RB_MAIN, command_name] + args)
  File "/usr/lib/python2.7/site-packages/rbtools/commands/__init__.py", line 612, in run_from_argv
    exit_code = self.main(*args) or 0
  File "/usr/lib/python2.7/site-packages/rbtools/commands/status.py", line 39, in main
    api_client, api_root = self.get_api(server_url)
  File "/usr/lib/python2.7/site-packages/rbtools/commands/__init__.py", line 768, in get_api
    raise CommandError('Unexpected API Error: %s' % e)
rbtools.commands.CommandError: Unexpected API Error: You are not logged in (HTTP 401, API Error 103)

2. Remove ~/.rbtools-cookies and run `rbt status --username username`  *** PASSED

~/ws/myproject/code > rm -f /home/jstobern/.rbtools-cookies                                                               
~/ws/myproject/code > rbt status --debug --server "https://review.my.server.com/" --repository "myRepo" --username "jstobern"
>>> RBTools 0.7.4
>>> Python 2.7.5 (default, Jun 24 2015, 00:41:19)
[GCC 4.8.3 20140911 (Red Hat 4.8.3-9)]
>>> Running on Linux-3.10.0-229.7.2.el7.x86_64-x86_64-with-centos-7.1.1503-Core
>>> Home = /home/jstobern
>>> Current directory = /home/jstobern/ws/myproject/code
>>> Running: git version
>>> Checking for a Subversion repository...
>>> Running: svn --non-interactive info
>>> Command exited with rc 1: ['svn', '--non-interactive', u'info']
svn: E155007: '/home/jstobern/ws/myproject/code' is not a working copy
---
>>> Checking for a Git repository...
>>> Running: git rev-parse --git-dir
>>> Command exited with rc 128: ['git', 'rev-parse', '--git-dir']
fatal: Not a git repository (or any parent up to mount point /home)
Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).
---
>>> Checking for a Mercurial repository...
>>> Unable to execute "hg --help": skipping Mercurial
>>> Checking for a CVS repository...
>>> repository info: Path: cvspc.my.server.com:/swdev/cvsrep, Base path: None, Supports changesets: False
>>> Making HTTP GET request to https://review.my.server.com/api/

Please log in to the Review Board server at review.my.server.com.
Password:
>>> Making HTTP GET request to https://review.my.server.com/api/info/
>>> Making HTTP GET request to https://review.my.server.com/api/session/?expand=user
>>> Making HTTP GET request to https://review.my.server.com/api/repositories/?only-links=&only-fields=id%2Cname%2Cmirror_path%2Cpath
>>> Making HTTP GET request to https://review.my.server.com/api/review-requests/?status=pending&expand=draft&repository=14&from-user=jstobern

<snip ... lots of output>

>>> Making HTTP GET request to https://review.my.server.com/api/review-requests/?max-results=25&start=25&status=pending&from-user=jstobern&repository=14&expand=draft

<snip ... lots of output>

~/ws/myproject/code >

3. Change the session ID in ~/.rbtools-cookies and run `rbt status --username username --password password`.  *** FAILED

~/ws/myproject/code > vi /home/jstobern/.rbtools-cookies
~/ws/myproject/code > rbt status --debug --server "https://review.my.server.com/" --repository "myRepo" --username "jstobern" --password "mypassword"
>>> RBTools 0.7.4
>>> Python 2.7.5 (default, Jun 24 2015, 00:41:19)
[GCC 4.8.3 20140911 (Red Hat 4.8.3-9)]
>>> Running on Linux-3.10.0-229.7.2.el7.x86_64-x86_64-with-centos-7.1.1503-Core
>>> Home = /home/jstobern
>>> Current directory = /home/jstobern/ws/myproject/code
>>> Running: git version
>>> Checking for a Subversion repository...
>>> Running: svn --non-interactive info
>>> Command exited with rc 1: ['svn', '--non-interactive', u'info']
svn: E155007: '/home/jstobern/ws/myproject/code' is not a working copy
---
>>> Checking for a Git repository...
>>> Running: git rev-parse --git-dir
>>> Command exited with rc 128: ['git', 'rev-parse', '--git-dir']
fatal: Not a git repository (or any parent up to mount point /home)
Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).
---
>>> Checking for a Mercurial repository...
>>> Unable to execute "hg --help": skipping Mercurial
>>> Checking for a CVS repository...
>>> repository info: Path: cvspc.my.server.com:/swdev/cvsrep, Base path: None, Supports changesets: False
>>> Making HTTP GET request to https://review.my.server.com/api/
>>> Got API Error 103 (HTTP code 401): You are not logged in
>>> Error data: {u'stat': u'fail', u'err': {u'msg': u'You are not logged in', u'code': 103}}
Traceback (most recent call last):
  File "/usr/bin/rbt", line 9, in <module>
    load_entry_point('RBTools==0.7.4', 'console_scripts', 'rbt')()
  File "/usr/lib/python2.7/site-packages/rbtools/commands/main.py", line 133, in main
    command.run_from_argv([RB_MAIN, command_name] + args)
  File "/usr/lib/python2.7/site-packages/rbtools/commands/__init__.py", line 612, in run_from_argv
    exit_code = self.main(*args) or 0
  File "/usr/lib/python2.7/site-packages/rbtools/commands/status.py", line 39, in main
    api_client, api_root = self.get_api(server_url)
  File "/usr/lib/python2.7/site-packages/rbtools/commands/__init__.py", line 768, in get_api
    raise CommandError('Unexpected API Error: %s' % e)
rbtools.commands.CommandError: Unexpected API Error: You are not logged in (HTTP 401, API Error 103)
~/ws/myproject/code >

4. Change the session ID in ~/.rbtools-cookies and run `rbt status --username username`.  ***PASSED

~/ws/myproject/code > vi /home/jstobern/.rbtools-cookies
~/ws/myproject/code > rbt status --debug --server "https://review.my.server.com/" --repository "myRepo" --username "jstobern"
>>> RBTools 0.7.4
>>> Python 2.7.5 (default, Jun 24 2015, 00:41:19)
[GCC 4.8.3 20140911 (Red Hat 4.8.3-9)]
>>> Running on Linux-3.10.0-229.7.2.el7.x86_64-x86_64-with-centos-7.1.1503-Core
>>> Home = /home/jstobern
>>> Current directory = /home/jstobern/ws/myproject/code
>>> Running: git version
>>> Checking for a Subversion repository...
>>> Running: svn --non-interactive info
>>> Command exited with rc 1: ['svn', '--non-interactive', u'info']
svn: E155007: '/home/jstobern/ws/myproject/code' is not a working copy
---
>>> Checking for a Git repository...
>>> Running: git rev-parse --git-dir
>>> Command exited with rc 128: ['git', 'rev-parse', '--git-dir']
fatal: Not a git repository (or any parent up to mount point /home)
Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).
---
>>> Checking for a Mercurial repository...
>>> Unable to execute "hg --help": skipping Mercurial
>>> Checking for a CVS repository...
>>> repository info: Path: cvspc.my.server.com:/swdev/cvsrep, Base path: None, Supports changesets: False
>>> Making HTTP GET request to https://review.my.server.com/api/

Please log in to the Review Board server at review.my.server.com.
Password:
>>> Making HTTP GET request to https://review.my.server.com/api/info/
>>> Making HTTP GET request to https://review.my.server.com/api/session/?expand=user
>>> Making HTTP GET request to https://review.my.server.com/api/repositories/?only-links=&only-fields=id%2Cname%2Cmirror_path%2Cpath
>>> Making HTTP GET request to https://review.my.server.com/api/review-requests/?status=pending&expand=draft&repository=14&from-user=jstobern

<snip ... lots of output>

>>> Making HTTP GET request to https://review.my.server.com/api/review-requests/?max-results=25&start=25&status=pending&from-user=jstobern&repository=14&expand=draft

<snip ... lots of output>

~/ws/myproject/code >
brennie
#3 brennie 
Thanks for replying.

I'll try again to reproduce this.
#4 jsto****@gmai***** (Google Code) (Is this you? Claim this profile.) 
Hi Barret,

Do you know if the characters that make up the password passed in on the command-line have restrictions? My password consists of letter, numbers and special characters like ! and $. This wasn't an issue when using RBTools 0.7.2 but maybe something has changed.

Regards,
Jens.
brennie
#5 brennie 
There should be no restrictions. If you're quoting the password, it should be sent correctly, even with special characters.
#6 jsto****@gmai***** (Google Code) (Is this you? Claim this profile.) 
Hi Barret,

I have determined what the issue is. The password which is passed to rbt in the --password argument looks something like this mypswd$!123. If I echo that string in bash then result is mypswd123 because we are using double quotes. If I change the echo command to use single quotes, I get to correct output of mypswd$!123. The fix for me is in the perl snippet that I have pasted here. 

I changed:     $rbt_options   .= qq( --password "$password");
to     $rbt_options   .= qq( --password '$password');

And everything started to work. 

Sorry for inconveniencing you with this issue, user error again!

Regards,
Jens.
#7 jsto****@gmai***** (Google Code) (Is this you? Claim this profile.) 
This point is also valid for the command-line interface. Once I used single quotes around the --password argument it worked as expected.
david
#8 david
  • +UserError