4819: "rbt post" is broken (will not accept Perforce changelist number) in 1.0.2 under Python 3.7.3

c0d3h4x0r
chipx86
chipx86

What version are you running?

RBTools==1.0.2

What steps will reproduce the problem?

  1. Locally apply the code fix I described in ticket 4818.
  2. cd into the root of an existing Perforce workspace with a file opened for "p4 edit" under a numbered CL.
  3. rbt post <CL#>

What is the expected output? What do you see instead?

Expected: It should work.

Actual:

CRITICAL: Path 1802276 does not match a valid Perforce path.

What operating system are you using?

Windows 10 Enterprise 64-bit

Attach the debug out from the command.

>>> RBTools 1.0.2
>>> Python 3.7.3 (v3.7.3:ef4ec6ed12, Mar 25 2019, 22:22:05) [MSC v.1916 64 bit (AMD64)]
>>> Running on Windows-10-10.0.16299-SP0
>>> Home = C:\Users\kkelly\AppData\Roaming
>>> Current directory = D:\p4\depot\teams\near
>>> Command line: rbt post --debug 1802276
>>> Running: tf vc help
>>> Checking for a Bazaar repository...
>>> Unable to execute "bzr help": skipping Bazaar
>>> Checking for a ClearCase repository...
>>> Unable to execute "cleartool help": skipping ClearCase
>>> Checking for a CVS repository...
>>> Unable to execute "cvs": skipping CVS
>>> Checking for a Git repository...
>>> Running: git rev-parse --git-dir
>>> Command exited with rc 128: ['git', 'rev-parse', '--git-dir']
fatal: not a git repository (or any of the parent directories): .git
---
>>> Checking for a Mercurial repository...
>>> Unable to execute "hg --help": skipping Mercurial
>>> Checking for a Perforce repository...
>>> Running: p4 info
>>> Running: diff --version
>>> Repository info: Path: dvp4edgepl009.tsi.lan:1666, Base path: None, Supports changesets: True
>>> Checking for a Plastic repository...
>>> Unable to execute "cm version": skipping Plastic
>>> Checking for a Subversion repository...
>>> Running: svn --non-interactive info
>>> Command exited with rc 1: ['svn', '--non-interactive', 'info']
>>> Checking for a Team Foundation Server repository...
>>> Unable to execute "tf help": skipping TFS
>>> Making HTTP GET request to http://reviewboard/api/
>>> Running: p4 info
Traceback (most recent call last):
  File "c:\python37\lib\runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "c:\python37\lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "C:\Python37\Scripts\rbt.exe\__main__.py", line 9, in <module>
  File "c:\python37\lib\site-packages\rbtools\commands\main.py", line 120, in main
    command.run_from_argv([RB_MAIN, command_name] + args)
  File "c:\python37\lib\site-packages\rbtools\commands\__init__.py", line 725, in run_from_argv
    exit_code = self.main(*args) or 0
  File "c:\python37\lib\site-packages\rbtools\commands\post.py", line 806, in main
    extra_args=extra_args)
  File "c:\python37\lib\site-packages\rbtools\clients\perforce.py", line 729, in diff
    return self._path_diff(extra_args, exclude_patterns)
  File "c:\python37\lib\site-packages\rbtools\clients\perforce.py", line 1470, in _path_diff
    % path)
rbtools.clients.errors.SCMError: Path 1802276 does not match a valid Perforce path.

Please provide any additional information below.

I tried upgrading my p4.exe command-line client to the latest (Rev. P4/NTX64/2019.1/1796703 (2019/04/30)), but that did not help.

david
#1 david

I'm not able to reproduce this problem.

Do you think you could trace through parse_revision_spec in rbtools/clients/perforce.py and see what the revisions list looks like and where it's failing to parse it correctly?

  • -New
    +NeedInfo
#2 jose.ortiz

I am also getting this issue on CentOS. I am using RBTools 1.0.2, Python 3.6.8 and p4 P4/LINUX26X86_64/2019.1/1813586 (2019/06/06). I debugged the issue and found out that p4 -G is generating Python dictionaries that contain byte literals instead of regular strings. I patched RBTools to convert all of these byte literals to strings before they are used, and I managed to make it work. The patch is just a workaround, but it's attached to this comment in case it can provide any insight.

  • +
    --- /perforce.py 2019-07-17 11:44:17.642836093 +0100
    +++ /perforce.py 2019-06-20 15:10:47.788724985 +0100
    @@ -281,7 +281,10 @@
    except EOFError:
    break
    else:
    - result.append(data)
    + dataNew = {key.decode(): value.decode() for key, value in data.items()}
    + result.append(dataNew)
    if data.get('code', None) == 'error':
    has_error = True
    @@ -1799,7 +1802,10 @@
    change = self.p4.change(changelist)
    if len(change) == 1 and 'Description' in change[0]:
    - return change[0]['Description'].decode(getpreferredencoding())
    + return change[0]['Description']
    else:
    return ''
dan_regan
#3 dan_regan

I encountered this exact issue when debugging why rbt 1.0.2 wasn't working on Linux. 'Status' wasn't found in the dict because it only contained b'Status'. This was on a coworker's system, I'll see if I can reproduce it on my own system and try the patch.

dan_regan
#4 dan_regan

This patch doesn't look like it will fix my exact issue, all of the dict keys should be normal strings.

c0d3h4x0r
#5 c0d3h4x0r

This issue has been tagged "NeedInfo" for a long time now. What additional information is actually needed? You should have all the information you need to repro this and work on fixing it. This is blocking Python migration work at my company because we can't remove obsolete versions of Python from our engineering workstations due to rbt still not working correctly with the latest version of Python. Please treat this issue as ultra-high priority, especially given the impending Python 2.7.x end-of-life date rapidly approaching on 1/1/2020.

chipx86
#6 chipx86

I'm sorry, this fell through the cracks. This will be fixed in the next release of RBTools, alongside other Python 3.x fixes.

  • -NeedInfo
    +Confirmed
  • +Release-1.0.x
  • -Priority:Medium
    +Priority:High
    +SCM:Perforce
  • +chipx86
chipx86
#7 chipx86

@dan_regan, I'm interested in what issue you're seeing exactly, because the patch (well, a variation on the concept) does fix the issue here. Can you provide a full backtrace?

chipx86
#8 chipx86

(+ debug log)

chipx86
#9 chipx86

Fix is up for review: https://reviews.reviewboard.org/r/10666/

Dan, I'm also curious if you already applied the previous patch that was referenced. This is part 2 of a fix.

  • -Confirmed
    +PendingReview
dan_regan
#10 dan_regan

My coworker switched to using the Python 2.7 version instead and I haven't pursued the issue, since I am also using the Python 2.7 version.

c0d3h4x0r
#11 c0d3h4x0r

Awesome chipx86, thank you! Is there a way I can be notified when the next release (containing this fix) is officially available?

chipx86
#12 chipx86

Yep, we have a mailing list where we announce all new releases (or other important updates). It's fairly low-traffic -- we try not to waste your time. https://www.reviewboard.org/mailing-lists/

c0d3h4x0r
#13 c0d3h4x0r

My employer's "get off of Python 2.7" initiative is still blocked by the inability of the latest publicly-available version of rbtools to work under Python 3.7.3. When will a new public release finally be issued so that we can safely pick up the fix for this?

david
#14 david
  • -PendingReview
    +Fixed