2879: support for event streams

matthew********@kitwa****** (Google Code) (Is this you? Claim this profile.) 
What version are you running?
RB 1.7.1

Describe the enhancement and the motivation for it.
It would be really great if RB supported event streams a la http://gerrit.googlecode.com/svn/documentation/2.1.2/cmd-stream-events.html (especially for porting third-party tools - e.g. auto-builders - using gerrit event streams to RB).

Please provide any additional information below.
Obviously this is a candidate for an extension, but would be a neat place to start on a pool of 'official' extensions.

The 'tricky' part is what to use for IPC, since the RB server needs to be able to throw events 'over the wall' to any listeners that may (or may not) exist at the time. With Linux now being the only supported platform, one possibility is to use the dbus system bus that exists on most modern Linux systems. To that end, in the hope it may be useful, I've attached some initial experiments I did in that direction. The client (dbus-cli.py) is more or less complete, except would obviously need to be named differently. The server component would optimally be integrated into the extension (so, basically take the dbus-srv.py code and meld it onto something based on RBWebHooks, with a more complete signal set). The dbus config to allow use of the system bus would be a template that needs to be modified by the site installer to have the appropriate context for the RB server, based on how the RB server is set up.
dbus-srv.py 
import gobject
import dbus
import dbus.service
from dbus.mainloop.glib import DBusGMainLoop
def handler(sender = None):
    print "got signal from %r" % sender
class EventStreamObject(dbus.service.Object):
    def __init__(self):
        bus = dbus.SystemBus(mainloop = DBusGMainLoop())
        bus_name = dbus.service.BusName('org.reviewboard.EventStream', bus = bus)
        dbus.service.Object.__init__(self, bus_name, '/org/reviewboard/EventStream')
    @dbus.service.method('org.reviewboard.EventStream', in_signature = 's', out_signature = '')
    def notify(self, input):
        print "method 'notify' invoked with input '%s'" % input
        self.event(input)
    @dbus.service.signal('org.reviewboard.EventStream', signature = 's')
    def event(self, input):
        pass
    def run(self):
        loop = gobject.MainLoop()
        loop.run()
if __name__ == '__main__':
    stream = EventStreamObject()
    stream.run()

 






 

  dbus-cli.py
  



    
     
    


  

 


 

  
 
import gobject
import dbus
from dbus.mainloop.glib import DBusGMainLoop
def handler(sender = None):
    print "got signal from %r" % sender
if __name__ == '__main__':
    bus = dbus.SystemBus(mainloop = DBusGMainLoop())
    stream = bus.get_object('org.reviewboard.EventStream', '/org/reviewboard/EventStream')
    stream.connect_to_signal('event', handler, dbus_interface = 'org.reviewboard.EventStream')
    loop = gobject.MainLoop()
    loop.run()

 






 

  org.reviewboard.EventStream.conf
  



    
     
    


  

 


 

  

 






  


 










 

   

 
 

  

    
     

   
   #1
   raja****@gmai***** (Google Code)



(Is this you? Claim this profile.)


  


  
Apologize if this is a naive question , but doesnt the django signals do the "publishing" part of the event. Right now, reviewboard throws certain events during the lifecycle of a review process, which can be caught by listeners to do certain actions. The catch is that, its synchronous but this post (http://www.chrisdpratt.com/2008/02/16/signals-in-django-stuff-thats-not-documented-well/) talks about running things asynchronously and there are definitely other ways of doing it than mentioned in the thread.

Is the DBus option provided an alternative to the sending part of django as well ? 



 








 

   

 
 

  

    
     

   
   #2
   matthew********@kitwa****** (Google Code)



(Is this you? Claim this profile.)


  


  
If you look at the gerrit link, the way gerrit does this (which has value in a: makes it easier to port from gerrit to RB, and b: allows for much more loosely coupled, not to mention cross-server, interaction) is you ssh into the RB server and run a command that starts listening to the event stream. One important characteristic is that this works not only cross-process, but cross-*user*.

Do django signals do that? My naïve assumption was that they are in-process only (maybe I am wrong)? Ergo, some mechanism is needed to 'forward' the django signals to some IPC system. Since dbus is often running on modern linux systems anyway, and there is a good chance python dbus bindings are already installed also, my thought was to use that, but any IPC system could be used. Using the dbus system bus allows the (dbus) signals to be read by any process on the system (running as any user*), which allows for the 'ssh in and run the listener', similar to gerrit.

(* more fine-grained access control can be provided by modifying the dbus ACL's, e.g. to limit to only certain users.)

There could also be value using a network-based IPC system that wouldn't require being logged in, but then you're much more likely talking about something that isn't already installed, and needs to be installed on the client also. That's not to say that the benefits may not outweigh the drawbacks, just why my first thought went to dbus.

So... short answer to your question: no, dbus doesn't replace djang. Django signals the extension (in-process), which forwards across dbus (inter-process) to listeners (which are separate processes, which are almost certainly running as different users, since you don't want people logging in as the httpd user). I suppose you *could* replace django with dbus, but I doubt you'd want to; using IPC for intra-process communication is usually overkill.



 








 

  

   david

  

 
 

  

    
     

   
   #3
   david


  



  
    


   
  • 
    
    
    +ExtensionIdea
    
   
    • 


  


 








 

  

   david

  

 
 

  

    
     

   
   #4
   david


  


  
This is covered by the status updates feature.



  
    


   
  • 
    
    
    -New
    +Fixed
    
   
    •