Welcome to the Splat alpha!

The command I'm using is (copied from search-cron.conf):

sudo "/usr/bin/python" "/usr/local/bin/rb-site" manage "<site-name>" index -- --full

I think this is an error in the command you used: where you have "<site-name>", it should be the path to the site installation files on the filesystem.

• Status:

  +	UserError

Where I have "<site-name>", I put the full path to my site:

/var/www/test-site

However, this seems to be not working as I get the DeprecationWarning regarding the empty SECRET_KEY.

SECRET_KEY should be defined in the settings_local.py file in /var/www/test-site/conf/

SECRET_KEY was already defined in my settings_local.py at /var/www/test-site/conf. So I don't understand why I got this warning in the first place.

But I think this is unrelated to the actual problem I'm having: not being able to update/create the index (because of the error: 'Unknown command: 'index').

And if I do 'rb-site help' I get the following (in which I don't see the 'index'-command):

Usage: rb-site command [options] path

Options:
  --version             show program's version number and exit
  -h, --help            show this help message and exit
  --console             force the console UI
  -d, --debug           display debug output

  'install' command:
    Installs a new Review Board site tree and generates web server
    configuration files. This will ask several questions about the
    site before performing the installation.

    --copy-media        copy media files instead of symlinking
    --noinput           run non-interactively using configuration provided in
                        command-line options
    --domain-name=DOMAIN_NAME
                        fully-qualified host name of the site, excluding the
                        http://, port or path
    --site-root=SITE_ROOT
                        path to the site relative to the domain name
    --static-url=STATIC_URL
                        the URL containing the static (shipped) media files
    --media-url=MEDIA_URL
                        the URL containing the uploaded media files
    --db-type=DB_TYPE   database type (mysql, postgresql or sqlite3)
    --db-name=DB_NAME   database name (not for sqlite3)
    --db-host=DB_HOST   database host (not for sqlite3)
    --db-user=DB_USER   database user (not for sqlite3)
    --db-pass=DB_PASS   password for the database user (not for sqlite3)
    --cache-type=CACHE_TYPE
                        cache server type (memcached or file)
    --cache-info=CACHE_INFO
                        cache identifier (memcached connection string or file
                        cache directory)
    --web-server-type=WEB_SERVER_TYPE
                        web server (apache or lighttpd)
    --web-server-port=WEB_SERVER_PORT
                        port that the web server should listen on
    --python-loader=PYTHON_LOADER
                        python loader for apache (modpython, fastcgi or wsgi)
    --admin-user=ADMIN_USER
                        the site administrator's username
    --admin-password=ADMIN_PASSWORD
                        the site administrator's password
    --admin-email=ADMIN_EMAIL
                        the site administrator's e-mail address

  'upgrade' command:
    Upgrades an existing site installation, synchronizing media trees and
    upgrading the database, unless otherwise specified.

    --no-db-upgrade     don't upgrade the database

The 'manage' command just doesn't have help defined, it's still there.

It sounds like something's going wrong with the path to the site, so it's not finding the django management commands, and not finding your settings_local.py file.

• Status:

  -	UserError
  +	New

Is there something I could check in my installation that might be causing this problem?

This is an error we've never seen before, and frankly, it doesn't make any sense. That is, unless you either have a stray settings_local.py or settings_local.pyc file somewhere on your filesystem.

Does the 'shell' management command work? If so, can you do that and then:

    from django.conf import settings
    print settings.__file__
    print settings.SECRET_KEY

Don't actually show us the SECRET_KEY, but do show the file path, and let us know if the SECRET_KEY matches what is in your settings_local.py file.

• Status:

  -	New
  +	NeedInfo

When I issued:
sudo "/usr/bin/python" "/usr/local/bin/rb-site" manage <path/to/site> shell

I got the following deprecation warning:
/usr/local/lib/python2.6/dist-packages/Django-1.4.5-py2.6.egg/django/conf/__init__.py:110: DeprecationWarning: The SECRET_KEY setting must not be empty.
  warnings.warn("The SECRET_KEY setting must not be empty.", DeprecationWarning)

Although, the shell did start. However, when I issue following commands:
from django.conf import settings
print settings.__file__

I got:
/usr/local/lib/python2.6/dist-packages/Django-1.4.5-py2.6.egg/django/utils/functional.pyc in inner(self, *args)
    183         if self._wrapped is empty:
    184             self._setup()
--> 185         return func(self._wrapped, *args)
    186     return inner
    187

AttributeError: 'Settings' object has no attribute '__file__'

I think I have solved part of the problem :D

In our company, we have a 'commit-reader'-script running on the codereview server, which monitors any svn-commits. For each such commit, a draft review-request (or update review, depending on the commit-message) is created. During initialization of this script, it tries to synchronize svn and Review Board, by issuing many webapi-requests onto the Review Board server in order to find out which svn-revisions have already been added to Review Board.

However, each webapi http-request creates a new record in the django_sessions-table. This gave problems, as the hard-disk quickly ran out of memory. In order to fix this, I 'patched' the Sessions-middleware, by creating the following file (about 2 years ago):

# settings_local_sessionmiddleware.py ==========================

from django.contrib.sessions.middleware import SessionMiddleware


def dontsave(must_create=True):
    pass


class PatchedSessionMiddleware(SessionMiddleware):
    def process_request(self, request):
        super(PatchedSessionMiddleware, self).process_request(request)

        # Because the commit_reader performs lots of requests to /api/... urls,
        # for each request a session-id would be stored in the database.
        # This would grow the database in size very quickly.
        #
        # This check prevents that.
        if request.path_info[0:5] == '/api/':
            request.session.save = dontsave

# First import the original MIDDLEWARE_CLASSES-list
from reviewboard.settings import MIDDLEWARE_CLASSES

# Now overwrite the SessionMiddleware-class with our patched version
cls = SessionMiddleware.__module__ + '.' + SessionMiddleware.__name__
idx = MIDDLEWARE_CLASSES.index(cls)
MIDDLEWARE_CLASSES[idx] = __name__ + '.' + PatchedSessionMiddleware.__name__

#============================================================================

And finally I added the following line to the settings_local.py file:

from settings_local_sessionmiddleware import MIDDLEWARE_CLASSES

Apparently, importing this file was the cause of the current problem (perhaps due to a circular dependency?). By removing this line, it looks like everything works again.

Indexing now works fine, as well as the 'rb-site manage shell' (although issuing 'print settings.__file__' still complains about the 'Settings' object not having this attribute).

The only issue that remains is the Sessions-middleware. If I inspect my database, I see:

                      relation                       |  size
-----------------------------------------------------+---------
 public.django_session_pkey                          | 1785 MB
 public.django_session_expire_date                   | 1030 MB
 public.django_session                               | 699 MB
 pg_toast.pg_toast_16671                             | 203 MB
 public.diffviewer_filediff                          | 70 MB
 pg_toast.pg_toast_119386                            | 51 MB
 public.diffviewer_filediffdata                      | 19 MB
 public.reviews_reviewrequest                        | 3776 kB
 public.diffviewer_filediff_diff_hash_id             | 3040 kB
 public.diffviewer_filediff_diff_hash_id_like        | 3040 kB
 public.reviews_comment                              | 2752 kB
 pg_toast.pg_toast_16671_index                       | 2600 kB
 public.diffviewer_filediff_diffset_id               | 2088 kB
 public.changedescs_changedescription                | 1912 kB
 public.diffviewer_filediff_parent_diff_hash_id_like | 1848 kB
 public.diffviewer_filediff_parent_diff_hash_id      | 1840 kB
 public.diffviewer_filediffdata_pkey                 | 1776 kB
 public.diffviewer_filediff_pkey                     | 1728 kB
 public.reviews_review                               | 1424 kB
 public.diffviewer_diffset                           | 1136 kB

As can be seen, the session-data uses quite some space. Furthermore, I believe I already emptied the django_session table back then, when I created the patched middleware-class.

Is this a known problem and is there a better way to solve this?

There is a 'clearsessions' management command that you can hook up to a cron job to clear out expired sessions.

I'm going to repurpose this bug to see if we can disable session saving for API requests generally.

• Status:

  -	NeedInfo
  +	New

• Tags:

  +	Component-API

• Summary:

  +	API requests create a ton of entries in the 'sessions' table.

The 'clearsessions' command is not available in rbtools 0.5.1, is that right?

It's a management command to be run on the server. So you'd use something like this:

$ rb-site manage /path/to/site clearsessions

My bad, it's part of reviewboard and not rbtools. However, in our version (1.7.9) the command 'clearsessions' is not available.

What exactly does the command do? Perhaps I can do it manually for the time being, until we upgrade to a newer version.

It's a command provided by Django to clear the expired sessions from the database.

Older versions appear to call this "cleanup" instead of "clearsessions", so give that a try.