introduce user groups by t-lenz · Pull Request #1621 · cms-dev/cms

t-lenz · 2026-01-18T12:21:42Z

We've been using CMS for the German IOI selection for many years now, and we often have offsite contestants who cannot compete at the same time as the onsite contestants, e.g. because they are ill at the time of the contest or because they live in a different timezone.

If one wants to allot different time slots for the users of a contest in vanilla CMS, this would require setting delay (and possibly extratime) for users individually. This can get pretty inconvenient, in particular if there are multiple contestants affected. Moreover, this does not allow having one group of contestants (in the above example, the onsite contestants) compete in a fixed timeslot and others in a timeslot of their choice (USACO style).

This pull request changes the DB format to introduce user groups. Participations are now assigned a group, and start, stop, per_user_time, etc. are no longer properties of a contest, but instead of a user group. In the above example usecase, one could then simply have one group for the onsite contestants and one for offsite contestants, and one could e.g. have the first group compete at a fixed time, with the offsite group being able to (more or less) freely choose a timeslot. As a proof of concept, we have also adjusted the Italian yaml loader so that it is able to assign users to groups; old yaml configs are still valid and result in all users being assigned to the same group.

This is based on code that has been in use in the German fork of CMS for over 10 years now, but has been cleaned up and updated for the latest version of CMS. Most of the original code was written by @fagu and @magula; the present version also contains contributions by @chuyang-wang-dev.

@fagu

much of this is based on code originally written by @fagu and @magula

prandla

hi, sorry for taking this long to get around to this!

overall, we (or well, at least Luca and i) agree that this would be a good feature to have. i left some comments, some of them minor, some of them requiring more work. i could do all of these fixes myself too, though then i think someone else will need to re-review it :)

in addition to the inline comments, we will need to fix the test suite failures (and possibly add new tests, but currently we don't have a good way to write tests for AWS UI, so i think it's fine to skip that for now). also, we need a DumpUpdater and an sql migration. also, you should run Ruff on modified lines, and some places could use more type hints.

and please do let me know if you disagree with some of the proposed changes, i'm always up for some healthy debate :)

cms/db/contest.py

cms/db/user.py

prandla · 2026-02-11T16:58:56Z

cms/db/user.py

    __table_args__ = (UniqueConstraint("contest_id", "user_id"),)

+    # Group this user belongs to
+    group_id = Column(


you added this group_id column, but there is still the participation.contest column. this seems like a normalization violation to me. it seems we should delete participation.contest and use participation.group.contest instead (or possibly add an @property such that participation.contest is a shorthand for participation.group.contest).

This is actually somewhat tricky. Right now, there is a UniquenessConstraint on the pair (user_id, contest_id) in the Participation table to ensure that each user has at most one participation per contest. As far as I understand, SQL would not allow a UniquenessConstraint on the pair (user_id, group.contest_id) since this involves more than one table. As we probably still want to enforce the UniquenessConstraint, my suggestion would be to leave the contest_id and contest columns for Participation (Contest.participations can instead be replaced by a @property as suggested) and to use a ForeignKeyConstraint to ensure that (group_id, contest_id) for Participation matches with (id, contest_id) for Group. This way, we would at least ensure that the data is consistent, although one would still have to set contest_id manually.

oh, i forgot about that unique constraint. in that case yeah, keeping contest_id makes sense.

enforcing it with a foreign key is definitely good enough. i'm not worried about insertion convenience here, just data integrity.

cms/server/admin/handlers/user.py

cms/server/admin/templates/contest.html

cms/server/admin/templates/participation.html

cmscontrib/AddParticipation.py

Should fix all failures except DumpImporterTest and schema_diff_test (which both likely need more work).

codecov · 2026-02-14T18:03:34Z

❌ 6 Tests Failed:

Tests completed	Failed	Passed	Skipped
695	6	689	8

View the top 3 failed test(s) by shortest run time

cmstestsuite/unit_tests/cmscontrib/DumpImporterTest.py::TestDumpImporter::test_import_skip_generated

Stack Traces | 0.021s run time

self = <sqlalchemy.engine.base.Connection object at 0x7f75ef56bf90>
dialect = <sqlalchemy.dialects.postgresql.psycopg2.PGDialect_psycopg2 object at 0x7f75f2c9dc90>
constructor = <bound method DefaultExecutionContext._init_compiled of <class 'sqlalchemy.dialects.postgresql.psycopg2.PGExecutionContext_psycopg2'>>
statement = 'INSERT INTO participations (ip, starting_time, delay_time, extra_time, password, hidden, unrestricted, contest_id, us...d)s, %(hidden)s, %(unrestricted)s, %(contest_id)s, %(user_id)s, %(group_id)s, %(team_id)s) RETURNING participations.id'
parameters = {'contest_id': 8, 'delay_time': datetime.timedelta(0), 'extra_time': datetime.timedelta(0), 'group_id': None, ...}
args = (<sqlalchemy.dialects.postgresql.psycopg2.PGCompiler_psycopg2 object at 0x7f75f07a30d0>, [{'contest_id': 8, 'delay_time': datetime.timedelta(0), 'extra_time': datetime.timedelta(0), 'group_id': None, ...}])
conn = <sqlalchemy.pool.base._ConnectionFairy object at 0x7f75ef41b3d0>
context = <sqlalchemy.dialects.postgresql.psycopg2.PGExecutionContext_psycopg2 object at 0x7f75ef56a650>

    def _execute_context(
        self, dialect, constructor, statement, parameters, *args
    ):
        """Create an :class:`.ExecutionContext` and execute, returning
        a :class:`_engine.ResultProxy`.
    
        """
    
        try:
            try:
                conn = self.__connection
            except AttributeError:
                # escape "except AttributeError" before revalidating
                # to prevent misleading stacktraces in Py3K
                conn = None
            if conn is None:
                conn = self._revalidate_connection()
    
            context = constructor(dialect, self, conn, *args)
        except BaseException as e:
            self._handle_dbapi_exception(
                e, util.text_type(statement), parameters, None, None
            )
    
        if context.compiled:
            context.pre_exec()
    
        cursor, statement, parameters = (
            context.cursor,
            context.statement,
            context.parameters,
        )
    
        if not context.executemany:
            parameters = parameters[0]
    
        if self._has_events or self.engine._has_events:
            for fn in self.dispatch.before_cursor_execute:
                statement, parameters = fn(
                    self,
                    cursor,
                    statement,
                    parameters,
                    context,
                    context.executemany,
                )
    
        if self._echo:
            self.engine.logger.info(statement)
            if not self.engine.hide_parameters:
                self.engine.logger.info(
                    "%r",
                    sql_util._repr_params(
                        parameters, batches=10, ismulti=context.executemany
                    ),
                )
            else:
                self.engine.logger.info(
                    "[SQL parameters hidden due to hide_parameters=True]"
                )
    
        evt_handled = False
        try:
            if context.executemany:
                if self.dialect._has_events:
                    for fn in self.dialect.dispatch.do_executemany:
                        if fn(cursor, statement, parameters, context):
                            evt_handled = True
                            break
                if not evt_handled:
                    self.dialect.do_executemany(
                        cursor, statement, parameters, context
                    )
            elif not parameters and context.no_parameters:
                if self.dialect._has_events:
                    for fn in self.dialect.dispatch.do_execute_no_params:
                        if fn(cursor, statement, context):
                            evt_handled = True
                            break
                if not evt_handled:
                    self.dialect.do_execute_no_params(
                        cursor, statement, context
                    )
            else:
                if self.dialect._has_events:
                    for fn in self.dialect.dispatch.do_execute:
                        if fn(cursor, statement, parameters, context):
                            evt_handled = True
                            break
                if not evt_handled:
>                   self.dialect.do_execute(
                        cursor, statement, parameters, context
                    )

.........................................................../cms/lib/python3.11.../sqlalchemy/engine/base.py:1276: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <sqlalchemy.dialects.postgresql.psycopg2.PGDialect_psycopg2 object at 0x7f75f2c9dc90>
cursor = <cursor object at 0x7f75ef506890; closed: -1>
statement = 'INSERT INTO participations (ip, starting_time, delay_time, extra_time, password, hidden, unrestricted, contest_id, us...d)s, %(hidden)s, %(unrestricted)s, %(contest_id)s, %(user_id)s, %(group_id)s, %(team_id)s) RETURNING participations.id'
parameters = {'contest_id': 8, 'delay_time': datetime.timedelta(0), 'extra_time': datetime.timedelta(0), 'group_id': None, ...}
context = <sqlalchemy.dialects.postgresql.psycopg2.PGExecutionContext_psycopg2 object at 0x7f75ef56a650>

    def do_execute(self, cursor, statement, parameters, context=None):
>       cursor.execute(statement, parameters)
E       psycopg2.errors.NotNullViolation: null value in column "group_id" of relation "participations" violates not-null constraint
E       DETAIL:  Failing row contains (4, null, null, 00:00:00, 00:00:00, null, f, f, 8, 4, null, null).

.........................................................../cms/lib/python3.11.../sqlalchemy/engine/default.py:608: NotNullViolation

The above exception was the direct cause of the following exception:

self = <DumpImporterTest.TestDumpImporter testMethod=test_import_skip_generated>

    def test_import_skip_generated(self):
        """Test importing everything but the generated data."""
        self.write_dump(TestDumpImporter.DUMP)
        self.write_files(TestDumpImporter.FILES)
>       self.assertTrue(self.do_import(skip_generated=True))
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

.../unit_tests/cmscontrib/DumpImporterTest.py:261: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
.../unit_tests/cmscontrib/DumpImporterTest.py:150: in do_import
    skip_users=skip_users).do_import()
                           ^^^^^^^^^^^
cmscontrib/DumpImporter.py:312: in do_import
    session.flush()
.........................................................../cms/lib/python3.11.../sqlalchemy/orm/session.py:2540: in flush
    self._flush(objects)
.........................................................../cms/lib/python3.11.../sqlalchemy/orm/session.py:2681: in _flush
    with util.safe_reraise():
.........................................................../cms/lib/python3.11.../sqlalchemy/util/langhelpers.py:68: in __exit__
    compat.raise_(
.........................................................../cms/lib/python3.11.../sqlalchemy/util/compat.py:182: in raise_
    raise exception
.........................................................../cms/lib/python3.11.../sqlalchemy/orm/session.py:2642: in _flush
    flush_context.execute()
.........................................................../cms/lib/python3.11.../sqlalchemy/orm/unitofwork.py:422: in execute
    rec.execute(self)
.........................................................../cms/lib/python3.11.../sqlalchemy/orm/unitofwork.py:586: in execute
    persistence.save_obj(
.........................................................../cms/lib/python3.11.../sqlalchemy/orm/persistence.py:239: in save_obj
    _emit_insert_statements(
.........................................................../cms/lib/python3.11.../sqlalchemy/orm/persistence.py:1135: in _emit_insert_statements
    result = cached_connections[connection].execute(
.........................................................../cms/lib/python3.11.../sqlalchemy/engine/base.py:1011: in execute
    return meth(self, multiparams, params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
.........................................................../cms/lib/python3.11.../sqlalchemy/sql/elements.py:298: in _execute_on_connection
    return connection._execute_clauseelement(self, multiparams, params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
.........................................................../cms/lib/python3.11.../sqlalchemy/engine/base.py:1124: in _execute_clauseelement
    ret = self._execute_context(
.........................................................../cms/lib/python3.11.../sqlalchemy/engine/base.py:1316: in _execute_context
    self._handle_dbapi_exception(
.........................................................../cms/lib/python3.11.../sqlalchemy/engine/base.py:1510: in _handle_dbapi_exception
    util.raise_(
.........................................................../cms/lib/python3.11.../sqlalchemy/util/compat.py:182: in raise_
    raise exception
.........................................................../cms/lib/python3.11.../sqlalchemy/engine/base.py:1276: in _execute_context
    self.dialect.do_execute(
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <sqlalchemy.dialects.postgresql.psycopg2.PGDialect_psycopg2 object at 0x7f75f2c9dc90>
cursor = <cursor object at 0x7f75ef506890; closed: -1>
statement = 'INSERT INTO participations (ip, starting_time, delay_time, extra_time, password, hidden, unrestricted, contest_id, us...d)s, %(hidden)s, %(unrestricted)s, %(contest_id)s, %(user_id)s, %(group_id)s, %(team_id)s) RETURNING participations.id'
parameters = {'contest_id': 8, 'delay_time': datetime.timedelta(0), 'extra_time': datetime.timedelta(0), 'group_id': None, ...}
context = <sqlalchemy.dialects.postgresql.psycopg2.PGExecutionContext_psycopg2 object at 0x7f75ef56a650>

    def do_execute(self, cursor, statement, parameters, context=None):
>       cursor.execute(statement, parameters)
E       sqlalchemy.exc.IntegrityError: (psycopg2.errors.NotNullViolation) null value in column "group_id" of relation "participations" violates not-null constraint
E       DETAIL:  Failing row contains (4, null, null, 00:00:00, 00:00:00, null, f, f, 8, 4, null, null).
E       
E       [SQL: INSERT INTO participations (ip, starting_time, delay_time, extra_time, password, hidden, unrestricted, contest_id, user_id, group_id, team_id) VALUES (CAST(%(ip)s AS CIDR[])::CIDR[], %(starting_time)s, %(delay_time)s, %(extra_time)s, %(password)s, %(hidden)s, %(unrestricted)s, %(contest_id)s, %(user_id)s, %(group_id)s, %(team_id)s) RETURNING participations.id]
E       [parameters: {'ip': None, 'starting_time': None, 'delay_time': datetime.timedelta(0), 'extra_time': datetime.timedelta(0), 'password': None, 'hidden': False, 'unrestricted': False, 'contest_id': 8, 'user_id': 4, 'group_id': None, 'team_id': None}]
E       (Background on this error at: http://sqlalche..../e/13/gkpj)

.........................................................../cms/lib/python3.11.../sqlalchemy/engine/default.py:608: IntegrityError

cmstestsuite/unit_tests/cmscontrib/DumpImporterTest.py::TestDumpImporter::test_import_skip_files

Stack Traces | 0.022s run time

self = <sqlalchemy.engine.base.Connection object at 0x7f75ef62fc50>
dialect = <sqlalchemy.dialects.postgresql.psycopg2.PGDialect_psycopg2 object at 0x7f75f2c9dc90>
constructor = <bound method DefaultExecutionContext._init_compiled of <class 'sqlalchemy.dialects.postgresql.psycopg2.PGExecutionContext_psycopg2'>>
statement = 'INSERT INTO participations (ip, starting_time, delay_time, extra_time, password, hidden, unrestricted, contest_id, us...d)s, %(hidden)s, %(unrestricted)s, %(contest_id)s, %(user_id)s, %(group_id)s, %(team_id)s) RETURNING participations.id'
parameters = {'contest_id': 6, 'delay_time': datetime.timedelta(0), 'extra_time': datetime.timedelta(0), 'group_id': None, ...}
args = (<sqlalchemy.dialects.postgresql.psycopg2.PGCompiler_psycopg2 object at 0x7f75f07a30d0>, [{'contest_id': 6, 'delay_time': datetime.timedelta(0), 'extra_time': datetime.timedelta(0), 'group_id': None, ...}])
conn = <sqlalchemy.pool.base._ConnectionFairy object at 0x7f75f07ef250>
context = <sqlalchemy.dialects.postgresql.psycopg2.PGExecutionContext_psycopg2 object at 0x7f75f07ef5d0>

    def _execute_context(
        self, dialect, constructor, statement, parameters, *args
    ):
        """Create an :class:`.ExecutionContext` and execute, returning
        a :class:`_engine.ResultProxy`.
    
        """
    
        try:
            try:
                conn = self.__connection
            except AttributeError:
                # escape "except AttributeError" before revalidating
                # to prevent misleading stacktraces in Py3K
                conn = None
            if conn is None:
                conn = self._revalidate_connection()
    
            context = constructor(dialect, self, conn, *args)
        except BaseException as e:
            self._handle_dbapi_exception(
                e, util.text_type(statement), parameters, None, None
            )
    
        if context.compiled:
            context.pre_exec()
    
        cursor, statement, parameters = (
            context.cursor,
            context.statement,
            context.parameters,
        )
    
        if not context.executemany:
            parameters = parameters[0]
    
        if self._has_events or self.engine._has_events:
            for fn in self.dispatch.before_cursor_execute:
                statement, parameters = fn(
                    self,
                    cursor,
                    statement,
                    parameters,
                    context,
                    context.executemany,
                )
    
        if self._echo:
            self.engine.logger.info(statement)
            if not self.engine.hide_parameters:
                self.engine.logger.info(
                    "%r",
                    sql_util._repr_params(
                        parameters, batches=10, ismulti=context.executemany
                    ),
                )
            else:
                self.engine.logger.info(
                    "[SQL parameters hidden due to hide_parameters=True]"
                )
    
        evt_handled = False
        try:
            if context.executemany:
                if self.dialect._has_events:
                    for fn in self.dialect.dispatch.do_executemany:
                        if fn(cursor, statement, parameters, context):
                            evt_handled = True
                            break
                if not evt_handled:
                    self.dialect.do_executemany(
                        cursor, statement, parameters, context
                    )
            elif not parameters and context.no_parameters:
                if self.dialect._has_events:
                    for fn in self.dialect.dispatch.do_execute_no_params:
                        if fn(cursor, statement, context):
                            evt_handled = True
                            break
                if not evt_handled:
                    self.dialect.do_execute_no_params(
                        cursor, statement, context
                    )
            else:
                if self.dialect._has_events:
                    for fn in self.dialect.dispatch.do_execute:
                        if fn(cursor, statement, parameters, context):
                            evt_handled = True
                            break
                if not evt_handled:
>                   self.dialect.do_execute(
                        cursor, statement, parameters, context
                    )

.........................................................../cms/lib/python3.11.../sqlalchemy/engine/base.py:1276: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <sqlalchemy.dialects.postgresql.psycopg2.PGDialect_psycopg2 object at 0x7f75f2c9dc90>
cursor = <cursor object at 0x7f75ef5896c0; closed: -1>
statement = 'INSERT INTO participations (ip, starting_time, delay_time, extra_time, password, hidden, unrestricted, contest_id, us...d)s, %(hidden)s, %(unrestricted)s, %(contest_id)s, %(user_id)s, %(group_id)s, %(team_id)s) RETURNING participations.id'
parameters = {'contest_id': 6, 'delay_time': datetime.timedelta(0), 'extra_time': datetime.timedelta(0), 'group_id': None, ...}
context = <sqlalchemy.dialects.postgresql.psycopg2.PGExecutionContext_psycopg2 object at 0x7f75f07ef5d0>

    def do_execute(self, cursor, statement, parameters, context=None):
>       cursor.execute(statement, parameters)
E       psycopg2.errors.NotNullViolation: null value in column "group_id" of relation "participations" violates not-null constraint
E       DETAIL:  Failing row contains (3, null, null, 00:00:00, 00:00:00, null, f, f, 6, 3, null, null).

.........................................................../cms/lib/python3.11.../sqlalchemy/engine/default.py:608: NotNullViolation

The above exception was the direct cause of the following exception:

self = <DumpImporterTest.TestDumpImporter testMethod=test_import_skip_files>

    def test_import_skip_files(self):
        """Test importing the json but not the files."""
        self.write_dump(TestDumpImporter.DUMP)
        self.write_files(TestDumpImporter.FILES)
>       self.assertTrue(self.do_import(load_files=False))
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

.../unit_tests/cmscontrib/DumpImporterTest.py:277: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
.../unit_tests/cmscontrib/DumpImporterTest.py:150: in do_import
    skip_users=skip_users).do_import()
                           ^^^^^^^^^^^
cmscontrib/DumpImporter.py:312: in do_import
    session.flush()
.........................................................../cms/lib/python3.11.../sqlalchemy/orm/session.py:2540: in flush
    self._flush(objects)
.........................................................../cms/lib/python3.11.../sqlalchemy/orm/session.py:2681: in _flush
    with util.safe_reraise():
.........................................................../cms/lib/python3.11.../sqlalchemy/util/langhelpers.py:68: in __exit__
    compat.raise_(
.........................................................../cms/lib/python3.11.../sqlalchemy/util/compat.py:182: in raise_
    raise exception
.........................................................../cms/lib/python3.11.../sqlalchemy/orm/session.py:2642: in _flush
    flush_context.execute()
.........................................................../cms/lib/python3.11.../sqlalchemy/orm/unitofwork.py:422: in execute
    rec.execute(self)
.........................................................../cms/lib/python3.11.../sqlalchemy/orm/unitofwork.py:586: in execute
    persistence.save_obj(
.........................................................../cms/lib/python3.11.../sqlalchemy/orm/persistence.py:239: in save_obj
    _emit_insert_statements(
.........................................................../cms/lib/python3.11.../sqlalchemy/orm/persistence.py:1135: in _emit_insert_statements
    result = cached_connections[connection].execute(
.........................................................../cms/lib/python3.11.../sqlalchemy/engine/base.py:1011: in execute
    return meth(self, multiparams, params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
.........................................................../cms/lib/python3.11.../sqlalchemy/sql/elements.py:298: in _execute_on_connection
    return connection._execute_clauseelement(self, multiparams, params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
.........................................................../cms/lib/python3.11.../sqlalchemy/engine/base.py:1124: in _execute_clauseelement
    ret = self._execute_context(
.........................................................../cms/lib/python3.11.../sqlalchemy/engine/base.py:1316: in _execute_context
    self._handle_dbapi_exception(
.........................................................../cms/lib/python3.11.../sqlalchemy/engine/base.py:1510: in _handle_dbapi_exception
    util.raise_(
.........................................................../cms/lib/python3.11.../sqlalchemy/util/compat.py:182: in raise_
    raise exception
.........................................................../cms/lib/python3.11.../sqlalchemy/engine/base.py:1276: in _execute_context
    self.dialect.do_execute(
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <sqlalchemy.dialects.postgresql.psycopg2.PGDialect_psycopg2 object at 0x7f75f2c9dc90>
cursor = <cursor object at 0x7f75ef5896c0; closed: -1>
statement = 'INSERT INTO participations (ip, starting_time, delay_time, extra_time, password, hidden, unrestricted, contest_id, us...d)s, %(hidden)s, %(unrestricted)s, %(contest_id)s, %(user_id)s, %(group_id)s, %(team_id)s) RETURNING participations.id'
parameters = {'contest_id': 6, 'delay_time': datetime.timedelta(0), 'extra_time': datetime.timedelta(0), 'group_id': None, ...}
context = <sqlalchemy.dialects.postgresql.psycopg2.PGExecutionContext_psycopg2 object at 0x7f75f07ef5d0>

    def do_execute(self, cursor, statement, parameters, context=None):
>       cursor.execute(statement, parameters)
E       sqlalchemy.exc.IntegrityError: (psycopg2.errors.NotNullViolation) null value in column "group_id" of relation "participations" violates not-null constraint
E       DETAIL:  Failing row contains (3, null, null, 00:00:00, 00:00:00, null, f, f, 6, 3, null, null).
E       
E       [SQL: INSERT INTO participations (ip, starting_time, delay_time, extra_time, password, hidden, unrestricted, contest_id, user_id, group_id, team_id) VALUES (CAST(%(ip)s AS CIDR[])::CIDR[], %(starting_time)s, %(delay_time)s, %(extra_time)s, %(password)s, %(hidden)s, %(unrestricted)s, %(contest_id)s, %(user_id)s, %(group_id)s, %(team_id)s) RETURNING participations.id]
E       [parameters: {'ip': None, 'starting_time': None, 'delay_time': datetime.timedelta(0), 'extra_time': datetime.timedelta(0), 'password': None, 'hidden': False, 'unrestricted': False, 'contest_id': 6, 'user_id': 3, 'group_id': None, 'team_id': None}]
E       (Background on this error at: http://sqlalche..../e/13/gkpj)

.........................................................../cms/lib/python3.11.../sqlalchemy/engine/default.py:608: IntegrityError

cmstestsuite/unit_tests/cmscontrib/DumpImporterTest.py::TestDumpImporter::test_import_old

Stack Traces | 0.057s run time

self = <sqlalchemy.engine.base.Connection object at 0x7f75ef4f0bd0>
dialect = <sqlalchemy.dialects.postgresql.psycopg2.PGDialect_psycopg2 object at 0x7f75f2c9dc90>
constructor = <bound method DefaultExecutionContext._init_compiled of <class 'sqlalchemy.dialects.postgresql.psycopg2.PGExecutionContext_psycopg2'>>
statement = 'INSERT INTO participations (ip, starting_time, delay_time, extra_time, password, hidden, unrestricted, contest_id, us...d)s, %(hidden)s, %(unrestricted)s, %(contest_id)s, %(user_id)s, %(group_id)s, %(team_id)s) RETURNING participations.id'
parameters = {'contest_id': 4, 'delay_time': datetime.timedelta(0), 'extra_time': datetime.timedelta(0), 'group_id': None, ...}
args = (<sqlalchemy.dialects.postgresql.psycopg2.PGCompiler_psycopg2 object at 0x7f75f07a30d0>, [{'contest_id': 4, 'delay_time': datetime.timedelta(0), 'extra_time': datetime.timedelta(0), 'group_id': None, ...}])
conn = <sqlalchemy.pool.base._ConnectionFairy object at 0x7f75ef4fe850>
context = <sqlalchemy.dialects.postgresql.psycopg2.PGExecutionContext_psycopg2 object at 0x7f75ef4fc610>

    def _execute_context(
        self, dialect, constructor, statement, parameters, *args
    ):
        """Create an :class:`.ExecutionContext` and execute, returning
        a :class:`_engine.ResultProxy`.
    
        """
    
        try:
            try:
                conn = self.__connection
            except AttributeError:
                # escape "except AttributeError" before revalidating
                # to prevent misleading stacktraces in Py3K
                conn = None
            if conn is None:
                conn = self._revalidate_connection()
    
            context = constructor(dialect, self, conn, *args)
        except BaseException as e:
            self._handle_dbapi_exception(
                e, util.text_type(statement), parameters, None, None
            )
    
        if context.compiled:
            context.pre_exec()
    
        cursor, statement, parameters = (
            context.cursor,
            context.statement,
            context.parameters,
        )
    
        if not context.executemany:
            parameters = parameters[0]
    
        if self._has_events or self.engine._has_events:
            for fn in self.dispatch.before_cursor_execute:
                statement, parameters = fn(
                    self,
                    cursor,
                    statement,
                    parameters,
                    context,
                    context.executemany,
                )
    
        if self._echo:
            self.engine.logger.info(statement)
            if not self.engine.hide_parameters:
                self.engine.logger.info(
                    "%r",
                    sql_util._repr_params(
                        parameters, batches=10, ismulti=context.executemany
                    ),
                )
            else:
                self.engine.logger.info(
                    "[SQL parameters hidden due to hide_parameters=True]"
                )
    
        evt_handled = False
        try:
            if context.executemany:
                if self.dialect._has_events:
                    for fn in self.dialect.dispatch.do_executemany:
                        if fn(cursor, statement, parameters, context):
                            evt_handled = True
                            break
                if not evt_handled:
                    self.dialect.do_executemany(
                        cursor, statement, parameters, context
                    )
            elif not parameters and context.no_parameters:
                if self.dialect._has_events:
                    for fn in self.dialect.dispatch.do_execute_no_params:
                        if fn(cursor, statement, context):
                            evt_handled = True
                            break
                if not evt_handled:
                    self.dialect.do_execute_no_params(
                        cursor, statement, context
                    )
            else:
                if self.dialect._has_events:
                    for fn in self.dialect.dispatch.do_execute:
                        if fn(cursor, statement, parameters, context):
                            evt_handled = True
                            break
                if not evt_handled:
>                   self.dialect.do_execute(
                        cursor, statement, parameters, context
                    )

.........................................................../cms/lib/python3.11.../sqlalchemy/engine/base.py:1276: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <sqlalchemy.dialects.postgresql.psycopg2.PGDialect_psycopg2 object at 0x7f75f2c9dc90>
cursor = <cursor object at 0x7f75ef5064d0; closed: -1>
statement = 'INSERT INTO participations (ip, starting_time, delay_time, extra_time, password, hidden, unrestricted, contest_id, us...d)s, %(hidden)s, %(unrestricted)s, %(contest_id)s, %(user_id)s, %(group_id)s, %(team_id)s) RETURNING participations.id'
parameters = {'contest_id': 4, 'delay_time': datetime.timedelta(0), 'extra_time': datetime.timedelta(0), 'group_id': None, ...}
context = <sqlalchemy.dialects.postgresql.psycopg2.PGExecutionContext_psycopg2 object at 0x7f75ef4fc610>

    def do_execute(self, cursor, statement, parameters, context=None):
>       cursor.execute(statement, parameters)
E       psycopg2.errors.NotNullViolation: null value in column "group_id" of relation "participations" violates not-null constraint
E       DETAIL:  Failing row contains (2, null, null, 00:00:00, 00:00:00, null, f, f, 4, 2, null, null).

.........................................................../cms/lib/python3.11.../sqlalchemy/engine/default.py:608: NotNullViolation

The above exception was the direct cause of the following exception:

self = <DumpImporterTest.TestDumpImporter testMethod=test_import_old>

    def test_import_old(self):
        """Test importing an old dump.
    
        This does not pretend to be exhaustive, just makes sure the happy
        path of the updaters run successfully.
    
        """
        self.write_dump({
            "contest_key": {
                "_class": "Contest",
                "name": "contestname",
                "description": "contest description",
                "start": 1_234_567_890.000,
                "stop": 1_324_567_890.000,
                "token_initial": 2,
                "token_gen_number": 1,
                "token_gen_time": 10,
                "token_total": 100,
                "token_max": 100,
                "tasks": ["task_key"],
            },
            "task_key": {
                "_class": "Task",
                "name": "taskname",
                "title": "task title",
                "num": 0,
                "primary_statements": "[\"en\", \"ja\"]",
                "token_initial": None,
                "token_gen_number": 0,
                "token_gen_time": 0,
                "token_total": None,
                "token_max": None,
                "task_type": "Batch",
                "task_type_parameters": "[]",
                "score_type": "Sum",
                "score_type_parameters": "[]",
                "time_limit": 0.0,
                "memory_limit": None,
                "contest": "contest_key",
                "attachments": {},
                "managers": {},
                "testcases": {},
                "submissions": ["sub1_key", "sub2_key"],
                "user_tests": [],
            },
            "user_key": {
                "_class": "User",
                "username": "username",
                "first_name": "First Name",
                "last_name": "Last Name",
                "password": "pwd",
                "email": "",
                "ip": "0.0.0.0",
                "preferred_languages": "[\"en\", \"it_IT\"]",
                "contest": "contest_key",
                "submissions": ["sub1_key", "sub2_key"],
            },
            "sub1_key": {
                "_class": "Submission",
                "timestamp": 1_234_567_890.123,
                "language": "c",
                "user": "user_key",
                "task": "task_key",
                "compilation_text": "OK [1.234 - 20]",
                "files": {},
                "executables": {"exe": "exe_key"},
                "evaluations": [],
            },
            "sub2_key": {
                "_class": "Submission",
                "timestamp": 1_234_567_900.123,
                "language": "c",
                "user": "user_key",
                "task": "task_key",
                "compilation_text": "Killed with signal 11 [0.123 - 10]\n",
                "files": {},
                "executables": {},
                "evaluations": [],
            },
            "exe_key": {
                "_class": "Executable",
                "submission": "sub1_key",
                "filename": "exe",
                "digest": TestDumpImporter.GENERATED_FILE_DIGEST,
            },
            "_version": 1,
            "_objects": ["contest_key", "user_key"],
        })
        self.write_files(TestDumpImporter.FILES)
>       self.assertTrue(self.do_import(skip_generated=True))
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

.../unit_tests/cmscontrib/DumpImporterTest.py:394: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
.../unit_tests/cmscontrib/DumpImporterTest.py:150: in do_import
    skip_users=skip_users).do_import()
                           ^^^^^^^^^^^
cmscontrib/DumpImporter.py:312: in do_import
    session.flush()
.........................................................../cms/lib/python3.11.../sqlalchemy/orm/session.py:2540: in flush
    self._flush(objects)
.........................................................../cms/lib/python3.11.../sqlalchemy/orm/session.py:2681: in _flush
    with util.safe_reraise():
.........................................................../cms/lib/python3.11.../sqlalchemy/util/langhelpers.py:68: in __exit__
    compat.raise_(
.........................................................../cms/lib/python3.11.../sqlalchemy/util/compat.py:182: in raise_
    raise exception
.........................................................../cms/lib/python3.11.../sqlalchemy/orm/session.py:2642: in _flush
    flush_context.execute()
.........................................................../cms/lib/python3.11.../sqlalchemy/orm/unitofwork.py:422: in execute
    rec.execute(self)
.........................................................../cms/lib/python3.11.../sqlalchemy/orm/unitofwork.py:586: in execute
    persistence.save_obj(
.........................................................../cms/lib/python3.11.../sqlalchemy/orm/persistence.py:239: in save_obj
    _emit_insert_statements(
.........................................................../cms/lib/python3.11.../sqlalchemy/orm/persistence.py:1135: in _emit_insert_statements
    result = cached_connections[connection].execute(
.........................................................../cms/lib/python3.11.../sqlalchemy/engine/base.py:1011: in execute
    return meth(self, multiparams, params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
.........................................................../cms/lib/python3.11.../sqlalchemy/sql/elements.py:298: in _execute_on_connection
    return connection._execute_clauseelement(self, multiparams, params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
.........................................................../cms/lib/python3.11.../sqlalchemy/engine/base.py:1124: in _execute_clauseelement
    ret = self._execute_context(
.........................................................../cms/lib/python3.11.../sqlalchemy/engine/base.py:1316: in _execute_context
    self._handle_dbapi_exception(
.........................................................../cms/lib/python3.11.../sqlalchemy/engine/base.py:1510: in _handle_dbapi_exception
    util.raise_(
.........................................................../cms/lib/python3.11.../sqlalchemy/util/compat.py:182: in raise_
    raise exception
.........................................................../cms/lib/python3.11.../sqlalchemy/engine/base.py:1276: in _execute_context
    self.dialect.do_execute(
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <sqlalchemy.dialects.postgresql.psycopg2.PGDialect_psycopg2 object at 0x7f75f2c9dc90>
cursor = <cursor object at 0x7f75ef5064d0; closed: -1>
statement = 'INSERT INTO participations (ip, starting_time, delay_time, extra_time, password, hidden, unrestricted, contest_id, us...d)s, %(hidden)s, %(unrestricted)s, %(contest_id)s, %(user_id)s, %(group_id)s, %(team_id)s) RETURNING participations.id'
parameters = {'contest_id': 4, 'delay_time': datetime.timedelta(0), 'extra_time': datetime.timedelta(0), 'group_id': None, ...}
context = <sqlalchemy.dialects.postgresql.psycopg2.PGExecutionContext_psycopg2 object at 0x7f75ef4fc610>

    def do_execute(self, cursor, statement, parameters, context=None):
>       cursor.execute(statement, parameters)
E       sqlalchemy.exc.IntegrityError: (psycopg2.errors.NotNullViolation) null value in column "group_id" of relation "participations" violates not-null constraint
E       DETAIL:  Failing row contains (2, null, null, 00:00:00, 00:00:00, null, f, f, 4, 2, null, null).
E       
E       [SQL: INSERT INTO participations (ip, starting_time, delay_time, extra_time, password, hidden, unrestricted, contest_id, user_id, group_id, team_id) VALUES (CAST(%(ip)s AS CIDR[])::CIDR[], %(starting_time)s, %(delay_time)s, %(extra_time)s, %(password)s, %(hidden)s, %(unrestricted)s, %(contest_id)s, %(user_id)s, %(group_id)s, %(team_id)s) RETURNING participations.id]
E       [parameters: {'ip': None, 'starting_time': None, 'delay_time': datetime.timedelta(0), 'extra_time': datetime.timedelta(0), 'password': None, 'hidden': False, 'unrestricted': False, 'contest_id': 4, 'user_id': 2, 'group_id': None, 'team_id': None}]
E       (Background on this error at: http://sqlalche..../e/13/gkpj)

.........................................................../cms/lib/python3.11.../sqlalchemy/engine/default.py:608: IntegrityError

cmstestsuite/unit_tests/cmscontrib/DumpImporterTest.py::TestDumpImporter::test_import

Stack Traces | 0.262s run time

self = <sqlalchemy.engine.base.Connection object at 0x7f75f07d0990>
dialect = <sqlalchemy.dialects.postgresql.psycopg2.PGDialect_psycopg2 object at 0x7f75f2c9dc90>
constructor = <bound method DefaultExecutionContext._init_compiled of <class 'sqlalchemy.dialects.postgresql.psycopg2.PGExecutionContext_psycopg2'>>
statement = 'INSERT INTO participations (ip, starting_time, delay_time, extra_time, password, hidden, unrestricted, contest_id, us...d)s, %(hidden)s, %(unrestricted)s, %(contest_id)s, %(user_id)s, %(group_id)s, %(team_id)s) RETURNING participations.id'
parameters = {'contest_id': 2, 'delay_time': datetime.timedelta(0), 'extra_time': datetime.timedelta(0), 'group_id': None, ...}
args = (<sqlalchemy.dialects.postgresql.psycopg2.PGCompiler_psycopg2 object at 0x7f75f07a30d0>, [{'contest_id': 2, 'delay_time': datetime.timedelta(0), 'extra_time': datetime.timedelta(0), 'group_id': None, ...}])
conn = <sqlalchemy.pool.base._ConnectionFairy object at 0x7f75f08d3290>
context = <sqlalchemy.dialects.postgresql.psycopg2.PGExecutionContext_psycopg2 object at 0x7f75ef6581d0>

    def _execute_context(
        self, dialect, constructor, statement, parameters, *args
    ):
        """Create an :class:`.ExecutionContext` and execute, returning
        a :class:`_engine.ResultProxy`.
    
        """
    
        try:
            try:
                conn = self.__connection
            except AttributeError:
                # escape "except AttributeError" before revalidating
                # to prevent misleading stacktraces in Py3K
                conn = None
            if conn is None:
                conn = self._revalidate_connection()
    
            context = constructor(dialect, self, conn, *args)
        except BaseException as e:
            self._handle_dbapi_exception(
                e, util.text_type(statement), parameters, None, None
            )
    
        if context.compiled:
            context.pre_exec()
    
        cursor, statement, parameters = (
            context.cursor,
            context.statement,
            context.parameters,
        )
    
        if not context.executemany:
            parameters = parameters[0]
    
        if self._has_events or self.engine._has_events:
            for fn in self.dispatch.before_cursor_execute:
                statement, parameters = fn(
                    self,
                    cursor,
                    statement,
                    parameters,
                    context,
                    context.executemany,
                )
    
        if self._echo:
            self.engine.logger.info(statement)
            if not self.engine.hide_parameters:
                self.engine.logger.info(
                    "%r",
                    sql_util._repr_params(
                        parameters, batches=10, ismulti=context.executemany
                    ),
                )
            else:
                self.engine.logger.info(
                    "[SQL parameters hidden due to hide_parameters=True]"
                )
    
        evt_handled = False
        try:
            if context.executemany:
                if self.dialect._has_events:
                    for fn in self.dialect.dispatch.do_executemany:
                        if fn(cursor, statement, parameters, context):
                            evt_handled = True
                            break
                if not evt_handled:
                    self.dialect.do_executemany(
                        cursor, statement, parameters, context
                    )
            elif not parameters and context.no_parameters:
                if self.dialect._has_events:
                    for fn in self.dialect.dispatch.do_execute_no_params:
                        if fn(cursor, statement, context):
                            evt_handled = True
                            break
                if not evt_handled:
                    self.dialect.do_execute_no_params(
                        cursor, statement, context
                    )
            else:
                if self.dialect._has_events:
                    for fn in self.dialect.dispatch.do_execute:
                        if fn(cursor, statement, parameters, context):
                            evt_handled = True
                            break
                if not evt_handled:
>                   self.dialect.do_execute(
                        cursor, statement, parameters, context
                    )

.........................................................../cms/lib/python3.11.../sqlalchemy/engine/base.py:1276: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <sqlalchemy.dialects.postgresql.psycopg2.PGDialect_psycopg2 object at 0x7f75f2c9dc90>
cursor = <cursor object at 0x7f75f397e5c0; closed: -1>
statement = 'INSERT INTO participations (ip, starting_time, delay_time, extra_time, password, hidden, unrestricted, contest_id, us...d)s, %(hidden)s, %(unrestricted)s, %(contest_id)s, %(user_id)s, %(group_id)s, %(team_id)s) RETURNING participations.id'
parameters = {'contest_id': 2, 'delay_time': datetime.timedelta(0), 'extra_time': datetime.timedelta(0), 'group_id': None, ...}
context = <sqlalchemy.dialects.postgresql.psycopg2.PGExecutionContext_psycopg2 object at 0x7f75ef6581d0>

    def do_execute(self, cursor, statement, parameters, context=None):
>       cursor.execute(statement, parameters)
E       psycopg2.errors.NotNullViolation: null value in column "group_id" of relation "participations" violates not-null constraint
E       DETAIL:  Failing row contains (1, null, null, 00:00:00, 00:00:00, null, f, f, 2, 1, null, null).

.........................................................../cms/lib/python3.11.../sqlalchemy/engine/default.py:608: NotNullViolation

The above exception was the direct cause of the following exception:

self = <DumpImporterTest.TestDumpImporter testMethod=test_import>

    def test_import(self):
        """Test importing everything, while keeping the existing contest."""
        self.write_dump(TestDumpImporter.DUMP)
        self.write_files(TestDumpImporter.FILES)
>       self.assertTrue(self.do_import())
                        ^^^^^^^^^^^^^^^^

.../unit_tests/cmscontrib/DumpImporterTest.py:224: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
.../unit_tests/cmscontrib/DumpImporterTest.py:150: in do_import
    skip_users=skip_users).do_import()
                           ^^^^^^^^^^^
cmscontrib/DumpImporter.py:312: in do_import
    session.flush()
.........................................................../cms/lib/python3.11.../sqlalchemy/orm/session.py:2540: in flush
    self._flush(objects)
.........................................................../cms/lib/python3.11.../sqlalchemy/orm/session.py:2681: in _flush
    with util.safe_reraise():
.........................................................../cms/lib/python3.11.../sqlalchemy/util/langhelpers.py:68: in __exit__
    compat.raise_(
.........................................................../cms/lib/python3.11.../sqlalchemy/util/compat.py:182: in raise_
    raise exception
.........................................................../cms/lib/python3.11.../sqlalchemy/orm/session.py:2642: in _flush
    flush_context.execute()
.........................................................../cms/lib/python3.11.../sqlalchemy/orm/unitofwork.py:422: in execute
    rec.execute(self)
.........................................................../cms/lib/python3.11.../sqlalchemy/orm/unitofwork.py:586: in execute
    persistence.save_obj(
.........................................................../cms/lib/python3.11.../sqlalchemy/orm/persistence.py:239: in save_obj
    _emit_insert_statements(
.........................................................../cms/lib/python3.11.../sqlalchemy/orm/persistence.py:1135: in _emit_insert_statements
    result = cached_connections[connection].execute(
.........................................................../cms/lib/python3.11.../sqlalchemy/engine/base.py:1011: in execute
    return meth(self, multiparams, params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
.........................................................../cms/lib/python3.11.../sqlalchemy/sql/elements.py:298: in _execute_on_connection
    return connection._execute_clauseelement(self, multiparams, params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
.........................................................../cms/lib/python3.11.../sqlalchemy/engine/base.py:1124: in _execute_clauseelement
    ret = self._execute_context(
.........................................................../cms/lib/python3.11.../sqlalchemy/engine/base.py:1316: in _execute_context
    self._handle_dbapi_exception(
.........................................................../cms/lib/python3.11.../sqlalchemy/engine/base.py:1510: in _handle_dbapi_exception
    util.raise_(
.........................................................../cms/lib/python3.11.../sqlalchemy/util/compat.py:182: in raise_
    raise exception
.........................................................../cms/lib/python3.11.../sqlalchemy/engine/base.py:1276: in _execute_context
    self.dialect.do_execute(
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <sqlalchemy.dialects.postgresql.psycopg2.PGDialect_psycopg2 object at 0x7f75f2c9dc90>
cursor = <cursor object at 0x7f75f397e5c0; closed: -1>
statement = 'INSERT INTO participations (ip, starting_time, delay_time, extra_time, password, hidden, unrestricted, contest_id, us...d)s, %(hidden)s, %(unrestricted)s, %(contest_id)s, %(user_id)s, %(group_id)s, %(team_id)s) RETURNING participations.id'
parameters = {'contest_id': 2, 'delay_time': datetime.timedelta(0), 'extra_time': datetime.timedelta(0), 'group_id': None, ...}
context = <sqlalchemy.dialects.postgresql.psycopg2.PGExecutionContext_psycopg2 object at 0x7f75ef6581d0>

    def do_execute(self, cursor, statement, parameters, context=None):
>       cursor.execute(statement, parameters)
E       sqlalchemy.exc.IntegrityError: (psycopg2.errors.NotNullViolation) null value in column "group_id" of relation "participations" violates not-null constraint
E       DETAIL:  Failing row contains (1, null, null, 00:00:00, 00:00:00, null, f, f, 2, 1, null, null).
E       
E       [SQL: INSERT INTO participations (ip, starting_time, delay_time, extra_time, password, hidden, unrestricted, contest_id, user_id, group_id, team_id) VALUES (CAST(%(ip)s AS CIDR[])::CIDR[], %(starting_time)s, %(delay_time)s, %(extra_time)s, %(password)s, %(hidden)s, %(unrestricted)s, %(contest_id)s, %(user_id)s, %(group_id)s, %(team_id)s) RETURNING participations.id]
E       [parameters: {'ip': None, 'starting_time': None, 'delay_time': datetime.timedelta(0), 'extra_time': datetime.timedelta(0), 'password': None, 'hidden': False, 'unrestricted': False, 'contest_id': 2, 'user_id': 1, 'group_id': None, 'team_id': None}]
E       (Background on this error at: http://sqlalche..../e/13/gkpj)

.........................................................../cms/lib/python3.11.../sqlalchemy/engine/default.py:608: IntegrityError

cmstestsuite/unit_tests/cmscontrib/DumpImporterTest.py::TestDumpImporter::test_import_with_drop

Stack Traces | 0.321s run time

self = <sqlalchemy.engine.base.Connection object at 0x7f75ef51b110>
dialect = <sqlalchemy.dialects.postgresql.psycopg2.PGDialect_psycopg2 object at 0x7f75f2c9dc90>
constructor = <bound method DefaultExecutionContext._init_compiled of <class 'sqlalchemy.dialects.postgresql.psycopg2.PGExecutionContext_psycopg2'>>
statement = 'INSERT INTO participations (ip, starting_time, delay_time, extra_time, password, hidden, unrestricted, contest_id, us...d)s, %(hidden)s, %(unrestricted)s, %(contest_id)s, %(user_id)s, %(group_id)s, %(team_id)s) RETURNING participations.id'
parameters = {'contest_id': 1, 'delay_time': datetime.timedelta(0), 'extra_time': datetime.timedelta(0), 'group_id': None, ...}
args = (<sqlalchemy.dialects.postgresql.psycopg2.PGCompiler_psycopg2 object at 0x7f75f07a30d0>, [{'contest_id': 1, 'delay_time': datetime.timedelta(0), 'extra_time': datetime.timedelta(0), 'group_id': None, ...}])
conn = <sqlalchemy.pool.base._ConnectionFairy object at 0x7f75ef518dd0>
context = <sqlalchemy.dialects.postgresql.psycopg2.PGExecutionContext_psycopg2 object at 0x7f75f17b0a50>

    def _execute_context(
        self, dialect, constructor, statement, parameters, *args
    ):
        """Create an :class:`.ExecutionContext` and execute, returning
        a :class:`_engine.ResultProxy`.
    
        """
    
        try:
            try:
                conn = self.__connection
            except AttributeError:
                # escape "except AttributeError" before revalidating
                # to prevent misleading stacktraces in Py3K
                conn = None
            if conn is None:
                conn = self._revalidate_connection()
    
            context = constructor(dialect, self, conn, *args)
        except BaseException as e:
            self._handle_dbapi_exception(
                e, util.text_type(statement), parameters, None, None
            )
    
        if context.compiled:
            context.pre_exec()
    
        cursor, statement, parameters = (
            context.cursor,
            context.statement,
            context.parameters,
        )
    
        if not context.executemany:
            parameters = parameters[0]
    
        if self._has_events or self.engine._has_events:
            for fn in self.dispatch.before_cursor_execute:
                statement, parameters = fn(
                    self,
                    cursor,
                    statement,
                    parameters,
                    context,
                    context.executemany,
                )
    
        if self._echo:
            self.engine.logger.info(statement)
            if not self.engine.hide_parameters:
                self.engine.logger.info(
                    "%r",
                    sql_util._repr_params(
                        parameters, batches=10, ismulti=context.executemany
                    ),
                )
            else:
                self.engine.logger.info(
                    "[SQL parameters hidden due to hide_parameters=True]"
                )
    
        evt_handled = False
        try:
            if context.executemany:
                if self.dialect._has_events:
                    for fn in self.dialect.dispatch.do_executemany:
                        if fn(cursor, statement, parameters, context):
                            evt_handled = True
                            break
                if not evt_handled:
                    self.dialect.do_executemany(
                        cursor, statement, parameters, context
                    )
            elif not parameters and context.no_parameters:
                if self.dialect._has_events:
                    for fn in self.dialect.dispatch.do_execute_no_params:
                        if fn(cursor, statement, context):
                            evt_handled = True
                            break
                if not evt_handled:
                    self.dialect.do_execute_no_params(
                        cursor, statement, context
                    )
            else:
                if self.dialect._has_events:
                    for fn in self.dialect.dispatch.do_execute:
                        if fn(cursor, statement, parameters, context):
                            evt_handled = True
                            break
                if not evt_handled:
>                   self.dialect.do_execute(
                        cursor, statement, parameters, context
                    )

.........................................................../cms/lib/python3.11.../sqlalchemy/engine/base.py:1276: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <sqlalchemy.dialects.postgresql.psycopg2.PGDialect_psycopg2 object at 0x7f75f2c9dc90>
cursor = <cursor object at 0x7f75ef506e30; closed: -1>
statement = 'INSERT INTO participations (ip, starting_time, delay_time, extra_time, password, hidden, unrestricted, contest_id, us...d)s, %(hidden)s, %(unrestricted)s, %(contest_id)s, %(user_id)s, %(group_id)s, %(team_id)s) RETURNING participations.id'
parameters = {'contest_id': 1, 'delay_time': datetime.timedelta(0), 'extra_time': datetime.timedelta(0), 'group_id': None, ...}
context = <sqlalchemy.dialects.postgresql.psycopg2.PGExecutionContext_psycopg2 object at 0x7f75f17b0a50>

    def do_execute(self, cursor, statement, parameters, context=None):
>       cursor.execute(statement, parameters)
E       psycopg2.errors.NotNullViolation: null value in column "group_id" of relation "participations" violates not-null constraint
E       DETAIL:  Failing row contains (1, null, null, 00:00:00, 00:00:00, null, f, f, 1, 1, null, null).

.........................................................../cms/lib/python3.11.../sqlalchemy/engine/default.py:608: NotNullViolation

The above exception was the direct cause of the following exception:

self = <DumpImporterTest.TestDumpImporter testMethod=test_import_with_drop>

    def test_import_with_drop(self):
        """Test importing everything, but dropping existing data."""
        self.write_dump(TestDumpImporter.DUMP)
        self.write_files(TestDumpImporter.FILES)
    
        # Need to close the session and reopen it, otherwise the drop hangs.
        self.session.close()
>       self.assertTrue(self.do_import(drop=True))
                        ^^^^^^^^^^^^^^^^^^^^^^^^^

.../unit_tests/cmscontrib/DumpImporterTest.py:244: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
.../unit_tests/cmscontrib/DumpImporterTest.py:150: in do_import
    skip_users=skip_users).do_import()
                           ^^^^^^^^^^^
cmscontrib/DumpImporter.py:312: in do_import
    session.flush()
.........................................................../cms/lib/python3.11.../sqlalchemy/orm/session.py:2540: in flush
    self._flush(objects)
.........................................................../cms/lib/python3.11.../sqlalchemy/orm/session.py:2681: in _flush
    with util.safe_reraise():
.........................................................../cms/lib/python3.11.../sqlalchemy/util/langhelpers.py:68: in __exit__
    compat.raise_(
.........................................................../cms/lib/python3.11.../sqlalchemy/util/compat.py:182: in raise_
    raise exception
.........................................................../cms/lib/python3.11.../sqlalchemy/orm/session.py:2642: in _flush
    flush_context.execute()
.........................................................../cms/lib/python3.11.../sqlalchemy/orm/unitofwork.py:422: in execute
    rec.execute(self)
.........................................................../cms/lib/python3.11.../sqlalchemy/orm/unitofwork.py:586: in execute
    persistence.save_obj(
.........................................................../cms/lib/python3.11.../sqlalchemy/orm/persistence.py:239: in save_obj
    _emit_insert_statements(
.........................................................../cms/lib/python3.11.../sqlalchemy/orm/persistence.py:1135: in _emit_insert_statements
    result = cached_connections[connection].execute(
.........................................................../cms/lib/python3.11.../sqlalchemy/engine/base.py:1011: in execute
    return meth(self, multiparams, params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
.........................................................../cms/lib/python3.11.../sqlalchemy/sql/elements.py:298: in _execute_on_connection
    return connection._execute_clauseelement(self, multiparams, params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
.........................................................../cms/lib/python3.11.../sqlalchemy/engine/base.py:1124: in _execute_clauseelement
    ret = self._execute_context(
.........................................................../cms/lib/python3.11.../sqlalchemy/engine/base.py:1316: in _execute_context
    self._handle_dbapi_exception(
.........................................................../cms/lib/python3.11.../sqlalchemy/engine/base.py:1510: in _handle_dbapi_exception
    util.raise_(
.........................................................../cms/lib/python3.11.../sqlalchemy/util/compat.py:182: in raise_
    raise exception
.........................................................../cms/lib/python3.11.../sqlalchemy/engine/base.py:1276: in _execute_context
    self.dialect.do_execute(
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <sqlalchemy.dialects.postgresql.psycopg2.PGDialect_psycopg2 object at 0x7f75f2c9dc90>
cursor = <cursor object at 0x7f75ef506e30; closed: -1>
statement = 'INSERT INTO participations (ip, starting_time, delay_time, extra_time, password, hidden, unrestricted, contest_id, us...d)s, %(hidden)s, %(unrestricted)s, %(contest_id)s, %(user_id)s, %(group_id)s, %(team_id)s) RETURNING participations.id'
parameters = {'contest_id': 1, 'delay_time': datetime.timedelta(0), 'extra_time': datetime.timedelta(0), 'group_id': None, ...}
context = <sqlalchemy.dialects.postgresql.psycopg2.PGExecutionContext_psycopg2 object at 0x7f75f17b0a50>

    def do_execute(self, cursor, statement, parameters, context=None):
>       cursor.execute(statement, parameters)
E       sqlalchemy.exc.IntegrityError: (psycopg2.errors.NotNullViolation) null value in column "group_id" of relation "participations" violates not-null constraint
E       DETAIL:  Failing row contains (1, null, null, 00:00:00, 00:00:00, null, f, f, 1, 1, null, null).
E       
E       [SQL: INSERT INTO participations (ip, starting_time, delay_time, extra_time, password, hidden, unrestricted, contest_id, user_id, group_id, team_id) VALUES (CAST(%(ip)s AS CIDR[])::CIDR[], %(starting_time)s, %(delay_time)s, %(extra_time)s, %(password)s, %(hidden)s, %(unrestricted)s, %(contest_id)s, %(user_id)s, %(group_id)s, %(team_id)s) RETURNING participations.id]
E       [parameters: {'ip': None, 'starting_time': None, 'delay_time': datetime.timedelta(0), 'extra_time': datetime.timedelta(0), 'password': None, 'hidden': False, 'unrestricted': False, 'contest_id': 1, 'user_id': 1, 'group_id': None, 'team_id': None}]
E       (Background on this error at: http://sqlalche..../e/13/gkpj)

.........................................................../cms/lib/python3.11.../sqlalchemy/engine/default.py:608: IntegrityError

cmstestsuite/unit_tests/schema_diff_test.py::TestSchemaDiff::test_schema_diff

Stack Traces | 0.636s run time

self = <cmstestsuite.unit_tests.schema_diff_test.TestSchemaDiff testMethod=test_schema_diff>

    def test_schema_diff(self):
        dirname = os.path.dirname(__file__)
        schema_file = os.path.join(dirname, "schema_v1.5.sql")
        updater_file = os.path.join(dirname, "../...../cmscontrib/updaters/update_from_1.5.sql")
        updated_schema = split_schema(get_updated_schema(schema_file, updater_file))
        fresh_schema = split_schema(get_fresh_schema())
        errors = compare_schemas(updated_schema, fresh_schema)
        self.longMessage = False
>       self.assertTrue(errors == "", errors)
E       AssertionError: Statement differs between updated and fresh schema:
E       CREATE TABLE public.contests (
E       -     CONSTRAINT contests_check CHECK ((start <= stop)),
E       -     CONSTRAINT contests_check1 CHECK ((stop <= analysis_start)),
E       -     CONSTRAINT contests_check2 CHECK ((analysis_start <= analysis_stop)),
E       -     CONSTRAINT contests_check3 CHECK ((token_gen_initial <= token_gen_max)),
E       ?                              -
E       +     CONSTRAINT contests_check CHECK ((token_gen_initial <= token_gen_max)),
E             CONSTRAINT contests_max_submission_number_check CHECK ((max_submission_number > 0)),
E             CONSTRAINT contests_max_user_test_number_check CHECK ((max_user_test_number > 0)),
E             CONSTRAINT contests_min_submission_interval_check CHECK ((min_submission_interval > '00:00:00'::interval)),
E             CONSTRAINT contests_min_submission_interval_grace_period_check CHECK ((min_submission_interval_grace_period > '00:00:00'::interval)),
E             CONSTRAINT contests_min_user_test_interval_check CHECK ((min_user_test_interval > '00:00:00'::interval)),
E             CONSTRAINT contests_per_user_time_check CHECK ((per_user_time >= '00:00:00'::interval)),
E             CONSTRAINT contests_score_precision_check CHECK ((score_precision >= 0)),
E             CONSTRAINT contests_token_gen_initial_check CHECK ((token_gen_initial >= 0)),
E             CONSTRAINT contests_token_gen_interval_check CHECK ((token_gen_interval > '00:00:00'::interval)),
E             CONSTRAINT contests_token_gen_max_check CHECK ((token_gen_max > 0)),
E             CONSTRAINT contests_token_gen_number_check CHECK ((token_gen_number >= 0)),
E             CONSTRAINT contests_token_max_number_check CHECK ((token_max_number > 0)),
E             CONSTRAINT contests_token_min_interval_check CHECK ((token_min_interval >= '00:00:00'::interval)),
E             allow_password_authentication boolean NOT NULL,
E             allow_questions boolean NOT NULL,
E             allow_registration boolean NOT NULL,
E             allow_unofficial_submission_before_analysis_mode boolean NOT NULL,
E             allow_user_tests boolean NOT NULL,
E             allowed_localizations character varying[] NOT NULL,
E       -     analysis_enabled boolean NOT NULL,
E       -     analysis_start timestamp without time zone NOT NULL,
E       -     analysis_stop timestamp without time zone NOT NULL,
E             block_hidden_participations boolean NOT NULL,
E             description character varying NOT NULL,
E             id integer NOT NULL,
E             ip_autologin boolean NOT NULL,
E             ip_restriction boolean NOT NULL,
E             languages character varying[] NOT NULL,
E       +     main_group_id integer,
E             max_submission_number integer,
E             max_user_test_number integer,
E             min_submission_interval interval,
E             min_submission_interval_grace_period interval,
E             min_user_test_interval interval,
E             name public.codename NOT NULL,
E             per_user_time interval,
E             score_precision integer NOT NULL,
E       -     start timestamp without time zone NOT NULL,
E       -     stop timestamp without time zone NOT NULL,
E             submissions_download_allowed boolean NOT NULL,
E             timezone character varying,
E             token_gen_initial integer NOT NULL,
E             token_gen_interval interval NOT NULL,
E             token_gen_max integer,
E             token_gen_number integer NOT NULL,
E             token_max_number integer,
E             token_min_interval interval NOT NULL,
E             token_mode public.token_mode NOT NULL,
E         );
E       Statement differs between updated and fresh schema:
E       CREATE TABLE public.participations (
E             CONSTRAINT participations_delay_time_check CHECK ((delay_time >= '00:00:00'::interval)),
E             CONSTRAINT participations_extra_time_check CHECK ((extra_time >= '00:00:00'::interval)),
E             contest_id integer NOT NULL,
E             delay_time interval NOT NULL,
E             extra_time interval NOT NULL,
E       +     group_id integer NOT NULL,
E             hidden boolean NOT NULL,
E             id integer NOT NULL,
E             ip cidr[],
E             password character varying,
E             starting_time timestamp without time zone,
E             team_id integer,
E             unrestricted boolean NOT NULL,
E             user_id integer NOT NULL,
E         );
E       Fresh schema contains extra statement:
E       CREATE TABLE public.groups (
E           CONSTRAINT groups_check CHECK ((start <= stop)),
E           CONSTRAINT groups_check1 CHECK ((stop <= analysis_start)),
E           CONSTRAINT groups_check2 CHECK ((analysis_start <= analysis_stop)),
E           CONSTRAINT groups_per_user_time_check CHECK ((per_user_time >= '00:00:00'::interval)),
E           analysis_enabled boolean NOT NULL,
E           analysis_start timestamp without time zone NOT NULL,
E           analysis_stop timestamp without time zone NOT NULL,
E           contest_id integer,
E           id integer NOT NULL,
E           name character varying NOT NULL,
E           per_user_time interval,
E           start timestamp without time zone NOT NULL,
E           stop timestamp without time zone NOT NULL,
E       );
E       Fresh schema contains extra statement:
E       ALTER TABLE public.groups OWNER TO postgres;
E       Fresh schema contains extra statement:
E       CREATE SEQUENCE public.groups_id_seq
E           AS integer
E           START WITH 1
E           INCREMENT BY 1
E           NO MINVALUE
E           NO MAXVALUE
E           CACHE 1;
E       Fresh schema contains extra statement:
E       ALTER TABLE public.groups_id_seq OWNER TO postgres;
E       Fresh schema contains extra statement:
E       ALTER SEQUENCE public.groups_id_seq OWNED BY public.groups.id;
E       Fresh schema contains extra statement:
E       ALTER TABLE ONLY public.groups ALTER COLUMN id SET DEFAULT nextval('public.groups_id_seq'::regclass);
E       Fresh schema contains extra statement:
E       ALTER TABLE ONLY public.groups ADD CONSTRAINT groups_contest_id_name_key
E       UNIQUE (contest_id, name);
E       Fresh schema contains extra statement:
E       ALTER TABLE ONLY public.groups ADD CONSTRAINT groups_id_contest_id_key
E       UNIQUE (id, contest_id);
E       Fresh schema contains extra statement:
E       ALTER TABLE ONLY public.groups ADD CONSTRAINT groups_pkey
E       PRIMARY KEY (id);
E       Fresh schema contains extra statement:
E       CREATE INDEX ix_contests_main_group_id ON public.contests USING btree (main_group_id);
E       Fresh schema contains extra statement:
E       CREATE INDEX ix_groups_contest_id ON public.groups USING btree (contest_id);
E       Fresh schema contains extra statement:
E       CREATE INDEX ix_participations_group_id ON public.participations USING btree (group_id);
E       Fresh schema contains extra statement:
E       ALTER TABLE ONLY public.contests ADD CONSTRAINT fk_contest_main_group_id
E       FOREIGN KEY (main_group_id) REFERENCES public.groups(id) ON UPDATE CASCADE ON DELETE SET NULL;
E       Fresh schema contains extra statement:
E       ALTER TABLE ONLY public.groups ADD CONSTRAINT groups_contest_id_fkey
E       FOREIGN KEY (contest_id) REFERENCES public.contests(id) ON UPDATE CASCADE ON DELETE CASCADE;
E       Fresh schema contains extra statement:
E       ALTER TABLE ONLY public.participations ADD CONSTRAINT participations_group_id_contest_id_fkey
E       FOREIGN KEY (group_id, contest_id) REFERENCES public.groups(id, contest_id);
E       Fresh schema contains extra statement:
E       ALTER TABLE ONLY public.participations ADD CONSTRAINT participations_group_id_fkey
E       FOREIGN KEY (group_id) REFERENCES public.groups(id) ON UPDATE CASCADE ON DELETE CASCADE;

cmstestsuite/unit_tests/schema_diff_test.py:173: AssertionError

To view more test analytics, go to the Test Analytics Dashboard
_{📋 Got 3 mins? Take this short survey to help us improve Test Analytics.}

cms/db/user.py

prandla

noticed a few more things when reading through the code one more time, and poking around in AWS a bit. i think after addressing these comments, and the foreign key constraint mentioned in the previous discussion, the code itself is fine. again we still need a DumpUpdater and a sql updater. i can write those myself if you'd prefer (i assume you haven't needed to deal with those features in your fork...)

cms/db/user.py

prandla · 2026-02-15T19:04:24Z

cms/db/user.py

+        Integer,
+        ForeignKey(Contest.id,
+                   onupdate="CASCADE", ondelete="CASCADE"),
+        # nullable=False,


we have an explicit nullable=True or nullable=False on every column, so why not have it here too?

(i don't know who initially did this, but i like it being explicit because then i don't have to remember what sqlalchemy's default is :) )

cms/db/user.py

cms/server/admin/handlers/user.py

cmscontrib/AddParticipation.py

cmscontrib/loaders/italy_yaml.py

…ipations a @Property

t-lenz · 2026-02-16T15:52:59Z

This should address the requested changes (I also fixed a typo in the fallback page for AddGroupHandler). It would indeed be great if you could take care of the DumpUpdater and SQL updater. Thanks a lot!

prandla · 2026-02-18T03:15:28Z

oh, it seems like there might have been a bit of a miscommunication: if we're keeping Participation.contest_id then i don't see any reason not to also keep Contest.participations as a relationship. i reverted that part of your commit; please let me know if you actually had a reason for doing it like that.

also i changed Group.contest_id to be non-null, for i hope obvious reasons. and i fixed a few type hints.

i just realized i can't push changes into this PR since you made it from an organization repo, not a personal one. (github is very good software......) I've pushed my changes into the pr-groups branch of https://github.com/prandla/cms . you can pull from that branch and push it to the pr's branch, or i can just close this PR and create a new one from my own fork.

introduce user groups

ef2d6be

much of this is based on code originally written by @fagu and @magula

prandla requested changes Feb 11, 2026

View reviewed changes

j-bmn added 3 commits February 14, 2026 16:39

Fix missing name for implicitly-created main group

84b8ba3

Gender-neutral help text for participant's group

383aac6

Fix (most) failing tests

b1ee39d

Should fix all failures except DumpImporterTest and schema_diff_test (which both likely need more work).

Add --group argument and interactive chooser for cmsAddParticipation

402e1cc

prandla reviewed Feb 15, 2026

View reviewed changes

cms/db/user.py Outdated Show resolved Hide resolved

j-bmn and others added 5 commits February 15, 2026 15:02

fixup! Fix (most) failing tests

4a25bd6

Allow setting all main group parameters in the contest UI

5bc8b03

fixup! Allow setting all main group parameters in the contest UI

53acd80

explicitly declare backrefs in db, type hints

7430dbf

rename table "group" to "groups"

3ec9be7

prandla requested changes Feb 15, 2026

View reviewed changes

t-lenz added 2 commits February 16, 2026 14:53

ForeignKeyConstraint on Participation.contest_id, make Contest.partic…

1a8748a

…ipations a @Property

requested changes

8904bdf

Conversation

t-lenz commented Jan 18, 2026

Uh oh!

prandla left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

prandla Feb 11, 2026

Choose a reason for hiding this comment

Uh oh!

t-lenz Feb 15, 2026

Choose a reason for hiding this comment

Uh oh!

prandla Feb 15, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

codecov bot commented Feb 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

❌ 6 Tests Failed:

Uh oh!

Uh oh!

prandla left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

prandla Feb 15, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

t-lenz commented Feb 16, 2026

Uh oh!

prandla commented Feb 18, 2026

Uh oh!

Reviewers

Assignees

Labels

Milestone

Development

Uh oh!

3 participants

codecov bot commented Feb 14, 2026 •

edited

Loading