Dlists Schema and Foundations

The coding period began on the 25th of May and I have covered my first milestone with the help of my mentors Steve and Thomas!!

I started working on the foundation of dlists since last month and figured out it won't be so easy of a task to add the configuration_id in the first milestone as was expected from the proposal I had drawn up and assumed that it would sprawl and keep sprawling but Steve reassured me that it won't be much of a hassle and I am glad I communicated this with him because I wouldn't have known that it wasn't so difficult and I was being apprehensive for no reason >_<

Anyhoo, I started working right away on the logic and also the tests, keeping that in mind every thing I added I had it in my mind that I will be drawing up a testcase for it accordingly. Also, when I sent the first draft for review I figured out a few things that I will definitely follow through for the upcoming milestones. Learnt a few things about how git terminologies work because I was getting them a little mixed up and also learnt many good practices! It has been quite a good learning experience for me and I am so glad to be working on GNU Mailman.

My first Milestone lays down the foundational database architecture and REST infrastructure required to group mailing lists into parent-child hierarchies. Getting here required a massive architectural examination of how Mailman manages list settings, and I want to break down exactly how it works under the hood.

Stuff I worked on for this Milestone

Historically, a mailing list's configuration (like its archive policy or moderation rules) was tightly bound directly to the mailinglist database row. To allow child lists to inherit settings, I wrote an Alembic migration that makes a brand new mailinglistconfiguration table. The migration dynamically extracts all existing configuration data from old lists, ports it to the new table, and links them via a configuration_id foreign key. I also introduced a parent_id column to define the hierarchy.

After reviewing the deletion lifecycles with my mentors, I implemented an ondelete='SET NULL' constraint on the parent relationship. This ensures that if a parent list is deleted, Mailman won't recursively destroy all of its sublists it will orphan them so administrators don't lose data. Earlier I had taken the recursive deletion route but that was too harsh if you truly think about it, a child sublist becoming obsolete after you delete its parent...orphaning sounded like a better way to go, if an admin deletes a parent list, they shouldn't lose the entire history and membership of every child list. By switching to an 'orphan' model (ondelete='SET NULL'), the sublists are preserved and also admins are allowed to make whatever changes they wish to make.

Inheritance logic

I started my work on the inheritance engine Child lists will now maintain local nullable configuration rows. I implemented a parent-chasing lookup engine (modeled after Mailman’s native Member._lookup() mechanism) using Python properties.

At runtime, if a child list's local configuration field evaluates to None, Mailman automatically walks up the tree path (self -> parent -> grandparent) until it hits a concrete fallback configuration.

To expose this mechanism to admins, the REST API configuration endpoints were updated, custom getter/setters were used to natively understand this hierarchy.

If an admin wants a sublist to break away from its parent's rules, they can PATCH the endpoint with a new rule to create a local override. If they ever want to revert back, they can simply send a PATCH with the value inherit. The API natively takes this, wipes the local database row, and reattaches the list to the parent-chasing lookup chain. And this is how I beta tested it on a Mailman server instance, terminal output below:

(mm3-dlists) % curl --user restadmin:restpass -X DELETE http://localhost:8001/3.1/lists/topic.example.com
(mm3-dlists) % curl --user restadmin:restpass -X DELETE http://localhost:8001/3.1/lists/parent.example.com
(mm3-dlists) % curl --user restadmin:restpass -X POST http://localhost:8001/3.1/lists -d fqdn_listname=parent@example.com
(mm3-dlists) % curl --user restadmin:restpass -X PATCH http://localhost:8001/3.1/lists/parent.example.com/config -d dynamic_sublists_enabled=True -d archive_policy=private
(mm3-dlists) % curl --user restadmin:restpass -X POST http://localhost:8001/3.1/lists/parent.example.com/sublists -d fqdn_listname=topic@example.com
(mm3-dlists) % curl --user restadmin:restpass http://localhost:8001/3.1/lists/topic.example.com/config/archive_policy 
{"archive_policy": "private", "http_etag": ""34b8dbee29c9ffcf23aa881a97a74af5f79bba04""} 
(mm3-dlists) % curl --user restadmin:restpass -X PATCH http://localhost:8001/3.1/lists/topic.example.com/config/archive_policy -d archive_policy=never 
(mm3-dlists) % curl --user restadmin:restpass http://localhost:8001/3.1/lists/topic.example.com/config/archive_policy 
{"archive_policy": "never", "http_etag": ""97a53b5fa0afc34fe59fc74f12f8da2c0383dfd1""} 
(mm3-dlists) % curl --user restadmin:restpass -X PATCH http://localhost:8001/3.1/lists/topic.example.com/config/archive_policy -d archive_policy=inherit 
(mm3-dlists) % curl --user restadmin:restpass http://localhost:8001/3.1/lists/topic.example.com/config/archive_policy 
{"archive_policy": "private", "http_etag": ""34b8dbee29c9ffcf23aa881a97a74af5f79bba04""} 
(mm3-dlists) % curl --user restadmin:restpass http://localhost:8001/3.1/lists/parent.example.com/sublists | python3 -m json.tool 

% Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   503  100   503    0     0  27107      0 --:--:-- --:--:-- --:--:-- 27944

{ "start": 0, "total_size": 1, "entries": [ { "advertised": true, "display_name": "Topic", "dynamic_sublists_enabled": false, "sublist_count": 0, "fqdn_listname": "topic@example.com", "list_id": "topic.example.com", "list_name": "topic", "mail_host": "example.com", "member_count": 0, "volume": 1, "description": "", "self_link": "http://localhost:8001/3.1/lists/topic.example.com", "http_etag": ""e8e21e5922eb84d73d792487f1e2f80207e5ec0e"" } ], "http_etag": ""ae08c446e5dfb7632f86d46d5d352e46c1a20d6d"" }

Now I will be moving onward to work towards my next Milestone and understand the MTA layer to build the -new email command pipeline so users can make sublists organically via email.

Until next time :)