feat: Support Multiple session stores #277

ElijahAhianyo · 2025-04-06T03:49:58Z

This Pr lays the foundation for supporting multiple session stores. Currently, there are 4 supported store types:
Memory, Cache, File and DB(implementation to be added in a follow up PR).

Memory

This is the default store which stores the session data in a thread-safe hashmap. It is suitable for development environments

Toml Example

...
[middlewares.session.store]
type = "memory"

Config Example

use cot::config::{MiddlewareConfig, ProjectConfig, SessionMiddlewareConfig, SessionStoreConfig, SessionStoreTypeConfig};
...
struct FooProject;

impl Project for FooProject {

    fn config(&self, _config_name: &str) -> cot::Result<ProjectConfig> {
        Ok(ProjectConfig::builder()
            .middlewares(
                MiddlewareConfig::builder()
                    .session(SessionMiddlewareConfig::builder()
                        .secure(false)
                        .store(SessionStoreConfig::builder()
                            .store_type(SessionStoreTypeConfig::Memory)
                            .build()
                        )
                        .build())
                    .build(),
            )
            .build()
        )
    }
}

File Store

The file store persists sessions in a specified directory as files on a file system.

Toml Example

...
[middlewares.session.store]
type = "file"
path = "/path/to/session/storage"

Config Example

use std::path::PathBuf;
use cot::config::{MiddlewareConfig, ProjectConfig, SessionMiddlewareConfig, SessionStoreConfig, SessionStoreTypeConfig};

...
struct FooProject;

impl Project for FooProject {

    fn config(&self, _config_name: &str) -> cot::Result<ProjectConfig> {
        Ok(ProjectConfig::builder()
            .middlewares(
                MiddlewareConfig::builder()
                    .session(SessionMiddlewareConfig::builder()
                        .secure(false)
                        .store(SessionStoreConfig::builder()
                            .store_type(SessionStoreTypeConfig::File{ path : PathBuf::from("/path/to/dir")})
                            .build()
                        )
                        .build())
                    .build(),
            )
            .build()
        )
    }
}

Cache Store

The cache store uses a configured cache backed to persist data. This PR ships with support for a Redis backend. Support for other cache backends will be added in follow up PRs.
The cache store is gated behind the cache feature, while the redis implementation is gated behind the redis feature. To use the redis cache store, you will have to enable both.

Toml Example

...
[middlewares.session.store]
type = "cache"
uri = "redis://localhost:6379"

Config Example

use cot::config::{CacheUrl, MiddlewareConfig, ProjectConfig, SessionMiddlewareConfig, SessionStoreConfig, SessionStoreTypeConfig};

...
struct FooProject;

impl Project for FooProject {

    fn config(&self, _config_name: &str) -> cot::Result<ProjectConfig> {
        Ok(ProjectConfig::builder()
            .middlewares(
                MiddlewareConfig::builder()
                    .session(SessionMiddlewareConfig::builder()
                        .secure(false)
                        .store(SessionStoreConfig::builder()
                            .store_type(SessionStoreTypeConfig::Cache{ uri : CacheUri::from("redis://localhost:6379")})
                            .build()
                        )
                        .build())
                    .build(),
            )
            .build()
        )
    }
}

DB store

The DB store uses Cot's ORM to persist session data, Its implementation details will be added in a follow up PR.

codecov · 2025-04-06T04:14:51Z

Codecov Report

Attention: Patch coverage is 90.33816% with 60 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
cot/src/session/store/file.rs	86.45%	13 Missing and 13 partials ⚠️
cot/src/session/store/redis.rs	84.10%	10 Missing and 14 partials ⚠️
cot/src/config.rs	94.80%	8 Missing ⚠️
cot/src/middleware.rs	90.90%	2 Missing ⚠️

Flag	Coverage Δ
rust	`88.35% <90.33%> (+0.14%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines	Coverage Δ
cot/src/session.rs	`82.35% <ø> (ø)`
cot/src/session/store.rs	`100.00% <100.00%> (ø)`
cot/src/session/store/memory.rs	`100.00% <100.00%> (ø)`
cot/src/middleware.rs	`82.60% <90.90%> (+0.33%)`	⬆️
cot/src/config.rs	`95.73% <94.80%> (+4.38%)`	⬆️
cot/src/session/store/redis.rs	`84.10% <84.10%> (ø)`
cot/src/session/store/file.rs	`86.45% <86.45%> (ø)`

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

ElijahAhianyo · 2025-04-25T14:43:36Z

@m4tx @seqre this PR is not fully ready, but I'd love some initial feedback if this design is heading in the right direction or makes any sense to you at all.

Example usages

No session storage specified(using the admin example)

when the session_store is not provided, it defaults to the MemoryStore in the config

struct AdminProject;

impl Project for AdminProject {
   ...
    fn config(&self, _config_name: &str) -> cot::Result<ProjectConfig> {

        Ok(ProjectConfig::builder()
            .debug(true)
            .database(
                DatabaseConfig::builder()
                    .url("sqlite://db.sqlite3?mode=rwc")
                    .build(),
            )
            .auth_backend(AuthBackendConfig::Database)
            .middlewares(
                MiddlewareConfig::builder()
                    .session(SessionMiddlewareConfig::builder()
                        .secure(false)
                        .build())
                    .build(),
            )
            .build())
    }

    ...
}

Session storage provided using MemoryStore provided by Cot

struct AdminProject;

impl Project for AdminProject {
   ...
    fn config(&self, _config_name: &str) -> cot::Result<ProjectConfig> {

        Ok(ProjectConfig::builder()
            .debug(true)
            .database(
                DatabaseConfig::builder()
                    .url("sqlite://db.sqlite3?mode=rwc")
                    .build(),
            )
            .auth_backend(AuthBackendConfig::Database)
            .middlewares(
                MiddlewareConfig::builder()
                    .session(SessionMiddlewareConfig::builder()
                        .secure(false)
                        .session_store(Arc::new(MemoryStore::default()))
                        .build())
                    .build(),
            )
            .build())
    }

    ...
}

(Also haven't played around custom SessionStores yet)

m4tx · 2025-04-25T15:17:48Z

@ElijahAhianyo I think we should do this similarly to the Auth backend, i.e. the ProjectConfig should only include a serializable enum of possible choices for the Session store (Memory Store, ORM, Redis, etc., possibly with some inner configs as well), and the Bootstrapper should initialize a proper Session Store backend by calling a method (called session_store?) on the Project trait. The idea is that ProjectConfig must be serializable to/from a TOML file, and if you want to override the default behavior, you can do it by overriding a method in the Project trait. See the following snippets:

cot/cot/src/project.rs

Lines 335 to 351 in 70995f0

    
               fn auth_backend(&self, context: &AuthBackendContext) -> Arc<dyn AuthBackend> { 
        
                   #[expect(trivial_casts)] // cast to Arc<dyn AuthBackend> 
        
                   match &context.config().auth_backend { 
        
                       AuthBackendConfig::None => Arc::new(NoAuthBackend) as Arc<dyn AuthBackend>, 
        
                       #[cfg(feature = "db")] 
        
                       AuthBackendConfig::Database => Arc::new(DatabaseUserBackend::new( 
        
                           context 
        
                               .try_database() 
        
                               .expect( 
        
                                   "Database missing when constructing database auth backend. \ 
        
                                   Make sure the database config is set up correctly or disable \ 
        
                                   authentication in the config.", 
        
                               ) 
        
                               .clone(), 
        
                       )) as Arc<dyn AuthBackend>, 
        
                   } 
        
               }

cot/cot/src/project.rs

Lines 1240 to 1241 in 70995f0

    
           let auth_backend = self.project.auth_backend(&self.context); 
        
           let context = self.context.with_auth(auth_backend);

It's not ideal (mainly because it inflates the number of methods in the Project trait), but it's the best solution that we could come up with, allowing the users to use the "common" options easily, but not limiting them in what they can do. However, if you have ideas for a better design, I'm happy to discuss them.

ElijahAhianyo · 2025-04-28T11:16:09Z

@ElijahAhianyo I think we should do this similarly to the Auth backend, i.e. the ProjectConfig should only include a serializable enum of possible choices for the Session store (Memory Store, ORM, Redis, etc., possibly with some inner configs as well), and the Bootstrapper should initialize a proper Session Store backend by calling a method (called session_store?) on the Project trait. The idea is that ProjectConfig must be serializable to/from a TOML file, and if you want to override the default behavior, you can do it by overriding a method in the Project trait. See the following snippets:

cot/cot/src/project.rs

Lines 335 to 351 in 70995f0

fn auth_backend(&self, context: &AuthBackendContext) -> Arc<dyn AuthBackend> {

#[expect(trivial_casts)] // cast to Arc<dyn AuthBackend>

match &context.config().auth_backend {

AuthBackendConfig::None => Arc::new(NoAuthBackend) as Arc<dyn AuthBackend>,

#[cfg(feature = "db")]

AuthBackendConfig::Database => Arc::new(DatabaseUserBackend::new(

context

.try_database()

.expect(

"Database missing when constructing database auth backend. \

Make sure the database config is set up correctly or disable \

authentication in the config.",

)

.clone(),

)) as Arc<dyn AuthBackend>,

}

}

cot/cot/src/project.rs

Lines 1240 to 1241 in 70995f0

let auth_backend = self.project.auth_backend(&self.context);

let context = self.context.with_auth(auth_backend);

It's not ideal (mainly because it inflates the number of methods in the Project trait), but it's the best solution that we could come up with, allowing the users to use the "common" options easily, but not limiting them in what they can do. However, if you have ideas for a better design, I'm happy to discuss them.

Based on your suggestion, I did some research and now have a clearer picture. My plan is to start by sketching out what the TOML file might look like, then work my way down into implementation details.

I looked at how other frameworks handle this and identified two main patterns we canexplore:

1 a. Self-contained store config

You declare the store type and provide any necessary connection details (or file paths, in the case of file storage):

secret_key = "{{ dev_secret_key }}"

[database]
url = "sqlite://db.sqlite3?mode=rwc"

[auth_backend]
type = "database"

[middlewares]
live_reload.enabled = true

[middlewares.session]
secure = false

[middlewares.session.store]
type = "redis"
connection = "redis://"

1 b Per-store subsection

The approach in 1.a could get quite noisy(or maybe not) if different store types have some configs specific to them.

secret_key = "{{ dev_secret_key }}"

[database]
url = "sqlite://db.sqlite3?mode=rwc"

[auth_backend]
type = "database"

[middlewares]
live_reload.enabled = true

[middlewares.session]
secure = false

[middlewares.session.store]
type = "file"

[middlewares.session.store.file]
dir = "/tmp/"

2. Named connections

The first pattern forces you to re-declare connection details even if you’ve already defined them under [databases] or [caches](currently not implemented). It also assumes a single DB/cache backend. We may need to structurally change the API to support this:

secret_key = "{{ dev_secret_key }}"

[databases]
default = "postgres://…"

[caches]
redis_main = "redis://…"

[middlewares.session]
store      = "redis"
connection = "cache.redis_main"  # or, alternatively, use a separate `connection_source` field instead of dotted notation

Option 1 should be easy to implement given our current design. However, with option 2, do we have any plans to support multiple DBs or caches?

@m4tx let me also know if you've got other opinions on what the TOML should look like.

m4tx · 2025-04-30T18:13:25Z

@ElijahAhianyo

do we have any plans to support multiple DBs or caches?

Ah, that's a good question. Definitely yes, but certainly not in the near future.

After some consideration, I think it would make sense to have some sort of hybrid between 1a and 2. I can imagine the following typical use cases for the session backends:

in-memory store, file store - for development or super low traffic websites where performance and reliability don't matter
database (using Cot's ORM) - for development or low/medium traffic websites that need simplicity and reliability, but don't have high enough traffic to require caching
cache (Redis, MongoDB, etc.) - for high traffic websites that need caching for other stuff anyway

If we want to store sessions externally, we probably should already have an API for that. This doesn't apply for in-memory or file stores (because they're mainly development-focused anyway), but we already have a way to define a DB connection in the config (for use with the Cot's ORM) and the framework user shouldn't need to provide the same credentials twice, but the session store should rather use the Cot's ORM directly. The same should be true for Redis/Mongo/whatever cache – we don't have any Cache API at the moment, but I think we should at some point. And when we have the Cache API, we should just be able to use it to implement session stores.

So I think we could try to translate these use cases into Config files:

In-memory/file backend

[middlewares.session.store]
type = "memory"

[middlewares.session.store]
type = "file"
path = "sessions.db"

Database backend

[middlewares.session.store]
type = "database"  # will just use Cot's ORM

Redis

For now, something like so should be good enough:

[middlewares.session.store]
type = "redis"
url = "redis://localhost:6379"

But, when Cot gets a dedicated Cache API with its own connection settings, the above can be just changed to:

[middlewares.session.store]
type = "cache"  # similarly to "database", this just means we'll use Cot's Cache API
name = "cache1"  # optional; only needed if more than one cache is defined

Do you think the above makes sense?

Obviously, implementing a Cache API is out of scope for this PR, but if you'd like to do it as well, you're more than welcome to.

ElijahAhianyo · 2025-04-30T19:37:14Z

@ElijahAhianyo

do we have any plans to support multiple DBs or caches?

Ah, that's a good question. Definitely yes, but certainly not in the near future.

After some consideration, I think it would make sense to have some sort of hybrid between 1a and 2. I can imagine the following typical use cases for the session backends:

in-memory store, file store - for development or super low traffic websites where performance and reliability don't matter

database (using Cot's ORM) - for development or low/medium traffic websites that need simplicity and reliability, but don't have high enough traffic to require caching

cache (Redis, MongoDB, etc.) - for high traffic websites that need caching for other stuff anyway

If we want to store sessions externally, we probably should already have an API for that. This doesn't apply for in-memory or file stores (because they're mainly development-focused anyway), but we already have a way to define a DB connection in the config (for use with the Cot's ORM) and the framework user shouldn't need to provide the same credentials twice, but the session store should rather use the Cot's ORM directly. The same should be true for Redis/Mongo/whatever cache – we don't have any Cache API at the moment, but I think we should at some point. And when we have the Cache API, we should just be able to use it to implement session stores.

So I think we could try to translate these use cases into Config files:

In-memory/file backend
[middlewares.session.store]
type = "memory"
[middlewares.session.store]
type = "file"
path = "sessions.db"
Database backend
[middlewares.session.store]
type = "database"  # will just use Cot's ORM
Redis

For now, something like so should be good enough:
[middlewares.session.store]
type = "redis"
url = "redis://localhost:6379"
But, when Cot gets a dedicated Cache API with its own connection settings, the above can be just changed to:
[middlewares.session.store]
type = "cache"  # similarly to "database", this just means we'll use Cot's Cache API
name = "cache1"  # optional; only needed if more than one cache is defined
Do you think the above makes sense?

Obviously, implementing a Cache API is out of scope for this PR, but if you'd like to do it as well, you're more than welcome to.

@m4tx Yes, that makes sense. I can look into having a Cache API(we can create an issue to track this), in a separate PR—almost async with this one. Would you still want that designed with a singular backend in mind for now, and support multi later?

m4tx · 2025-04-30T21:27:37Z

@ElijahAhianyo actually, I think cases 1 and 3 from my answer can be merged together – there's nothing stopping us from implementing in-memory and file-backed caching. This would leave us with only two session backends that we would need to implement - database and cache.

I've created an issue to track the Cache API: #308.

Would you still want that designed with a singular backend in mind for now, and support multi later?

I think having just one backend is 100% fine for now - we can work on this to add support for multiple ones later, as this doesn't sound like an everyday use case anyways.

ElijahAhianyo · 2025-05-01T13:01:55Z

@ElijahAhianyo actually, I think cases 1 and 3 from my answer can be merged together – there's nothing stopping us from implementing in-memory and file-backed caching. This would leave us with only two session backends that we would need to implement - database and cache.

I've created an issue to track the Cache API: #308.

Would you still want that designed with a singular backend in mind for now, and support multi later?

I think having just one backend is 100% fine for now - we can work on this to add support for multiple ones later, as this doesn't sound like an everyday use case anyways.

@m4tx I see, just so we're on the same page on what the TOML file should look like in the context of this PR, If we have two(database and cache) store variants. How do we differentiate what cache type(redis, file, memcache, etc) it is from the TOML file?
Would you rather

Ask the user for more help by introducing another config knob to identify what cache type they want:

[middlewares.session.store]
type = "cache"
cache_type = "redis" # very verbose
url = "redis://localhost:6379"

Do some implicit gymnastics using the url/path provided to determine which cache the user wants:

[middlewares.session.store]
type = "cache"
url = "redis://localhost:6379" # will be infered as redis store from the uri

Any other option?

m4tx · 2025-05-01T13:15:21Z

@ElijahAhianyo I think option number 2 should be good enough, as it doesn't duplicate the data in the config, and it will be consistent with the ORM behavior. When creating the Cache API, we just need to make sure it's possible to register new cache backends (for instance, the cache backend may need to provide all the URL schemas it supports, and our code would just try to match the URL to all registered backends).

By the way, just to be sure, this is fine for now:

[middlewares.session.store]
type = "cache"
url = "redis://localhost:6379"

But eventually we'll want to have something closer to this (when we have the Cache API):

[cache]
URL = "redis://localhost:6379"

[middlewares.session.store]
type = "cache"

…y refactor

ElijahAhianyo · 2025-05-20T09:28:38Z

cot/src/config.rs

+            },
+            #[cfg(feature = "db")]
+            Self::Database => {
+                unimplemented!();


I ended up moving the DB implementation into another PR since there are some nuances to it; The DB option requires a migration model and generated migration, which I haven't gotten to the bottom of

ElijahAhianyo · 2025-05-26T10:29:30Z

@m4tx @seqre This should somewhat be ready for review, Let me know what your initial thoughts are

github-actions bot added the C-lib Crate: cot (main library crate) label Apr 6, 2025

ElijahAhianyo force-pushed the elijah/session-store-dynamic branch from f8e50cd to 3c6dbd6 Compare April 24, 2025 13:33

m4tx mentioned this pull request Apr 30, 2025

Add Cache API #308

Open

ElijahAhianyo and others added 14 commits May 13, 2025 12:52

Support Multiple session stores

60d3123

chore(pre-commit.ci): auto fixes from pre-commit hooks

dfdc3cf

remove debug super trait req

9971b69

allow setting the session_store in the config

05fbd04

chore(pre-commit.ci): auto fixes from pre-commit hooks

8b01daa

expose sessionstores directly

cfd8ff2

chore(pre-commit.ci): auto fixes from pre-commit hooks

31c6b8b

proof of concept for memory store

2c8e862

remove the migration mod for now

cf727d7

config was omitted

5740ef0

Sloppy implementation of memory, redis, file and db stores. Need heav…

a7adf0c

…y refactor

do a better job with the errors

e4f108b

be consistent with imports

9a37a95

add some initial tests

fe3713b

ElijahAhianyo force-pushed the elijah/session-store-dynamic branch from 1c7763d to fe3713b Compare May 13, 2025 13:03

pre-commit-ci bot and others added 3 commits May 13, 2025 13:04

chore(pre-commit.ci): auto fixes from pre-commit hooks

94202e0

remove some debug statements

baf127a

add more tests

bcf991e

ElijahAhianyo added 8 commits May 14, 2025 10:14

gate redis implementations behind redis feature

1bf248c

Merge branch 'master' into elijah/session-store-dynamic

04ee7a6

Add some docs, remove db impl since there are some nuances to it

1eba7b9

format

0a79ba2

revert admin eg features

8b1fe16

remove db

a137c8f

fix failing doc tests

1dae529

test using redis instance

f8955c1

github-actions bot added the A-ci Area: CI (Continuous Integration) label May 15, 2025

ElijahAhianyo added 2 commits May 15, 2025 13:25

ignore the redis tests, a better approach

6d43188

resolve some minor issues

b47f7ff

github-actions bot added the C-cli Crate: cot-cli (issues and Pull Requests related to Cot CLI) label May 15, 2025

ElijahAhianyo added 6 commits May 16, 2025 09:21

resolve some clippy tests

0a5913f

Merge branch 'master' into elijah/session-store-dynamic

a11f17e

Merge branch 'master' into elijah/session-store-dynamic

28ec5c7

possible fix for tests

27c8142

this should fix the CI

0435495

some minor improvements

e9233b2

ElijahAhianyo commented May 20, 2025

View reviewed changes

ElijahAhianyo added 2 commits May 23, 2025 11:03

add a few more tests

da370b0

Merge branch 'master' into elijah/session-store-dynamic

7d3148f

ElijahAhianyo changed the title ~~[WIP]Support Multiple session stores~~ feat:Support Multiple session stores May 26, 2025

ElijahAhianyo marked this pull request as ready for review May 26, 2025 10:26

ElijahAhianyo changed the title ~~feat:Support Multiple session stores~~ feat: Support Multiple session stores May 26, 2025

minor refactor

a226ccd

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

feat: Support Multiple session stores #277

feat: Support Multiple session stores #277

Uh oh!

ElijahAhianyo commented Apr 6, 2025 •

edited

Loading

Uh oh!

codecov bot commented Apr 6, 2025 •

edited

Loading

Uh oh!

ElijahAhianyo commented Apr 25, 2025

Uh oh!

m4tx commented Apr 25, 2025

Uh oh!

ElijahAhianyo commented Apr 28, 2025

Uh oh!

m4tx commented Apr 30, 2025

Uh oh!

ElijahAhianyo commented Apr 30, 2025

Uh oh!

m4tx commented Apr 30, 2025 •

edited

Loading

Uh oh!

ElijahAhianyo commented May 1, 2025 •

edited

Loading

Uh oh!

m4tx commented May 1, 2025

Uh oh!

ElijahAhianyo May 20, 2025

Uh oh!

ElijahAhianyo commented May 26, 2025 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

feat: Support Multiple session stores #277

Are you sure you want to change the base?

feat: Support Multiple session stores #277

Uh oh!

Conversation

ElijahAhianyo commented Apr 6, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Memory

Toml Example

Config Example

File Store

Toml Example

Config Example

Cache Store

Toml Example

Config Example

DB store

Uh oh!

codecov bot commented Apr 6, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

ElijahAhianyo commented Apr 25, 2025

Example usages

No session storage specified(using the admin example)

Session storage provided using MemoryStore provided by Cot

Uh oh!

m4tx commented Apr 25, 2025

Uh oh!

ElijahAhianyo commented Apr 28, 2025

1 a. Self-contained store config

1 b Per-store subsection

2. Named connections

Uh oh!

m4tx commented Apr 30, 2025

Uh oh!

ElijahAhianyo commented Apr 30, 2025

Uh oh!

m4tx commented Apr 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ElijahAhianyo commented May 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

m4tx commented May 1, 2025

Uh oh!

ElijahAhianyo May 20, 2025

Choose a reason for hiding this comment

Uh oh!

ElijahAhianyo commented May 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

ElijahAhianyo commented Apr 6, 2025 •

edited

Loading

codecov bot commented Apr 6, 2025 •

edited

Loading

m4tx commented Apr 30, 2025 •

edited

Loading

ElijahAhianyo commented May 1, 2025 •

edited

Loading

ElijahAhianyo commented May 26, 2025 •

edited

Loading