Upgrade extension to manifest v3

[wirednkod]

Based on the timeline given by Chrome concerning Manifest V2 support timeline, 2021 is the last year that new manifest v2 extensions will be accepted (starting January 17, 2022).

Thus it is probably needed to update extension to v3

Potential blocking problems to switching to manifest v3:

• Service workers cannot spawn workers
• Background script must be replaced with service worker which cannot be persistent without a hacky workaround

[tomaka]

Background script must be replaced with service worker which cannot be persistent without a hacky workaround

Copy-pasting below some of what I wrote down in an Element channel:

Some context

So, right now, the substrate-connect library works as follows:

• If the substrate-connect extension isn't installed, a web page that uses substrate-connect simply runs its own instance of smoldot.
• If the substrate-connect extension is installed, it is the extension that runs smoldot, and web pages that use substrate-connect instead connect to the extension and exchange JSON-RPC messages with the instance of smoldot in the extension.
  For reminder, the reason why we encourage users to install the extension are:

1. Smoldot is a bit heavy (2.6 MiB, compressed), which takes a couple of seconds to download and a couple of seconds to decompress and instantiate. Having the extension means that this only happens once when you download the extension (to download smoldot) and start your browser (to instantiate it).
2. When within an extension, opening WebSocket connections to non-HTTPS servers is allowed, while they are forbidden when outside of an extension.
3. The extension starts connecting to Polkadot, Kusama, Rococo and Westend when the browser starts, meaning that when a web page wants to connect to one of these four chains or their parachain, it will generally be the case that the extension is already fully synced. If the extension isn't installed, each web page has to sync each chain they connect to.
4. The extension saves in a cache the state of Polkadot, Kusama, Rococo and Westend, meaning that when the browser starts they resume syncing from where they were when their browser shut down. We could do the same for web pages, but since each web site has their own isolated cache, they would only resume syncing from where they were when they last visited this specific web site, which is way less useful.
5. In general, it's less CPU, memory, and bandwidth intensive to just have one connection to each blockchain from every tab, instead of multiple ones.

The problem

The substrate-connect extension uses manifests v2. There is now a manifest v3, and it is way more restrictive. Extensions that use manifests v2 will stop working the 1st of January 2023. In other words, if we do nothing, the substrate-connect extension will stop working at the end of the year.

Manifest v3 do not allow long-lived scripts in extensions. You can have scripts, but they are started "on demand" and get terminated afterwards. For example: a web page can send a message to the extension, a script starts, handles the message, and once it's done the script ends and everything the script might have created (such as WebSocket connections) gets killed.
It's actually very unclear what is possible or not, but the general intention is there: you can't have long-running scripts in extensions anymore, meaning, for example, no long-lived connections to the blockchain.
Trying to hack against this intention is extremely risky. Even if we find a hack that works, if it goes against this general policy, they might consider it a bug and fix it.

Another read: https://www.eff.org/deeplinks/2021/12/chrome-users-beware-manifest-v3-deceitful-and-threatening

What is possible

So what is it still possible to do in an extension?

What an extension can now do is act as a "shared programmable cache". An extension can basically intercept HTTP requests and override their responses (for example, to load the response from the browser's storage instead), and can receive messages from web pages and send back answers, and that's roughly it.
As I've explained above, once an HTTP response has been overriden, or once a message has been answered, the script that was dedicated to this specific HTTP request or message terminates.

It is still possible to run long processes. For example, an extension can start downloading data from the Internet in the background and store it, but it seems that this comes and will always come with limitations. Right now it seems that a script will always get killed after 5 minutes no matter what. 5 minutes seems long enough, but this is an implementation detail we can't rely on in the long term, and Google might decide that since they're against long-running script they would add more restrictions.

How do we adjust substrate-connect?

Given the constraint of not having long-running scripts, it's clear that we can't run the blockchain node in the extension anymore.
This doesn't, however, make an extension completely useless.

We can redesign the extension to handle the download part of point 1. (in the list at the start of this message), and point 4.
We could maybe still handle point 3. by starting smoldot when the browser stats, and letting it sync as far as possible before it gets killed.

Points 2 and 5, however, go out of the window.
Point 2 would become obsolete if we manage to make WebRTC work, so it shouldn't be a problem.

More details

I think that we should run an instance of smoldot in the content script of each individual web page that wants to connect to a chain.
For context, the content script is a piece of code that extensions can inject to a page. This content script is treated like a regular piece of JavaScript that the page would have loaded and can run long-running scripts.
The advantage of having a content script is that the extension is sure that its code isn't malicious: if a content script sends a message to the extension, the extension can trust this message to be honest.

This means, for example, that a smoldot instance in a content script can sync a chain, then send its sync status to the extension, which then sends it to the other content scripts.

One possibility we have could be to start one instance of smoldot in each content script and have each of these smoldot instances connect to each other as if they were part of the peer-to-peer network, and exchange "networking" messages through the extension.
Thanks to this, only one content script needs to be actually connected to a full node, and would serve the other content scripts of the other pages. If this "master content script" goes down, another one takes the relay. When a full node sends a block announce, it gets sent only to one content script, which in turns propagates it to the other content scripts.

[wirednkod]

Closed by #1272 #1280 #1295