Feature Request: Allow config of `send_htlc_out` timeout via RPC

[willcl-ark]

With my low bandwidth (mesh net) application, I can send and receive all messages just fine, as timings do not appear to be an issue.

However, when specifically trying to pay an invoice, I encountered repeated errors, only manifested by C-Lightning closing the socket (to my proxy) with no additional messaging sent.

After much hunting, I have found the cause of the issue:

lightning/lightningd/peer_htlcs.c

Line 465 in 0495295

/* Give channel 30 seconds to commit (first) htlc. */

The timeout on L468 was causing payer to close the [edited]: s/channel/socket abruptly after 30 seconds, but I needed in the order of 180 seconds to send the first HTLC! Once I modified this to 300 seconds the failures were no longer.

I would propose that this either be increased, or if the UI associated with this is not desirable, allow it to be configured via an lightning-cli RPC command. This would allow our mesh net users to be able to pay lightning invoices in the future without needing to run a custom patched (and compiled) version of C-Lightning.

I don't see any security risk associated with doing this, as HTLCs are already "non-cancelable" once in-flight, and this only increased the time allowed for them to get there.

[darosior]

I wanted to submit a patch, but this was included (in #1811) apparently for performance reasons.
However we have a build for Android and this might be useful for low bandwidth devices (such as mobilephone) too. Maybe we could have a --slowmode option where such arbitrary timeouts would be adapted so that it does not hurt performances of ""regular"" nodes ?

[ZmnSCPxj]

Is it closing the channel? The code looks like it will only close the connection.

The same slowness would be tolerated if your node is forwarding, unless I suppose your nodes are not supporting forwarding (e.g. always unpublished channels).

[willcl-ark]

@ZmnSCPxj sorry you are correct, it's closing my socket, not channel! However it remains, without modification I cannot get my ~1500B initial message out within the 30 seconds before it retires.

[ZmnSCPxj]

My instinct is to add this config to command-line flags (which automatically adds it to configuration file options as well). Would that be desirable?

[rustyrussell]

Wow. If every lightning node takes 300 seconds to forward, the lightning network would become unusable.

The corner case is when its your own HTLC, in which case maybe it'd be OK to increase the default timeout.

Otherwise I think you'll need a config/cmdline option here as @ZmnSCPxj points out, which is pretty easy to add.

[willcl-ark]

@rustyrussell sure I agree with this by-and-large (nobody mention hold invoices! 😋) However, there are certainly use-cases I can think of in low bandwidth applications where having this as configurable could be useful

[ZmnSCPxj]

@rustyrussell should we restrict it only to own outgoing HTLCs? Or just apply the same timeout to all HTLCs (presumably if it is slow for own HTLCs it will also be slow on forwarded HTLCs).

Or make a separate flag --never-forward (though this can be done quite easily by never having a published channel).

[willcl-ark]

@ZmnSCPxj @rustyrussell, from my perspective -- and looking into my own future work -- I envision mesh network nodes to be each be their own participating lightning node (rather than a pure data relay), claiming routing fees from forwarded HTLCs from other mesh nodes as part of the route.

For this reason, I would prefer if there was a flag or RPC which could configure a global HTLC timeout parameter for environments such as this, where low latency is known, expected and understood.

For nodes who were not part of the mesh but tried a route which might travel through it, they would still only experience the regular timeouts so no difference would be observed for them (as far as I can tell)

[ZmnSCPxj]

I envision mesh network nodes to be each be their own participating lightning node (rather than a pure data relay), claiming routing fees from forwarded HTLCs from other mesh nodes as part of the route.

....snip.....

For nodes who were not part of the mesh but tried a route which might travel through it, they would still only experience the regular timeouts so no difference would be observed for them (as far as I can tell)

This would require a bridge node that can be accessed from both public network and mesh network, with itself having a high timeout, but which quickly responds to nodes on the public network.

But the concern we have here is that if a standard boring non-mesh node routes through the mesh network then routes out to another public bridge node (for example if two mesh networks get to connect to each other, because mesh) then the non-mesh node will find its payment attempt to be slow, taking entire minutes to pay.

Admittedly we have to face this problem some day anyway, once we have interplanetary Lightning network, but this seems to be something we need to consider now.

Looks like we want to have a limited-range channel publication after all...

[remyers]

Perhaps this would ultimately be solved by including 'communication delay' as something peers gossip about. Then your fast nodes could either opt to not route through high latency links (mesh, satellite, etc) or if they do, would have to set timeouts appropriately.