Skip to content

Install takes forever(?) #6849

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
Sjors opened this issue Nov 7, 2023 · 10 comments
Closed

Install takes forever(?) #6849

Sjors opened this issue Nov 7, 2023 · 10 comments
Assignees
@Sjors @vincenzopalazzo
Labels
in diagnostic issue under diagnostic

Comments

@Sjors
Copy link
Contributor

Sjors commented Nov 7, 2023

I'm trying to install v23.11rc1 on Ubuntu 23.04, but the installer seems to get stuck.

git checkout v23.11rc1
git clean -dfx
./configure --prefix=/home/lightning
make -j7
make -j7 install # this seems to repeat all of make ???

Log for the latter command:

mkdir -p /home/lightning/bin
mkdir -p /home/lightning/libexec/c-lightning
mkdir -p /home/lightning/libexec/c-lightning/plugins
mkdir -p /home/lightning/share/man/man1
mkdir -p /home/lightning/share/man/man5
mkdir -p /home/lightning/share/man/man7
mkdir -p /home/lightning/share/man/man8
mkdir -p /home/lightning/share/doc/c-lightning
cargo build --quiet --bin cln-grpc
xgettext plugins/bkpr/statements_gettextgen.po
cc plugins/bkpr/account.c
cc plugins/bkpr/account_entry.c
cc plugins/bkpr/bookkeeper.c
cc plugins/bkpr/chain_event.c
cc plugins/bkpr/channel_event.c
cc plugins/bkpr/channelsapy.c
cc plugins/bkpr/db.c
cc plugins/bkpr/incomestmt.c
cc plugins/bkpr/onchain_fee.c
cc plugins/bkpr/recorder.c
sql-rewrite plugins/bkpr/db_sqlite3_sqlgen.c
sql-rewrite plugins/bkpr/db_postgres_sqlgen.c
cd external/lowdown && CC="cc" CFLAGS="" ./configure LDFLAGS="" PREFIX=`pwd`/../../external/build-x86_64-linux-gnu/lowdown-build/
config.log: writing...
GNU submake detected: using --no-print-directory
make[1]: warning: jobserver unavailable: using -j1.  Add '+' to parent make rule.
make[1]: warning: jobserver unavailable: using -j1.  Add '+' to parent make rule.
cc plugins/bkpr/db_sqlite3_sqlgen.c
configure.local: no (fully automatic configuration)
cc plugins/bkpr/db_postgres_sqlgen.c
arc4random: yes 
b64_ntop: yes (with -lresolv)
capsicum: no
ld plugins/bookkeeper
crypt: yes (with -lcrypt)
endian_h: yes 
err: no
explicit_bzero: yes 
fts: yes 
getexecname: no
getprogname: no
INFTIM: no
lib_socket: yes 
md5: yes (with -lmd)
memmem: yes 
memrchr: yes 
memset_s: no
mkfifoat: yes 
mknodat: yes 
osbyteorder_h: no
PATH_MAX: yes 
pledge: no
program_invocation_short_name: yes 
readpassphrase: no
reallocarray: yes 
recallocarray: no
sandbox_init: no
seccomp-filter: yes 
setresgid: yes 
setresuid: yes 
sha2: yes (with -lmd)
SOCK_NONBLOCK: yes 
static: yes 
strlcat: no
strlcpy: no
strndup: yes 
strnlen: yes 
strtonum: no
sys_byteorder_h: no
sys_endian_h: no
sys_mkdev_h: no
sys_sysmacros_h: yes 
sys_queue: no
sys_tree: no
unveil: no
WAIT_ANY: yes 
__progname: yes 
config.h: written
Makefile.configure: written
make -C external/lowdown install
make[1]: Entering directory '/home/lightning/src/lightning/external/lowdown'
cc -g -W -Wall -Wextra -Wmissing-prototypes -Wstrict-prototypes -Wwrite-strings -Wno-unused-parameter -fPIC   -c -o autolink.o autolink.c
cc -g -W -Wall -Wextra -Wmissing-prototypes -Wstrict-prototypes -Wwrite-strings -Wno-unused-parameter -fPIC   -c -o buffer.o buffer.c
cc -g -W -Wall -Wextra -Wmissing-prototypes -Wstrict-prototypes -Wwrite-strings -Wno-unused-parameter -fPIC   -c -o diff.o diff.c
cc -g -W -Wall -Wextra -Wmissing-prototypes -Wstrict-prototypes -Wwrite-strings -Wno-unused-parameter -fPIC   -c -o document.o document.c
cc -g -W -Wall -Wextra -Wmissing-prototypes -Wstrict-prototypes -Wwrite-strings -Wno-unused-parameter -fPIC   -c -o entity.o entity.c
cc -g -W -Wall -Wextra -Wmissing-prototypes -Wstrict-prototypes -Wwrite-strings -Wno-unused-parameter -fPIC   -c -o gemini.o gemini.c
cc -g -W -Wall -Wextra -Wmissing-prototypes -Wstrict-prototypes -Wwrite-strings -Wno-unused-parameter -fPIC   -c -o html.o html.c
cc -g -W -Wall -Wextra -Wmissing-prototypes -Wstrict-prototypes -Wwrite-strings -Wno-unused-parameter -fPIC   -c -o html_escape.o html_escape.c
cc -g -W -Wall -Wextra -Wmissing-prototypes -Wstrict-prototypes -Wwrite-strings -Wno-unused-parameter -fPIC   -c -o latex.o latex.c
cc -g -W -Wall -Wextra -Wmissing-prototypes -Wstrict-prototypes -Wwrite-strings -Wno-unused-parameter -fPIC   -c -o library.o library.c
cc -g -W -Wall -Wextra -Wmissing-prototypes -Wstrict-prototypes -Wwrite-strings -Wno-unused-parameter -fPIC   -c -o libdiff.o libdiff.c
cc -g -W -Wall -Wextra -Wmissing-prototypes -Wstrict-prototypes -Wwrite-strings -Wno-unused-parameter -fPIC   -c -o nroff.o nroff.c
cc -g -W -Wall -Wextra -Wmissing-prototypes -Wstrict-prototypes -Wwrite-strings -Wno-unused-parameter -fPIC   -c -o odt.o odt.c
cc -g -W -Wall -Wextra -Wmissing-prototypes -Wstrict-prototypes -Wwrite-strings -Wno-unused-parameter -fPIC   -c -o smartypants.o smartypants.c
cc -g -W -Wall -Wextra -Wmissing-prototypes -Wstrict-prototypes -Wwrite-strings -Wno-unused-parameter -fPIC   -c -o term.o term.c
cc -g -W -Wall -Wextra -Wmissing-prototypes -Wstrict-prototypes -Wwrite-strings -Wno-unused-parameter -fPIC   -c -o tree.o tree.c
cc -g -W -Wall -Wextra -Wmissing-prototypes -Wstrict-prototypes -Wwrite-strings -Wno-unused-parameter -fPIC   -c -o util.o util.c
cc -g -W -Wall -Wextra -Wmissing-prototypes -Wstrict-prototypes -Wwrite-strings -Wno-unused-parameter -fPIC   -c -o compats.o compats.c
cc -g -W -Wall -Wextra -Wmissing-prototypes -Wstrict-prototypes -Wwrite-strings -Wno-unused-parameter -fPIC   -c -o main.o main.c
main.c:138:2: warning: #warning Compiling without sandbox support. [-Wcpp]
  138 | #warning Compiling without sandbox support.
      |  ^~~~~~~
ar rs liblowdown.a autolink.o buffer.o diff.o document.o entity.o gemini.o html.o html_escape.o latex.o library.o libdiff.o nroff.o odt.o smartypants.o term.o tree.o util.o compats.o
cc -o lowdown main.o liblowdown.a  -lmd -lm
ln -f lowdown lowdown-diff
mkdir -p /home/lightning/src/lightning/external/lowdown/../../external/build-x86_64-linux-gnu/lowdown-build//bin
mkdir -p /home/lightning/src/lightning/external/lowdown/../../external/build-x86_64-linux-gnu/lowdown-build//man/man1
mkdir -p /home/lightning/src/lightning/external/lowdown/../../external/build-x86_64-linux-gnu/lowdown-build//man/man5
mkdir -p /home/lightning/src/lightning/external/lowdown/../../external/build-x86_64-linux-gnu/lowdown-build//share/lowdown/odt
install -m 0444 share/odt/styles.xml /home/lightning/src/lightning/external/lowdown/../../external/build-x86_64-linux-gnu/lowdown-build//share/lowdown/odt
install -m 0555 lowdown /home/lightning/src/lightning/external/lowdown/../../external/build-x86_64-linux-gnu/lowdown-build//bin
install -m 0555 lowdown-diff /home/lightning/src/lightning/external/lowdown/../../external/build-x86_64-linux-gnu/lowdown-build//bin
for f in man/lowdown.1.html man/lowdown-diff.1.html man/lowdown.5.html ; do \
	name=`basename $f .html` ; \
	section=${name##*.} ; \
	install -m 0444 man/$name /home/lightning/src/lightning/external/lowdown/../../external/build-x86_64-linux-gnu/lowdown-build//man/man$section ; \
done
make[1]: Leaving directory '/home/lightning/src/lightning/external/lowdown'
touch tools/md2man.sh
md2man doc/lightning-cli.1.md
md2man doc/lightningd-config.5.md
md2man doc/lightningd-rpc.7.md
md2man doc/lightning-addgossip.7.md
md2man doc/lightning-autoclean-once.7.md
md2man doc/lightning-autoclean-status.7.md
md2man doc/lightning-batching.7.md
md2man doc/lightning-bkpr-channelsapy.7.md
md2man doc/lightning-bkpr-dumpincomecsv.7.md
md2man doc/lightning-bkpr-inspect.7.md
md2man doc/lightning-bkpr-listaccountevents.7.md
md2man doc/lightning-bkpr-listbalances.7.md
md2man doc/lightning-bkpr-listincome.7.md
md2man doc/lightning-blacklistrune.7.md
md2man doc/lightning-check.7.md
md2man doc/lightning-checkmessage.7.md
md2man doc/lightning-checkrune.7.md
md2man doc/lightning-close.7.md
md2man doc/lightning-connect.7.md
md2man doc/lightning-commando.7.md
md2man doc/lightning-commando-blacklist.7.md
md2man doc/lightning-commando-listrunes.7.md
md2man doc/lightning-commando-rune.7.md
md2man doc/lightning-createonion.7.md
md2man doc/lightning-createinvoice.7.md
md2man doc/lightning-createrune.7.md
md2man doc/lightning-datastore.7.md
md2man doc/lightning-datastoreusage.7.md
md2man doc/lightning-decodepay.7.md
md2man doc/lightning-decode.7.md
md2man doc/lightning-deldatastore.7.md
md2man doc/lightning-delexpiredinvoice.7.md
md2man doc/lightning-delforward.7.md
md2man doc/lightning-delinvoice.7.md
md2man doc/lightning-delpay.7.md
md2man doc/lightning-disableinvoicerequest.7.md
md2man doc/lightning-disableoffer.7.md
md2man doc/lightning-disconnect.7.md
md2man doc/lightning-emergencyrecover.7.md
md2man doc/lightning-feerates.7.md
md2man doc/lightning-fetchinvoice.7.md
md2man doc/lightning-fundchannel.7.md
md2man doc/lightning-fundchannel_start.7.md
md2man doc/lightning-fundchannel_complete.7.md
md2man doc/lightning-fundchannel_cancel.7.md
md2man doc/lightning-funderupdate.7.md
md2man doc/lightning-addpsbtoutput.7.md
md2man doc/lightning-fundpsbt.7.md
md2man doc/lightning-getroute.7.md
md2man doc/lightning-invoice.7.md
md2man doc/lightning-invoicerequest.7.md
md2man doc/lightning-keysend.7.md
md2man doc/lightning-listchannels.7.md
md2man doc/lightning-listclosedchannels.7.md
md2man doc/lightning-listdatastore.7.md
md2man doc/lightning-listforwards.7.md
md2man doc/lightning-listfunds.7.md
md2man doc/lightning-listhtlcs.7.md
md2man doc/lightning-listinvoices.7.md
md2man doc/lightning-listinvoicerequests.7.md
md2man doc/lightning-listoffers.7.md
md2man doc/lightning-listpays.7.md
md2man doc/lightning-listpeers.7.md
md2man doc/lightning-listpeerchannels.7.md
md2man doc/lightning-showrunes.7.md
md2man doc/lightning-listsendpays.7.md
md2man doc/lightning-makesecret.7.md
md2man doc/lightning-multifundchannel.7.md
md2man doc/lightning-multiwithdraw.7.md
md2man doc/lightning-newaddr.7.md
md2man doc/lightning-notifications.7.md
md2man doc/lightning-offer.7.md
md2man doc/lightning-openchannel_abort.7.md
md2man doc/lightning-openchannel_bump.7.md
md2man doc/lightning-openchannel_init.7.md
md2man doc/lightning-openchannel_signed.7.md
md2man doc/lightning-openchannel_update.7.md
md2man doc/lightning-pay.7.md
md2man doc/lightning-parsefeerate.7.md
md2man doc/lightning-plugin.7.md
md2man doc/lightning-preapproveinvoice.7.md
md2man doc/lightning-preapprovekeysend.7.md
md2man doc/lightning-recover.7.md
md2man doc/lightning-recoverchannel.7.md
md2man doc/lightning-renepay.7.md
md2man doc/lightning-renepaystatus.7.md
md2man doc/lightning-reserveinputs.7.md
md2man doc/lightning-sendinvoice.7.md
md2man doc/lightning-sendonion.7.md
md2man doc/lightning-sendonionmessage.7.md
md2man doc/lightning-sendpay.7.md
md2man doc/lightning-setchannel.7.md
md2man doc/lightning-setconfig.7.md
md2man doc/lightning-setpsbtversion.7.md
md2man doc/lightning-sendcustommsg.7.md
md2man doc/lightning-signinvoice.7.md
md2man doc/lightning-signmessage.7.md
md2man doc/lightning-splice_init.7.md
md2man doc/lightning-splice_update.7.md
md2man doc/lightning-splice_signed.7.md
md2man doc/lightning-staticbackup.7.md
md2man doc/lightning-txprepare.7.md
md2man doc/lightning-txdiscard.7.md
md2man doc/lightning-txsend.7.md
md2man doc/lightning-unreserveinputs.7.md
md2man doc/lightning-utxopsbt.7.md
md2man doc/lightning-wait.7.md
md2man doc/lightning-waitinvoice.7.md
md2man doc/lightning-waitanyinvoice.7.md
md2man doc/lightning-waitblockheight.7.md
md2man doc/lightning-waitsendpay.7.md
md2man doc/lightning-withdraw.7.md
md2man doc/lightning-ping.7.md
md2man doc/lightning-stop.7.md
md2man doc/lightning-signpsbt.7.md
md2man doc/lightning-sendpsbt.7.md
md2man doc/lightning-getinfo.7.md
md2man doc/lightning-listtransactions.7.md
md2man doc/lightning-listnodes.7.md
md2man doc/lightning-listconfigs.7.md
md2man doc/lightning-help.7.md
md2man doc/lightning-getlog.7.md
md2man doc/reckless.7.md
md2man doc/lightning-listsqlschemas.7.md
md2man doc/lightning-sql.7.md
md2man doc/lightningd.8.md
md2man doc/lightning-hsmtool.8.md
install -m 644 doc/lightning-cli.1 /home/lightning/share/man/man1
install -m 644 doc/lightningd-config.5 /home/lightning/share/man/man5
install -m 644 doc/lightningd-rpc.7 doc/lightning-addgossip.7 doc/lightning-autoclean-once.7 doc/lightning-autoclean-status.7 doc/lightning-batching.7 doc/lightning-bkpr-channelsapy.7 doc/lightning-bkpr-dumpincomecsv.7 doc/lightning-bkpr-inspect.7 doc/lightning-bkpr-listaccountevents.7 doc/lightning-bkpr-listbalances.7 doc/lightning-bkpr-listincome.7 doc/lightning-blacklistrune.7 doc/lightning-check.7 doc/lightning-checkmessage.7 doc/lightning-checkrune.7 doc/lightning-close.7 doc/lightning-connect.7 doc/lightning-commando.7 doc/lightning-commando-blacklist.7 doc/lightning-commando-listrunes.7 doc/lightning-commando-rune.7 doc/lightning-createonion.7 doc/lightning-createinvoice.7 doc/lightning-createrune.7 doc/lightning-datastore.7 doc/lightning-datastoreusage.7 doc/lightning-decodepay.7 doc/lightning-decode.7 doc/lightning-deldatastore.7 doc/lightning-delexpiredinvoice.7 doc/lightning-delforward.7 doc/lightning-delinvoice.7 doc/lightning-delpay.7 doc/lightning-disableinvoicerequest.7 doc/lightning-disableoffer.7 doc/lightning-disconnect.7 doc/lightning-emergencyrecover.7 doc/lightning-feerates.7 doc/lightning-fetchinvoice.7 doc/lightning-fundchannel.7 doc/lightning-fundchannel_start.7 doc/lightning-fundchannel_complete.7 doc/lightning-fundchannel_cancel.7 doc/lightning-funderupdate.7 doc/lightning-addpsbtoutput.7 doc/lightning-fundpsbt.7 doc/lightning-getroute.7 doc/lightning-invoice.7 doc/lightning-invoicerequest.7 doc/lightning-keysend.7 doc/lightning-listchannels.7 doc/lightning-listclosedchannels.7 doc/lightning-listdatastore.7 doc/lightning-listforwards.7 doc/lightning-listfunds.7 doc/lightning-listhtlcs.7 doc/lightning-listinvoices.7 doc/lightning-listinvoicerequests.7 doc/lightning-listoffers.7 doc/lightning-listpays.7 doc/lightning-listpeers.7 doc/lightning-listpeerchannels.7 doc/lightning-showrunes.7 doc/lightning-listsendpays.7 doc/lightning-makesecret.7 doc/lightning-multifundchannel.7 doc/lightning-multiwithdraw.7 doc/lightning-newaddr.7 doc/lightning-notifications.7 doc/lightning-offer.7 doc/lightning-openchannel_abort.7 doc/lightning-openchannel_bump.7 doc/lightning-openchannel_init.7 doc/lightning-openchannel_signed.7 doc/lightning-openchannel_update.7 doc/lightning-pay.7 doc/lightning-parsefeerate.7 doc/lightning-plugin.7 doc/lightning-preapproveinvoice.7 doc/lightning-preapprovekeysend.7 doc/lightning-recover.7 doc/lightning-recoverchannel.7 doc/lightning-renepay.7 doc/lightning-renepaystatus.7 doc/lightning-reserveinputs.7 doc/lightning-sendinvoice.7 doc/lightning-sendonion.7 doc/lightning-sendonionmessage.7 doc/lightning-sendpay.7 doc/lightning-setchannel.7 doc/lightning-setconfig.7 doc/lightning-setpsbtversion.7 doc/lightning-sendcustommsg.7 doc/lightning-signinvoice.7 doc/lightning-signmessage.7 doc/lightning-splice_init.7 doc/lightning-splice_update.7 doc/lightning-splice_signed.7 doc/lightning-staticbackup.7 doc/lightning-txprepare.7 doc/lightning-txdiscard.7 doc/lightning-txsend.7 doc/lightning-unreserveinputs.7 doc/lightning-utxopsbt.7 doc/lightning-wait.7 doc/lightning-waitinvoice.7 doc/lightning-waitanyinvoice.7 doc/lightning-waitblockheight.7 doc/lightning-waitsendpay.7 doc/lightning-withdraw.7 doc/lightning-ping.7 doc/lightning-stop.7 doc/lightning-signpsbt.7 doc/lightning-sendpsbt.7 doc/lightning-getinfo.7 doc/lightning-listtransactions.7 doc/lightning-listnodes.7 doc/lightning-listconfigs.7 doc/lightning-help.7 doc/lightning-getlog.7 doc/reckless.7 doc/lightning-listsqlschemas.7 doc/lightning-sql.7 /home/lightning/share/man/man7
install -m 644 doc/lightningd.8 doc/lightning-hsmtool.8 /home/lightning/share/man/man8
install -m 644 README.md LICENSE /home/lightning/share/doc/c-lightning

1 CPU seems to be 100% busy with cargo build --quiet --bin cln-grpc for 10+ minutes now. I guess I'll just wait, but the whole build never took more than a few minutes on this machine.
@Sjors
Copy link
Contributor Author

Sjors commented Nov 7, 2023

Update: the installer finished after about 15 minutes total. If this can't be made faster, then at least I think it should be more verbose and show some progress.

@vincenzopalazzo
Copy link
Collaborator

I suspect that was was cargo build --quiet --bin ... to take long time? I have the same behavior

This can be caused if you change your rust version, and the recompilation of all the crates (that are a few) is trigger

@Sjors
Copy link
Contributor Author

Sjors commented Nov 7, 2023 •
edited
Loading

The lightning user has its own rust installation. I didn't change it, but maybe CLN did?

Current version is cargo 1.67.1 (8ecd4f20a 2023-01-10)

@daywalker90
Copy link
Contributor

i think i saw a commit for cln that updated the rust version and it might be that cargo updated its repo which takes a long time (git resolving deltas). I gave it a quick search and found this: rust-lang/cargo#11014

@vincenzopalazzo
Copy link
Collaborator

The lightning user has its own rust installation. I didn't change it, but maybe CLN did?

No, we did not, we do not force the rustc version at the moment but it is possible that cargo is rebuilding.

If this happens could you run a patch version of the Makefile where you remove the cargo build --silent and use just cargo build? I can make a patch for you if you like

@Sjors
Copy link
Contributor Author

Sjors commented Nov 13, 2023

Good idea, I'll try that if it happens again.

@vincenzopalazzo vincenzopalazzo added the in diagnostic issue under diagnostic label Nov 14, 2023
@Sjors
Copy link
Contributor Author

Sjors commented Feb 29, 2024

It's being slow again while upgrading to v24.02, but I can't find the cargo build --silent line.

Cargo version before install: cargo 1.67.1

@Sjors
Copy link
Contributor Author

Sjors commented Feb 29, 2024

It was a bit faster than last time, only about 5 minutes. Cargo version after install is unchanged.

@vincenzopalazzo
Copy link
Collaborator

git grep returned some item

cln-rpc/Makefile:12:    cargo build ${CARGO_OPTS} --example cln-rpc-getinfo
cln-rpc/Makefile:15:    cargo build ${CARGO_OPTS} --example cln-plugin-startup
cln-rpc/Makefile:18:    cargo build ${CARGO_OPTS} --example cln-plugin-reentrant
plugins/Makefile:257:   cargo build ${CARGO_OPTS} --bin cln-grpc

Cargo version after install is unchanged.

The cln-rust library is changed in this case, causing rebuilding (maybe) but I do not know why it is taking all this time :/

@Sjors
Copy link
Contributor Author

Sjors commented Feb 29, 2024

Oh I guess --silent is in CARGO_OPTS? Will try that next time.

I did a git clean -dfx, ./configure ... after this to see what would happen. This time it built quickly.

@cdecker cdecker closed this as completed Jun 13, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@Sjors Sjors

@vincenzopalazzo vincenzopalazzo

Labels
in diagnostic issue under diagnostic
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@Sjors @cdecker @daywalker90 @vincenzopalazzo