Skip to content

Add ksh93 [new package] #5243

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 10 commits into
base: master
Choose a base branch
from

Conversation

xeiavicaflashrepository
Copy link
Contributor

Ksh93 build system is a series of shell scripts and custom make program. Why? I don't know, historical reasons?

Problem arises with fork() during the compilation process. After about 5 minutes, it finishes with libast, then begins libsum... which fails due to I think something related to ASLR? Not sure, can this be disabled by non admin?

# ... making src/lib/libsum ...
      0 [main] mamake 6978 dofork: child -1 - forked process 11256 died unexpectedly, retry 0, exit code 0xC0000142, errno 11
mamake: fork() failed: Resource temporarily unavailable
mamake: *** exit code 1 making all

If anyone knows more than I do, please help.

@gisburn
Copy link

gisburn commented Mar 9, 2025

Where can I find the script you used/try to use to build ksh93 on msys2 ?

@xeiavicaflashrepository
Copy link
Contributor Author

Where can I find the script you used/try to use to build ksh93 on msys2 ?

The PKGBUILD script I am trying to merge, is the script to build it. MSYS2 is based off arch linux including pacman and makepkg build system. If you're not familiar with this, you can read about it here.

All a PKGBUILD file is, a simple bourne shell script. It goes through a functions and package was made.

@xeiavicaflashrepository
Copy link
Contributor Author

Additional notes here, if you desire them:
https://www.msys2.org/wiki/Creating-Packages/

Upon setting up msys2 and everything is up to date, typing pacman -S make base-devel gcc should give you the correct applications to do a basic run.

It isn't ready for merge yet, but it finally builds.
@xeiavicaflashrepository
Copy link
Contributor Author

Please do not merge yet, there are some changes that need to be made. A good default config needs to be shipped and a few patches should be considered that red hat and debian use, plus the ones from the gisburn repo.

@xeiavicaflashrepository
Copy link
Contributor Author

I don't know who else to ask for a review, but
@jeremyd2019 and @lazka could you give me feedback on your thoughts?

@xeiavicaflashrepository xeiavicaflashrepository changed the title Add ksh93 - (Do not merge yet, running into issues, need some help) Add ksh93 [new package, ready for merge] Mar 10, 2025
@xeiavicaflashrepository
Copy link
Contributor Author

@gisburn
Let me know your thoughts on the CI built package for msys2.

@jeremyd2019
Copy link
Member

Problem arises with fork() during the compilation process. After about 5 minutes, it finishes with libast, then begins libsum... which fails due to I think something related to ASLR? Not sure, can this be disabled by non admin?

# ... making src/lib/libsum ...
      0 [main] mamake 6978 dofork: child -1 - forked process 11256 died unexpectedly, retry 0, exit code 0xC0000142, errno 11
mamake: fork() failed: Resource temporarily unavailable
mamake: *** exit code 1 making all

I'm pretty sure mamake/mamake.exe is removed and replaced right before this. If this is done while mamake is running, and then it expects to be able to fork afterwards, that is probably the problem. I saw a check go by to see if an open file can be removed. Maybe that is supposed to detect when this can't be done and do something else instead? Can that check be overridden (like autoconf can set ac_cv_something vars to override test results)?

@xeiavicaflashrepository
Copy link
Contributor Author

Now that you mention it, because the entire build system is a shell script and doesn't use autotools or anything like that, it's rather ridged in its customization. But there IS a small section if it detects a cygwin environment.

case $HOSTTYPE in
			cygwin.*)
				# Windows/Cygwin kills fork(2) after deleting *or* renaming an executable that's in use :-/
				# To allow relinking the binary against libast later, add a _bootstrap suffix and create
				# a symlink to it. The symlink can be removed/renamed instead of the _bootstrap binary.
				$exec $CC -O $CCFLAGS $LDFLAGS -o "$INSTALLROOT/bin/${i}_bootstrap" "$INITROOT/$i.c" || return
				$exec ln -sf "${i}_bootstrap" "$INSTALLROOT/bin/$i" || return
				;;
			*)	$exec $CC -O $CCFLAGS $LDFLAGS -o "$INSTALLROOT/bin/$i" "$INITROOT/$i.c" || return
				;;
			esac

But don't symlinks not work on windows even with the msys2 shell?

Copy link
Member

@jeremyd2019 jeremyd2019 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't really have time right now to look at this, maybe somebody else could help with this package

ksh93/PKGBUILD Outdated
pkgver=1.0.10
pkgrel=1
pkgdesc="The Original ATT Korn Shell, Now Maintained By The Community."
arch=('x86_64')
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
arch=('x86_64')
arch=('i686' 'x86_64')

ksh93/PKGBUILD Outdated
'SKIP')

# This is necessary
export PATH="/bin:/usr/bin:$gccdir:$gccdir/include:/usr/include"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't do this globally... Several things have to source PKGBUILDs to try to extract variables from them, and they don't need their paths messed up. BTW, where is gccdir set where this could see it, and why on earth would you want include dirs in the PATH?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I must have removed that variable and forgot to delete references to it. It works without it, so I'll remove it.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have had multiple failures including another one here because cc1.exe cannot be found. It only works if I explicitly put it in the PATH for some reason.

ksh93/PKGBUILD Outdated
Comment on lines 12 to 30
makedepends=(
'gcc'
'gdb'
'make'
'gettext'
'gettext-devel'
'flex'
'bison'
'unzip'
'pax'
'tar'
'libiconv-devel'
'ncurses-devel'
'gmp-devel'
'mpfr-devel'
'mpc-devel'
'isl-devel'
'procps-ng'
)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's no way all of these are actually needed.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't know what I was doing honestly. I just saw someone else from the ksh93 repo say you should install that and just dropped it in.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
makedepends=(
'gcc'
'gdb'
'make'
'gettext'
'gettext-devel'
'flex'
'bison'
'unzip'
'pax'
'tar'
'libiconv-devel'
'ncurses-devel'
'gmp-devel'
'mpfr-devel'
'mpc-devel'
'isl-devel'
'procps-ng'
)
makedepends=(
'gcc'
'gettext-devel'
'libiconv-devel'
'ncurses-devel'
)

maybe adding back

	'flex'
	'bison'
	'gmp-devel'
	'mpfr-devel'
	'mpc-devel'
	'isl-devel'

if it's demonstrated they're actually used

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not going to include libiconv-devel because it completely fails if the GNU one is installed, I have to patch out looking for the system wide libiconv and use the bundled one instead.

ksh93/PKGBUILD Outdated
build() {
cd "${srcdir}/ksh-${pkgver}"

export CYGWIN_CHOST="${CHOST/-msys/-cygwin}"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is used in other PKGBUILDs as an argument to configure. I don't see you using it anywhere here, and I doubt ksh93's build scripts expect it to be set. Maybe you could try local MSYSTEM=CYGWIN and see if anything useful changes?

ksh93/PKGBUILD Outdated
Comment on lines 112 to 113
install -Dm0644 "src/cmd/ksh93/${_docs}" "${pkgdir}/usr/share/doc/ksh/${_docs}"
done
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

formatting

ksh93/PKGBUILD Outdated

package() {
cd "${srcdir}"
install -dm0755 "${pkgdir}/etc/skel"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

were you intending to put something in /etc/skel?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably shouldn't have had that line, I was copying from the arch linux one I found on aur.

ksh93/PKGBUILD Outdated
arch=('x86_64')
url="https://www.github.com/ksh93/ksh"
license=('spdx:EPL-2.0')

Copy link
Member

@jeremyd2019 jeremyd2019 Mar 11, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe add a backup for etc/ksh.kshrc ?

backup=('etc/ksh.kshrc')

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the leading slash supposed to not be there?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it is not.

backup=('etc/fstab' 'etc/shells' 'etc/profile'
etc/bash.bashrc etc/skel/.bash{rc,_profile}
etc/nsswitch.conf)

@jeremyd2019
Copy link
Member

jeremyd2019 commented Mar 11, 2025

Now that you mention it, because the entire build system is a shell script and doesn't use autotools or anything like that, it's rather ridged in its customization. But there IS a small section if it detects a cygwin environment.

case $HOSTTYPE in
			cygwin.*)
				# Windows/Cygwin kills fork(2) after deleting *or* renaming an executable that's in use :-/
				# To allow relinking the binary against libast later, add a _bootstrap suffix and create
				# a symlink to it. The symlink can be removed/renamed instead of the _bootstrap binary.
				$exec $CC -O $CCFLAGS $LDFLAGS -o "$INSTALLROOT/bin/${i}_bootstrap" "$INITROOT/$i.c" || return
				$exec ln -sf "${i}_bootstrap" "$INSTALLROOT/bin/$i" || return
				;;
			*)	$exec $CC -O $CCFLAGS $LDFLAGS -o "$INSTALLROOT/bin/$i" "$INITROOT/$i.c" || return
				;;
			esac

But don't symlinks not work on windows even with the msys2 shell?

Depends. In this case you could try patching it to

case $HOSTTYPE in
			cygwin.*|msys.*)
				# Windows/Cygwin kills fork(2) after deleting *or* renaming an executable that's in use :-/
				# To allow relinking the binary against libast later, add a _bootstrap suffix and create
				# a symlink to it. The symlink can be removed/renamed instead of the _bootstrap binary.
				$exec $CC -O $CCFLAGS $LDFLAGS -o "$INSTALLROOT/bin/${i}_bootstrap" "$INITROOT/$i.c" || return
				MSYS="$MSYS winsymlinks:native" $exec ln -sf "${i}_bootstrap" "$INSTALLROOT/bin/$i" || return
				;;
			*)	$exec $CC -O $CCFLAGS $LDFLAGS -o "$INSTALLROOT/bin/$i" "$INITROOT/$i.c" || return
				;;
			esac

@xeiavicaflashrepository
Copy link
Contributor Author

Speaking of cygwin, I have the option to dynamic compile ksh93, would you prefer that?

@xeiavicaflashrepository
Copy link
Contributor Author

Just a note, @jeremyd2019 , it's not optional to put the explicit PATH right now for cc1.exe. It won't work otherwise. Everything else was highly simplified.

ksh93/PKGBUILD Outdated
cd "${srcdir}/ksh-${pkgver}"

# For now, it literally fails if cc1.exe is not explicitly on the PATH.
local GCC_DIR="/usr/lib/gcc/x86_64-pc-msys/13.3.0"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
local GCC_DIR="/usr/lib/gcc/x86_64-pc-msys/13.3.0"
local GCC_DIR="$(gcc -print-search-dirs | sed -ne '/^install: /s///p')"

ksh93/PKGBUILD Outdated
# For now, it literally fails if cc1.exe is not explicitly on the PATH.
local GCC_DIR="/usr/lib/gcc/x86_64-pc-msys/13.3.0"
export PATH="$PATH:$GCC_DIR"
export CYGWIN='ntsec'
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
export CYGWIN='ntsec'

Setting CYGWIN has no effect on MSYS2. Also, the ntsec option is obsolete and removed anyway. Is this just to shut up an error in their build script that is blindly insisting you set that option?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this just to shut up an error in their build script that is blindly insisting you set that option?

Yes, this build system is like I said, highly ridged and expects things to be set a certain way. Even if it does nothing at all. I'm thinking of reverting the ksh-mamake patch since that failed as well.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

that patch did seem to help, but then because you were specifying host system of cygwin.i386 it then also triggered the stupid CYGWIN env var check. Maybe if you did like I originally suggested and changed the case to cygwin.*|msys.*). or leave the host as cygwin.i386 and comment the setting of the CYGWIN env var that it is known that it does nothing but is there just to shut up the stupid build system.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm going to make it think this is a cygwin system. Because if I don't, it'll think it's an unknown system and won't let me build as a dynamic libraries. Do you want me to make it dynamic libraries? I'll add back in CYGWIN='ntsec' and HOSTTYPE=cygwin.i386.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unless other packages will use the libraries, I don't think it adds much, since there's only a couple of exes anyway, and they'll necessarily be rebuilt if the library changes since they're part of the same package.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm going to make it think this is a cygwin system. Because if I don't, it'll think it's an unknown system and won't let me build as a dynamic libraries. Do you want me to make it dynamic libraries? I'll add back in CYGWIN='ntsec' and HOSTTYPE=cygwin.i386.

That might not work for the ksh93 builtin getconf(1) and uname(1) commands, because then getconf(1) and uname(1) will think this is Cygwin, and will return identification information for Cygwin instead of MSYS2.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why wouldn't uname(1) just call uname(2) and get the correct information? I assume getconf(1) should also be backed by an API

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MSYS2 changes its uname output depending on the setting of the MSYSTEM environment variable, so if it doesn't call the system uname function, it will need to duplicate that logic. I don't think anybody wants that.

ksh93/PKGBUILD Outdated
Comment on lines 61 to 63
CC="/usr/bin/gcc" /usr/bin/bash bin/package make PACKAGEROOT="$PWD" HOSTTYPE="cygwin.i386" || true


Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
CC="/usr/bin/gcc" /usr/bin/bash bin/package make PACKAGEROOT="$PWD" HOSTTYPE="cygwin.i386" || true
CC="/usr/bin/gcc" /usr/bin/bash bin/package make PACKAGEROOT="$PWD" HOSTTYPE="cygwin.i386"

ksh93/PKGBUILD Outdated
Comment on lines 9 to 18
makedepends=(
'gcc'
'gettext'
'gettext-devel'
'ncurses-devel'
'gmp-devel'
'mpfr-devel'
'mpc-devel'
'isl-devel'
)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

there are more whitespace issues now. For one thing, PKGBUILD files should indent with 2 spaces. The arrays when split onto multiple lines should be lined up using spaces, like

Suggested change
makedepends=(
'gcc'
'gettext'
'gettext-devel'
'ncurses-devel'
'gmp-devel'
'mpfr-devel'
'mpc-devel'
'isl-devel'
)
makedepends=('gcc'
'gettext'
'gettext-devel'
'ncurses-devel'
'gmp-devel'
'mpfr-devel'
'mpc-devel'
'isl-devel')

also, your ksh-mamake.patch changed whitespace/indentation

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was puzzled why this was happening, but I discovered that because I was using nano, it by default inserted a literal tab which bonked with git when it was expecting spaces. I'll fix that.

@cedricblancher
Copy link

Did you package the shell script compiler (/bin/shcomp.exe) too?

@xeiavicaflashrepository
Copy link
Contributor Author

Did you package the shell script compiler (/bin/shcomp.exe) too?

Yup. If it matters, there is no bin on msys2, using for example ls /bin in ksh93 (or any other shell like dash), will yield what is in /usr/bin. I think upstream arch linux does this also, symlinking /bin to /usr/bin.

Here is the entire tree of the .zst package:

.BUILDINFO
.MTREE
.PKGINFO
etc/ksh.kshrc
usr/bin/ksh.exe
usr/bin/ksh93.exe
usr/bin/shcomp.exe
usr/share/doc/ksh
usr/share//ksh/functions
usr/share/licenses/ksh
usr/share/man/man1

@cedricblancher
Copy link

Did you package the shell script compiler (/bin/shcomp.exe) too?

Yup. If it matters, there is no bin on msys2, using for example ls /bin in ksh93 (or any other shell like dash), will yield what is in /usr/bin. I think upstream arch linux does this also, symlinking /bin to /usr/bin.

Here is the entire tree of the .zst package:

.BUILDINFO
.MTREE
.PKGINFO
etc/ksh.kshrc
usr/bin/ksh.exe
usr/bin/ksh93.exe
usr/bin/shcomp.exe
usr/share/doc/ksh
usr/share//ksh/functions
usr/share/licenses/ksh
usr/share/man/man1

Thank you for packaging shcomp.

One request: could you also please package the ksh93 tests/ dir? I think we need it to find Cygwin and MSYS2 signal and filesystem bugs.

@xeiavicaflashrepository
Copy link
Contributor Author

Sure, let me finish up my build script you requested in the other thread then I'll make the modification. Where should tests goto? /usr/share?

@cedricblancher
Copy link

Sure, let me finish up my build script you requested in the other thread then I'll make the modification. Where should tests goto? /usr/share?

https://github.com/kofemann/ms-nfs41-client/blob/master/cygwin/cygwin_ksh93/ksh93.cygport puts the tests into usr/share/ksh/tests/...

@xeiavicaflashrepository
Copy link
Contributor Author

Done, if it works after 15 minutes of building, you can test it.

@xeiavicaflashrepository
Copy link
Contributor Author

It worked, all the .sh scripts are under /usr/share/ksh/tests. Anything else that needs to be done?

@jeremyd2019
Copy link
Member

I managed to debug and remove a bunch of the hacks. Regarding the /usr/ast and custom config override: I removed those too, it looks an awful lot like everything in the source assumes the path is /opt/ast, and neither Gentoo nor Arch override this. I don't know where /usr/ast comes from... it occurs nowhere in the sources, but /opt/ast does

@xeiavicaflashrepository
Copy link
Contributor Author

@jeremyd2019
How radical of a patch is it? Do you think it can be merged into upstream or even modified to just know what to do with msys2 in general?

@jeremyd2019
Copy link
Member

I pushed here, so you can see for yourself what I did.

@xeiavicaflashrepository
Copy link
Contributor Author

Disregard what I asked, this would be a question for upstream about the ramifications of the patched files.

@jeremyd2019
Copy link
Member

most of your weird problems were due to the package script overriding the PATH to put /bin first. /usr/bin and /bin are the same directory (more like a bind mount than a symlink really), but gcc looks for its files relative to itself, and /lib is not the same as /usr/lib, so looking in /bin/../lib/gcc/... didn't work. I added a patch that if /bin and /usr/bin are the same (by test -ef), put /usr/bin first in the default PATH. That's kind of a hack, but solves the issue here. Note that setting CC=/usr/bin/gcc does not actually work, because in the case where CC is set to something other than cc, it tries to write a wrapper script called cc to its INSTALLROOT/bin, but neglects to include a shebang, so MSYS2 doesn't consider it executable and skips it when searching on the PATH - only to find /bin/cc next.

@xeiavicaflashrepository
Copy link
Contributor Author

That makes sense, What I'm going to do, is try to merge to upstream these patches, that are from observation incredibly minor so that someone else building ksh93 can get it without issue and not have to get lucky coming across this github merge request discussion.

@gisburn
Copy link

gisburn commented Mar 13, 2025

To answer your question, @jeremyd2019 I have not gotten an answer yet. You want me to remove the function that generates the header file, if it truly isn't essential?

Re Why /usr/ast/bin ?

"AST" stands "Advanced Software Technology", it's the evolution of the original UNIX SystemV utilities, updated to conform to XPG8 (OpenGroup/POSIX). The utilities were developed by David Korn, Glenn Fowler, Kiem-Phong. Vo&co.
Originally developed for AT&T Research, and used by AT&T products, like UWIN on Windows, in a busybox-style toolkit manner for settop boxes and as add-ons for UNIX.
Since UNIX flavours (Solaris/Illumos), AIX, HP/UX, IRIX, SuperUX, CrayOS etc usuallyhave to implement multiple (versions of a) standard(s), it is common to have something like /usr/bin (evolving system default), /usr/xpg4/bin+/usr/xpg6/bin+/usr/xpg8/bin for POSIX/SUS versions, /usr/gnu/bin for GNU flavours, and finally /usr/ast/bin for the AST tools.

Since some POSIX utilities are builtin into ksh93 they need to be looked-up in the correct manner, and that's why some utilities are bound to /bin+/usr/bin (if they are compatible to the platform default utilities), /usr/gnu/bin (if they are compatible to the GNU utilities), /usr/xpg[468]/bin for various POSIX standards etc.
But then there still needs to be a PATH element left which just enables ALL AST builtins (1. for scripts who need it, and 2. because the ELF/COFF symbols need to be public), and that is /usr/ast/bin.

Finally there needed to be a default if the platform does not have one, and that's why /opt/ast/bin is the pseudo-default. But /opt is ONLY for "optional add-on software", and not for something like a system shell.

That's why we have - for Cygwin - the tmp_gnulinux_builtin_header.h header, with a fixed list of builtins, all tested for compatibility with /bin+/usr/bin. And to /usr/ast/bin all available utilities are mapped too, so scripts can enable them by just sticking "/usr/ast/bin:" in front of PATH.

BTW: Please use Google for counting - more scripts use /usr/ast/bin than /opt/ast/bin ...

@gisburn
Copy link

gisburn commented Mar 13, 2025

I managed to debug and remove a bunch of the hacks. Regarding the /usr/ast and custom config override: I removed those too, it looks an awful lot like everything in the source assumes the path is /opt/ast, and neither Gentoo nor Arch override this. I don't know where /usr/ast comes from... it occurs nowhere in the sources, but /opt/ast does

The problem is the total "balkanisation" of utility defaults in Linux, because noone wants to honor or cooperate with the LSB (Linux Standards Base) anymore. ksh93 suffers from maintainers only shipping the MINIMUM (and technically untested) defaults from the package. Correct builtin utility path bindings requires testing and maintenance, and since noone wants to do that we end-up with the current situation.

For Cygwin the situation is better, because I used by knowledge of SUN's ksh93-integration (i.e. update Solaris /bin/sh to ksh93)+POSIX utility modernisation (i.e. update Solaris /usr/bin to AST&T AST utilities+XPG8 conformance testing) projects, and just did the testing+validation work.
For MSYS2 the same settings apply, and I am willing to do maintenance+updates if needed.

@xeiavicaflashrepository
Copy link
Contributor Author

@gisburn
Hi, in this patch I merged together what @jeremyd2019 did and I converted anything related to cygwin to msys/msys2. Because anyone doing UNIX stuff on windows is really going to be using msys rather than cygwin. Tell me your thoughts. I wish to see this and a sed command from the PKGBUILD script merged into upstream.
ksh93-fix-cygwin.patch.txt

@jeremyd2019
Copy link
Member

Debian's package doesn't seem to perform this override either.

$ builtin
.
:
[
alias
autoload
bg
break
builtin
cd
command
compound
continue
disown
echo
enum
eval
exec
exit
export
false
fc
fg
float
functions
getopts
hash
hist
integer
jobs
kill
let
nameref
/opt/ast/bin/basename
/opt/ast/bin/cat
/opt/ast/bin/cp
/opt/ast/bin/cut
/opt/ast/bin/dirname
/opt/ast/bin/getconf
/opt/ast/bin/ln
/opt/ast/bin/mktemp
/opt/ast/bin/mv
print
printf
pwd
read
readonly
redirect
return
set
shift
sleep
source
stop
suspend
test
times
trap
true
type
typeset
ulimit
umask
unalias
unset
wait
whence

@gisburn
Copy link

gisburn commented Mar 13, 2025

I managed to debug and remove a bunch of the hacks. Regarding the /usr/ast and custom config override: I removed those too, it looks an awful lot like everything in the source assumes the path is /opt/ast, and neither Gentoo nor Arch override this. I don't know where /usr/ast comes from... it occurs nowhere in the sources, but /opt/ast does

The problem is the total "balkanisation" of utility defaults in Linux, because noone wants to honor or cooperate with the LSB (Linux Standards Base) anymore. ksh93 suffers from maintainers only shipping the MINIMUM (and technically untested) defaults from the package. Correct builtin utility path bindings requires testing and maintenance, and since noone wants to do that we end-up with the current situation.

For Cygwin the situation is better, because I used by knowledge of SUN's ksh93-integration (i.e. update Solaris /bin/sh to ksh93)+POSIX utility modernisation (i.e. update Solaris /usr/bin to AST&T AST utilities+XPG8 conformance testing) projects, and just did the testing+validation work. For MSYS2 the same settings apply, and I am willing to do maintenance+updates if needed.

And for Linux the situation is easy to fix:

  1. If the Linux distribution conforms to LSB as need a "gnulinux_lsb_builtin_header.h"
  2. If the Linux distribution (example: The "ChickenMonster" disto) does not conform to the LSB it needs a "gnulinux_chickenmonster_builtin_header.h".
    That can be done, but requires to install all distros, do the testing, write (by hand) the header, and put it into the AST/ksh93 upstream git.

For Cygwin and Solaris/Illumos I did that work already.

@jeremyd2019
Copy link
Member

Because anyone doing UNIX stuff on windows is really going to be using msys rather than cygwin.

No. Cygwin is meant for people doing UNIX stuff on Windows. MSYS2 is only meant to provide just enough UNIX stuff to get open source software building natively for Windows (without the posix translation layer that Cygwin provides and MSYS2 mildly patches)

@jeremyd2019
Copy link
Member

@jeremyd2019
Copy link
Member

I'm not personally convinced about overriding defaults in such a way as to diverge from Linux distributions. @lazka ? Do you think this package is even in-scope for MSYS2-packages?

Build as Cygwin.  Add check function.  the sigchld tests take A LONG
TIME.  Several tests fail, but those can be investigated later.
@cedricblancher
Copy link

I'm not personally convinced about overriding defaults in such a way as to diverge from Linux distributions. @lazka ? Do you think this package is even in-scope for MSYS2-packages?

  1. Neither Cygwin or MSYS2 are Linux-compatible, they emulate some aspects of Linux, but mostly work more like Unix(-Services-For-Windows(SFU))
  2. ksh93 is VERY popular for scientific software, because it is the only POSIX shell which properly implements IEEE754-2008 math, including NAN, -NAN, INF, -INF, ISO-C99-style floating point math functions - and unlike bash&dash is fully POSIX.2008 sh conform. You don't have anything like that in MSYS2 yet. So yes, this package is IMO in the scope of MSYS2
  3. ALL of our scripts use /usr/ast/bin, and not /opt/ast/bin as default to enable all AST builtins. So if MSYS2 diverges from what has been the ksh93 defaults since Cygwin 3.2, that'll break stuff (unnecessarily!)

@cedricblancher
Copy link

FYI UWIN (the original Windows software package from AT&T, basically competition to Cygwin) is using /usr/ast/bin, and so do Cygwin, Solaris, AIX, Illumos.

@jeremyd2019
Copy link
Member

  1. ALL of our scripts use /usr/ast/bin, and not /opt/ast/bin as default to enable all AST builtins. So if MSYS2 diverges from what has been the ksh93 defaults since Cygwin 3.2, that'll break stuff (unnecessarily!)

https://cygwin.com/packages/package_list.html - I only see mksh there. If you mean your own group's unofficial Cygwin package, I don't find that particularly compelling. But I'm done with this PR. Maybe somebody else with write access will be persuaded, or you could maintain your own unofficial package that does what you want, like you already do for Cygwin.

@jeremyd2019 jeremyd2019 marked this pull request as ready for review March 14, 2025 17:44
@takeshinishimuralinux
Copy link

How can I install this package via pacman?

@xeiavicaflashrepository
Copy link
Contributor Author

How can I install this package via pacman?

That's easy, once you build it successfully, type pacman -U <package name> and it will install as a foreign package.

@takeshinishimuralinux
Copy link

How can I install this package via pacman?

That's easy, once you build it successfully, type pacman -U <package name> and it will install as a foreign package.

That does not help me, because I need to do that for 5200+ machines, and I should only install the "official packages". Everything extra needs the signature of my professor.

Why is the "ksh93" package not in the official packages yet?

@jeremyd2019
Copy link
Member

Why is the "ksh93" package not in the official packages yet?

I got burned out on this PR, and no one else appears to have taken it on to decide whether to merge it as-is (conforming to Linux precedent) or including a sizeable custom header to make it more like Solaris instead.

@xeiavicaflashrepository
Copy link
Contributor Author

Why is the "ksh93" package not in the official packages yet?

I got burned out on this PR, and no one else appears to have taken it on to decide whether to merge it as-is (conforming to Linux precedent) or including a sizeable custom header to make it more like Solaris instead.

Is there anyone who is the final say on a package? I just want to finish this already too.

@takeshinishimuralinux
Copy link

Why is the "ksh93" package not in the official packages yet?

I got burned out on this PR, and no one else appears to have taken it on to decide whether to merge it as-is (conforming to Linux precedent) or including a sizeable custom header to make it more like Solaris instead.

Is there anyone who is the final say on a package? I just want to finish this already too.

I would go for the "Solaris solution" [/usr/ast/bin/], because it looks people do CARE about it, and are willing to maintain it.

@xeiavicaflashrepository
Copy link
Contributor Author

@jeremyd2019
Can we please just go with the solaris solution and be done with this?

@jeremyd2019 jeremyd2019 requested a review from lazka April 13, 2025 22:25
@brownmarkliam
Copy link

brownmarkliam commented Apr 14, 2025 via email

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Package Request] Package ksh93u+m for msys2.
6 participants