-
Notifications
You must be signed in to change notification settings - Fork 497
Add ksh93 [new package] #5243
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Add ksh93 [new package] #5243
Conversation
Where can I find the script you used/try to use to build ksh93 on msys2 ? |
The PKGBUILD script I am trying to merge, is the script to build it. MSYS2 is based off arch linux including pacman and makepkg build system. If you're not familiar with this, you can read about it here. All a PKGBUILD file is, a simple bourne shell script. It goes through a functions and package was made. |
Additional notes here, if you desire them: Upon setting up msys2 and everything is up to date, typing |
It isn't ready for merge yet, but it finally builds.
Please do not merge yet, there are some changes that need to be made. A good default config needs to be shipped and a few patches should be considered that red hat and debian use, plus the ones from the gisburn repo. |
…shipping out a system kshrc file.
I don't know who else to ask for a review, but |
@gisburn |
I'm pretty sure mamake/mamake.exe is removed and replaced right before this. If this is done while mamake is running, and then it expects to be able to fork afterwards, that is probably the problem. I saw a check go by to see if an open file can be removed. Maybe that is supposed to detect when this can't be done and do something else instead? Can that check be overridden (like autoconf can set ac_cv_something vars to override test results)? |
Now that you mention it, because the entire build system is a shell script and doesn't use autotools or anything like that, it's rather ridged in its customization. But there IS a small section if it detects a cygwin environment.
But don't symlinks not work on windows even with the msys2 shell? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't really have time right now to look at this, maybe somebody else could help with this package
ksh93/PKGBUILD
Outdated
pkgver=1.0.10 | ||
pkgrel=1 | ||
pkgdesc="The Original ATT Korn Shell, Now Maintained By The Community." | ||
arch=('x86_64') |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
arch=('x86_64') | |
arch=('i686' 'x86_64') |
ksh93/PKGBUILD
Outdated
'SKIP') | ||
|
||
# This is necessary | ||
export PATH="/bin:/usr/bin:$gccdir:$gccdir/include:/usr/include" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Don't do this globally... Several things have to source PKGBUILDs to try to extract variables from them, and they don't need their paths messed up. BTW, where is gccdir set where this could see it, and why on earth would you want include dirs in the PATH?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I must have removed that variable and forgot to delete references to it. It works without it, so I'll remove it.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I have had multiple failures including another one here because cc1.exe cannot be found. It only works if I explicitly put it in the PATH for some reason.
ksh93/PKGBUILD
Outdated
makedepends=( | ||
'gcc' | ||
'gdb' | ||
'make' | ||
'gettext' | ||
'gettext-devel' | ||
'flex' | ||
'bison' | ||
'unzip' | ||
'pax' | ||
'tar' | ||
'libiconv-devel' | ||
'ncurses-devel' | ||
'gmp-devel' | ||
'mpfr-devel' | ||
'mpc-devel' | ||
'isl-devel' | ||
'procps-ng' | ||
) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There's no way all of these are actually needed.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't know what I was doing honestly. I just saw someone else from the ksh93 repo say you should install that and just dropped it in.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
makedepends=( | |
'gcc' | |
'gdb' | |
'make' | |
'gettext' | |
'gettext-devel' | |
'flex' | |
'bison' | |
'unzip' | |
'pax' | |
'tar' | |
'libiconv-devel' | |
'ncurses-devel' | |
'gmp-devel' | |
'mpfr-devel' | |
'mpc-devel' | |
'isl-devel' | |
'procps-ng' | |
) | |
makedepends=( | |
'gcc' | |
'gettext-devel' | |
'libiconv-devel' | |
'ncurses-devel' | |
) |
maybe adding back
'flex'
'bison'
'gmp-devel'
'mpfr-devel'
'mpc-devel'
'isl-devel'
if it's demonstrated they're actually used
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm not going to include libiconv-devel because it completely fails if the GNU one is installed, I have to patch out looking for the system wide libiconv and use the bundled one instead.
ksh93/PKGBUILD
Outdated
build() { | ||
cd "${srcdir}/ksh-${pkgver}" | ||
|
||
export CYGWIN_CHOST="${CHOST/-msys/-cygwin}" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this is used in other PKGBUILDs as an argument to configure. I don't see you using it anywhere here, and I doubt ksh93's build scripts expect it to be set. Maybe you could try local MSYSTEM=CYGWIN
and see if anything useful changes?
ksh93/PKGBUILD
Outdated
install -Dm0644 "src/cmd/ksh93/${_docs}" "${pkgdir}/usr/share/doc/ksh/${_docs}" | ||
done |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
formatting
ksh93/PKGBUILD
Outdated
|
||
package() { | ||
cd "${srcdir}" | ||
install -dm0755 "${pkgdir}/etc/skel" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
were you intending to put something in /etc/skel?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Probably shouldn't have had that line, I was copying from the arch linux one I found on aur.
ksh93/PKGBUILD
Outdated
arch=('x86_64') | ||
url="https://www.github.com/ksh93/ksh" | ||
license=('spdx:EPL-2.0') | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
maybe add a backup
for etc/ksh.kshrc ?
backup=('etc/ksh.kshrc')
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is the leading slash supposed to not be there?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
it is not.
MSYS2-packages/filesystem/PKGBUILD
Lines 18 to 20 in 5feac36
backup=('etc/fstab' 'etc/shells' 'etc/profile' | |
etc/bash.bashrc etc/skel/.bash{rc,_profile} | |
etc/nsswitch.conf) |
Depends. In this case you could try patching it to
|
Speaking of cygwin, I have the option to dynamic compile ksh93, would you prefer that? |
Just a note, @jeremyd2019 , it's not optional to put the explicit PATH right now for cc1.exe. It won't work otherwise. Everything else was highly simplified. |
ksh93/PKGBUILD
Outdated
cd "${srcdir}/ksh-${pkgver}" | ||
|
||
# For now, it literally fails if cc1.exe is not explicitly on the PATH. | ||
local GCC_DIR="/usr/lib/gcc/x86_64-pc-msys/13.3.0" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
local GCC_DIR="/usr/lib/gcc/x86_64-pc-msys/13.3.0" | |
local GCC_DIR="$(gcc -print-search-dirs | sed -ne '/^install: /s///p')" |
ksh93/PKGBUILD
Outdated
# For now, it literally fails if cc1.exe is not explicitly on the PATH. | ||
local GCC_DIR="/usr/lib/gcc/x86_64-pc-msys/13.3.0" | ||
export PATH="$PATH:$GCC_DIR" | ||
export CYGWIN='ntsec' |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
export CYGWIN='ntsec' |
Setting CYGWIN
has no effect on MSYS2. Also, the ntsec option is obsolete and removed anyway. Is this just to shut up an error in their build script that is blindly insisting you set that option?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is this just to shut up an error in their build script that is blindly insisting you set that option?
Yes, this build system is like I said, highly ridged and expects things to be set a certain way. Even if it does nothing at all. I'm thinking of reverting the ksh-mamake patch since that failed as well.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
that patch did seem to help, but then because you were specifying host system of cygwin.i386 it then also triggered the stupid CYGWIN
env var check. Maybe if you did like I originally suggested and changed the case to cygwin.*|msys.*)
. or leave the host as cygwin.i386 and comment the setting of the CYGWIN
env var that it is known that it does nothing but is there just to shut up the stupid build system.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm going to make it think this is a cygwin system. Because if I don't, it'll think it's an unknown system and won't let me build as a dynamic libraries. Do you want me to make it dynamic libraries? I'll add back in CYGWIN='ntsec'
and HOSTTYPE=cygwin.i386
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Unless other packages will use the libraries, I don't think it adds much, since there's only a couple of exes anyway, and they'll necessarily be rebuilt if the library changes since they're part of the same package.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm going to make it think this is a cygwin system. Because if I don't, it'll think it's an unknown system and won't let me build as a dynamic libraries. Do you want me to make it dynamic libraries? I'll add back in
CYGWIN='ntsec'
andHOSTTYPE=cygwin.i386
.
That might not work for the ksh93 builtin getconf(1) and uname(1) commands, because then getconf(1) and uname(1) will think this is Cygwin, and will return identification information for Cygwin instead of MSYS2.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
why wouldn't uname(1)
just call uname(2)
and get the correct information? I assume getconf(1)
should also be backed by an API
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
MSYS2 changes its uname output depending on the setting of the MSYSTEM
environment variable, so if it doesn't call the system uname
function, it will need to duplicate that logic. I don't think anybody wants that.
ksh93/PKGBUILD
Outdated
CC="/usr/bin/gcc" /usr/bin/bash bin/package make PACKAGEROOT="$PWD" HOSTTYPE="cygwin.i386" || true | ||
|
||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
CC="/usr/bin/gcc" /usr/bin/bash bin/package make PACKAGEROOT="$PWD" HOSTTYPE="cygwin.i386" || true | |
CC="/usr/bin/gcc" /usr/bin/bash bin/package make PACKAGEROOT="$PWD" HOSTTYPE="cygwin.i386" |
ksh93/PKGBUILD
Outdated
makedepends=( | ||
'gcc' | ||
'gettext' | ||
'gettext-devel' | ||
'ncurses-devel' | ||
'gmp-devel' | ||
'mpfr-devel' | ||
'mpc-devel' | ||
'isl-devel' | ||
) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
there are more whitespace issues now. For one thing, PKGBUILD files should indent with 2 spaces. The arrays when split onto multiple lines should be lined up using spaces, like
makedepends=( | |
'gcc' | |
'gettext' | |
'gettext-devel' | |
'ncurses-devel' | |
'gmp-devel' | |
'mpfr-devel' | |
'mpc-devel' | |
'isl-devel' | |
) | |
makedepends=('gcc' | |
'gettext' | |
'gettext-devel' | |
'ncurses-devel' | |
'gmp-devel' | |
'mpfr-devel' | |
'mpc-devel' | |
'isl-devel') |
also, your ksh-mamake.patch
changed whitespace/indentation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I was puzzled why this was happening, but I discovered that because I was using nano, it by default inserted a literal tab which bonked with git when it was expecting spaces. I'll fix that.
Did you package the shell script compiler (/bin/shcomp.exe) too? |
Yup. If it matters, there is no bin on msys2, using for example Here is the entire tree of the .zst package:
|
Thank you for packaging shcomp. One request: could you also please package the ksh93 tests/ dir? I think we need it to find Cygwin and MSYS2 signal and filesystem bugs. |
Sure, let me finish up my build script you requested in the other thread then I'll make the modification. Where should tests goto? /usr/share? |
https://github.com/kofemann/ms-nfs41-client/blob/master/cygwin/cygwin_ksh93/ksh93.cygport puts the tests into usr/share/ksh/tests/... |
Done, if it works after 15 minutes of building, you can test it. |
It worked, all the .sh scripts are under /usr/share/ksh/tests. Anything else that needs to be done? |
I managed to debug and remove a bunch of the hacks. Regarding the |
@jeremyd2019 |
I pushed here, so you can see for yourself what I did. |
Disregard what I asked, this would be a question for upstream about the ramifications of the patched files. |
most of your weird problems were due to the |
That makes sense, What I'm going to do, is try to merge to upstream these patches, that are from observation incredibly minor so that someone else building ksh93 can get it without issue and not have to get lucky coming across this github merge request discussion. |
Re Why /usr/ast/bin ? "AST" stands "Advanced Software Technology", it's the evolution of the original UNIX SystemV utilities, updated to conform to XPG8 (OpenGroup/POSIX). The utilities were developed by David Korn, Glenn Fowler, Kiem-Phong. Vo&co. Since some POSIX utilities are builtin into ksh93 they need to be looked-up in the correct manner, and that's why some utilities are bound to /bin+/usr/bin (if they are compatible to the platform default utilities), /usr/gnu/bin (if they are compatible to the GNU utilities), /usr/xpg[468]/bin for various POSIX standards etc. Finally there needed to be a default if the platform does not have one, and that's why /opt/ast/bin is the pseudo-default. But /opt is ONLY for "optional add-on software", and not for something like a system shell. That's why we have - for Cygwin - the tmp_gnulinux_builtin_header.h header, with a fixed list of builtins, all tested for compatibility with /bin+/usr/bin. And to /usr/ast/bin all available utilities are mapped too, so scripts can enable them by just sticking "/usr/ast/bin:" in front of PATH. BTW: Please use Google for counting - more scripts use /usr/ast/bin than /opt/ast/bin ... |
The problem is the total "balkanisation" of utility defaults in Linux, because noone wants to honor or cooperate with the LSB (Linux Standards Base) anymore. ksh93 suffers from maintainers only shipping the MINIMUM (and technically untested) defaults from the package. Correct builtin utility path bindings requires testing and maintenance, and since noone wants to do that we end-up with the current situation. For Cygwin the situation is better, because I used by knowledge of SUN's ksh93-integration (i.e. update Solaris /bin/sh to ksh93)+POSIX utility modernisation (i.e. update Solaris /usr/bin to AST&T AST utilities+XPG8 conformance testing) projects, and just did the testing+validation work. |
@gisburn |
Debian's package doesn't seem to perform this override either.
|
And for Linux the situation is easy to fix:
For Cygwin and Solaris/Illumos I did that work already. |
No. Cygwin is meant for people doing UNIX stuff on Windows. MSYS2 is only meant to provide just enough UNIX stuff to get open source software building natively for Windows (without the posix translation layer that Cygwin provides and MSYS2 mildly patches) |
I'm not personally convinced about overriding defaults in such a way as to diverge from Linux distributions. @lazka ? Do you think this package is even in-scope for MSYS2-packages? |
Build as Cygwin. Add check function. the sigchld tests take A LONG TIME. Several tests fail, but those can be investigated later.
|
FYI UWIN (the original Windows software package from AT&T, basically competition to Cygwin) is using /usr/ast/bin, and so do Cygwin, Solaris, AIX, Illumos. |
https://cygwin.com/packages/package_list.html - I only see mksh there. If you mean your own group's unofficial Cygwin package, I don't find that particularly compelling. But I'm done with this PR. Maybe somebody else with write access will be persuaded, or you could maintain your own unofficial package that does what you want, like you already do for Cygwin. |
How can I install this package via pacman? |
That's easy, once you build it successfully, type |
That does not help me, because I need to do that for 5200+ machines, and I should only install the "official packages". Everything extra needs the signature of my professor. Why is the "ksh93" package not in the official packages yet? |
I got burned out on this PR, and no one else appears to have taken it on to decide whether to merge it as-is (conforming to Linux precedent) or including a sizeable custom header to make it more like Solaris instead. |
Is there anyone who is the final say on a package? I just want to finish this already too. |
I would go for the "Solaris solution" [ |
@jeremyd2019 |
On Mon, Apr 14, 2025 at 12:23 AM xeiavicaflashrepository < ***@***.***> wrote:
@jeremyd2019 <https://github.com/jeremyd2019>
Can we please just go with the solaris solution and be done with this?
—
Reply to this email directly, view it on GitHub
<#5243 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/BJDU22PQ6QFWTBMLKZPAPQD2ZLPXNAVCNFSM6AAAAABYUIVF3SVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDQMBQGE2TEOJXGU>
.
You are receiving this because you are subscribed to this thread.Message
ID: ***@***.***>
*xeiavicaflashrepository* left a comment (msys2/MSYS2-packages#5243)
<#5243 (comment)>
@jeremyd2019 <https://github.com/jeremyd2019>
Can we please just go with the solaris solution and be done with this?
I agree with @xeiavicaflashrepository and Takeshi-san, please use the
solaris solution.
Mark
--
IT Infrastructure Consultant
Windows, Linux
|
Ksh93 build system is a series of shell scripts and custom make program. Why? I don't know, historical reasons?
Problem arises with fork() during the compilation process. After about 5 minutes, it finishes with libast, then begins libsum... which fails due to I think something related to ASLR? Not sure, can this be disabled by non admin?
If anyone knows more than I do, please help.