GeezerVison is showing github.com — segfault with extended globs · Issue #1464 · att/ast
This repository was archived by the owner on Jun 16, 2025. It is now read-only.
This repository was archived by the owner on Jun 16, 2025. It is now read-only.
segfault with extended globs #1464Descriptionthis code
i=0
v=a
s=
while ((i < 10000)); do
((++i))
s+=$v
done
[[ $s == +($v) ]]
reliably segfaults with current ksh2020, the problem is in this case inherited from ksh93v- (which in turn inherited it, for a change, from ksh93u+, so this is not a regression but an original ksh93 bug I presume).
the extended glob pattern in the final test is causing the segfault. Activitykrader1961 commented on Feb 1, 2020 Bug reports are great and welcomed. But given your intense interest in a project you've repeatedly claimed has been "hijacked" I'd be more impressed if you actually provided an analysis of the bug and, ideally, a fix. If nothing else you could have provided a backtrace of the failure. krader1961 commented on Feb 1, 2020 A backtrace of the failure shows the stack being exceeded. Here are just the last few lines of the backtrace:
frame #27214: 0x000000010fc020c5 ksh`parserep + 393
frame #27215: 0x000000010fc00324 ksh`regnexec_parse + 4989
frame #27216: 0x000000010fc00d20 ksh`regnexec_parse + 7545
frame #27217: 0x000000010fbfeced ksh`regnexec + 1125
frame #27218: 0x000000010fc17633 ksh`strngrpmatch + 387
frame #27219: 0x000000010fb7d411 ksh`test_strmatch + 157
frame #27220: 0x000000010fbd9904 ksh`sh_exec + 6164
frame #27221: 0x000000010fbb48cd ksh`exfile + 1437
frame #27222: 0x000000010fbb543c ksh`sh_main + 1779
frame #27223: 0x00007fff654307fd libdyld.dylib`start + 1
frame #27224: 0x00007fff654307fd libdyld.dylib`start + 1
Note the number of call frames. The three frames from #27214 to #27216 repeat until the stack size limit is exceeded. I would argue this is not a bug in the usual sense of an implementation bug; e.g., a buffer overflow or off by one error. But there is a strong argument to be made that it is a design bug in how the ksh regexp engine handles that pattern using recursion. Especially since, in this case, the shortest match is all that is needed and thus no recursion whatsoever is necessary even if recursion is otherwise required to match other inputs. krader1961 commented on Feb 1, 2020 Also, if you're using ksh for this type of problem you are using the wrong tool for the job. There are problems where ksh is the best tool. This isn't one of them. Google "if all you have is a hammer." Reacted by tibfulv Reacted by GutZuFusss jghub commented on Feb 1, 2020
Google "if all you have is a hammer."
this coming from you... so you can be funny (even if only involuntarily).
but let's leave it at that here: segfault reported. this one of the "111" kind (present in ksh93u+, present in ksh93v-, present in ksh2020). put it on the list.
oh, and maybe count the "111" segfaults and user-facing operational bugs you know of. and than count the bugs in the "011" set (only present in 93v- and 2020 but working in 93u+: I am suspecting this is the largest set) and the :"110" and "010" ones (that would be real fixes in 2020). and the "001" (only buggy in 2020) ones. jghub commented on Feb 1, 2020 you also wrote:
But there is a strong argument to be made that it is a design bug in how the ksh regexp engine handles that pattern using recursion. Especially since, in this case, the shortest match is all that is needed and thus no recursion whatsoever
what you state is wrong in multiple ways:
-
we are not talking "regexp" matching here but "extended glob patterns". I am semi-confident that you understand there is a difference and you just used the word "regexp" carelessly. but to be on the safe side this needs to be pointed out.
-
obviously, you don't understand extended glob patterns properly. +(a) means "only a's from start to end of the line", i.e. it is equivalent to the regexp ^a+$. so the matcher has to absorb the whole string to decide whether it matches. you can not restrict the match to the leading substring.
-
ksh's regexp matcher (which it also supports, in case you don't know) does not have the problem:
i=0
v=a
s=
while ((i < 10000)); do
((++i))
s+=$v
done
[[ $s =~ ^a+$ ]] && print match!
works just fine. jghub commented on Feb 1, 2020
Bug reports are great and welcomed. But given your intense interest in a project you've repeatedly claimed has been "hijacked" I'd be more impressed if you actually provided an analysis of the bug and, ideally, a fix. If nothing else you could have provided a backtrace of the failure.
fake fact fighting cont'd:
-
I don't have an "intense interest" in this project. I do have interest in availability of a fully functional ksh93 -- and ksh2020 is becoming something different and inferior in your hands.
-
it was someone else who described what has happened to the ast project as "hijack". you might check this and also whether I've adopted and used that term at all afterwards (I don't think so).
"hijack" is close but I'd rather describe the situation as a "hostile takeover" since the ksh community very obvious and nearly unanimously does not agree with your actions. and "hostile" is a good summary of your overall attitude anyway.
concerning the rest of your comment: nobody wants to impress you. nobody is impressed by you. you received a bug report. react any which inappropriate way you want. Reacted by Jens Elkner, Jens Maus and GutZuFusss krader1961 commented on Feb 2, 2020
react any which inappropriate way you want.
Okay, I'm closing this since the problem is due to an algorithm chosen long ago by the old AST team. Their choice of a recursive algorithm for that pattern is a pretty common choice. It is not an implementation bug. We're not going to change the algorithm they implemented since doing so would be incredibly risky with respect to backward compatibility.
@jghub I'll grant that you know more about the behavior of the ksh language than I do (despite my using ksh for more than 15 years). Heck, you might be the leading expert on the planet on the topic of ksh behavior. But you are rude, argumentative, and obnoxious. Please, go away. jghub commented on Feb 2, 2020
I'll grant that you know more about the behavior of the ksh language than I do (despite my using ksh for more than 15 years). Heck, you might be the leading expert on the planet on the topic of ksh behavior.
knowing more about KornShell language than you do is not difficult, unfortunately. so let's say I am just an average "real" ksh user while you are not and your grasp of the KornShell language and the ksh communities' needs is decidedly sub-standard (and manifestly inadequate for maintaining future ksh development).
if you would appreciate (and be willing to respect/follow) expert advice on finer details of core ksh (POSIX) behaviour I'd point you to @mirabilos -- if he could be bothered to suffer your ways gladly.
But you are rude, argumentative, and obnoxious. Please, go away.
I trust people will be able to judge by themselves to whom these attributes apply consistently in the first place. jghub commented on Feb 2, 2020 @siteshwar:
I'd expect that you reopen this issue. a reproducible segfault is a bug. what else? krader is acting and arguing foolishly by closing it. he seems to draw satisfaction from this sort of actions.
I have reported the bug intentionally very concisely and did not want to get into another stupid argument with krader but --- see above. I also made it very clear that this is broken even in ksh93u+ (which is the rare case in this project not the regular one).
I agree with krader that this bug cannot be fixed easily (not by him, that's for sure) and it might be there to stay. in which case it needs a clear documentation in the manpage under KNOWN BUGS/LIMITATIONS in my view (and hint at using regexp as a workaround). but it needs to remain documented in this project as a BUG. I do hope you do agree and will reopen the issue accordingly. jghub commented on Feb 2, 2020 @siteshwar:
I also would request that you break your silence and speak out how you view the state of affairs in this project. are you still caring or have you given up? I feel it is getting more alarming and ridiculous by the day. and there is a single person responsible for this sad situation: krader. it most definitely is not just a problem of "deficient communication skills" as you put it somewhere, I believe. many different people have voiced similar opinions by now. nobody has ever welcomed what krader is doing to ksh.
you know my position:
I believe he should do his thing in a private fork (where he might alter ksh beyond recognition without affecting anybody) but he definitely has no beneficial influence or purpose in the "official" ksh project. so he should not have commit rights. it was a grave error to grant him those rights (but apparently that was not obvious at the time). you should acknowledge that and correct this error.
what needs to happen seems rather clear: get together some sane and competent people and restart from ksh93u+ (presuming that nobody appears that is able to fully understand the ksh93v- code and to complete D. Korn's work to create a "ksh93v+").
"phase one" would be to replace the build system either by this meson/ninja thing (possible) or just by gmake/Cmake (preferable, probably). that is doable and would solve the single central issue of all the package maintainers out there as far as I have understood the situation: make ksh93u+ build easily and maintainable in the different distros with standard build tools.
considering all the breakage that still lurks in the ksh93v- heritage of ksh2020 and the additional booby traps planted by krader's actions of two years (case in point: what has happened regarding reverting the silly changes to cd in #1445? nothing, I bet) , considering the distinctly inferior performance of ksh2020 (#1449) while offering not a single new feature in comparison to ksh93u+ that I am aware of (I know that 93v- had new features but they do not yet work reliably and have already partly been removed by you like the "toupper", "tolower" string processing stuff...), it is very very obvious that ksh93u+ is a better choice as "the future of ksh" than ksh2020.
so the project now first and foremost should concentrate on 93u+ w/o for the time being touching the code at all (beyond what is necessary to make it build with gmake/Cmake).
I believe that almost everybody out there would be rather happy with this situation: ksh93u+ is already good enough for almost all purposes I'd say. but of course it does contain certain bugs (the one I reported here is one example). so "phase two" then would mean to start trying to fix known (or newly reported) real bugs of 93u+. but carefully and thoroughly. it goes without saying that your and krader's work at ksh2020 than could serve as one source where to look for successful fixes to problems that are common to 93u+ and 93v-.
it would be good if this strategy could be agreed on and further interference of krader be prevented.
so, what do you think? jghub commented on Feb 2, 2020 @gordonwoodhull
@johrstrom
@kindofblue
@MarshM
@s-u
@tremaineeto
dear owners of the att/ast repository,
having been encouraged to do so by some followers of this project and after having spend a couple of months trying to provide constructive feedback completely unsuccessfully I'd like to bring to your attention the sorry state of this project.
I would appreciate if you would take the time and closely look at what is happening to this repository and to assess what damage is done to the reputation of D. Korn and ATT by kurtis rader (krader1961) who effectively has taken over command in this project and is severely misbehaving while also harming the code base.
please read, e.g., through the full history of the present issue and look at the full #1449 issue or this conclusion #1449 (comment)
there are also multiple instances of diverse people objecting against what is happening here over the full history of the project if you go back to the history of this project. #88 is just one early example for the complete misguidedness and ignorance of krader
a few further examples of the inadequacy of krader's personality as the de-facto lead (or even a contributor) to this project:
"Gah! Don’t get me started about the issues in the nameref code. It’s a steaming pile of caca." see here: https://gitter.im/ksh93/users
"> P.S., I am deliberately not including the mailing list address in this reply. Same as my earlier, private, replies in order to avoid making disagreements that are personality based public. That you are deliberately making them public tells me you are an asshole." see here:
https://groups.google.com/d/msg/korn-shell/TVpHd5rrI5g/8Cj6ir7NAwAJ
someone acting in this way should no longer have commit access to this repository in my view. Reacted by Danny Weldon, Jens Elkner and GutZuFusss jghub commented on Feb 4, 2020 principally, I agree with everything @gordonwoodhull has proposed/decided: thank you.
I disagree with @siteshwar that this would be the "death of ksh". rather to the contrary. going on without a clear cut would do decisive further (maybe mortal) harm to ksh. and the 'visibility' he talks about seems to be mostly spurious (or identical with the visibility here where there is near unanimous agreement that ksh2020 went off on a bad tangent right from starting with ksh93v-). maybe @siteshwar can reconcider his position to prevent fragmentation?
I also second @kdudka that downstream users are just interested in a predictably working true ksh93(u+..., for the time being) and the consequences this has for distros (not only RH). except that I am not sure (and have not seen statements to that extent around here that I can recall) that support of ancient HW/SW (beyond strict backward compatibility of ksh language-wise) is a crucial issue for a sizeable part of the community. so maybe there is no collision of interests regarding your second point at all, making it a non-issue?
@kdudka: regarding how exactly to make the transition, I am not sure whether I completely understand your proposal/intention. but the main idea seems to keep the present repo (instead of starting over from "clean" repo) and restructure it? maybe that would be good, but depends on what exactly is the plan.
I would like the idea of putting ksh2020 on a "dead end" branch (if that is the plan) to maintain easy access to the issues (however questionable their factual correctness, partly...) and fixes that are relevant for 93u+ (possibly not so many since most fixes related to ksh93v- and ksh2020 only problems...) . but by no means should that branch see then further activity in my view: ksh2020 development (= krader) needs to happen in a fork and the ksh2020 branch in the present repo should not accept pull requests but be in a "sealed/archived" state just for reference. development would exclusively happen for the ksh93u+ line. I would find this clear separation of ksh2020 and ksh93 mandatory for all the reasons that have let to the present need of a reset and in order to definitely avoid replication of the confusion out there between ksh93 proper and ksh2020. the distinction needs to be very clear.
it then seems now consensus (except with @siteshwar, possibly) that master should be rolled back to prestine 93u+. this would be good. after changing the build system (and nothing else!) that would provide the legacy ATT ksh93u+ shell for distros to use (modulo first including the known solid patches from different source but without overdoing it at this point?).
if that state is reached the many year hiatus in easy ksh93u+ availability would be at an end (which is the only real danger for ksh's survival in my view). that would probably be the time to put the result on a "ksh93u+-legacy" branch.
and further bug fixes could than happen on master. but with a policy similar to the one observed by the AST team of D. Korn et al.: carefully and one by one and with a very conservative attitude of integrating patches/pull request after thorough discussion and testing (in order to avoid duplication of the ksh2020 mess with a multitude of cuts and behavioural and feature changes/deletes nobody ever kept track of or cared to document beyond commit comments of questionable quality and reliability).
if this is of interest: @jens-maus has signalled principal willingness to look into the build system issue (considering Cmake so far as the way to go). I am confident he would do a good job. I would be able to help in testing (not so much with the Cmake stuff). any opinions whether this would be welcome? or whether to categorically stay with autoconf? or even meson (but most people do not seem to like that road)? Reacted by Gordon Woodhull siteshwar commented on Feb 4, 2020
I disagree with @siteshwar that this would be the "death of ksh". rather to the contrary. going on without a clear cut would do decisive further (maybe mortal) harm to ksh.
We will see. I wish you good luck with your efforts. I will request to get my commit access revoked (it was going to happen anyways) and will move on. kdudka commented on Feb 4, 2020
@kdudka: regarding how exactly to make the transition
It does not really matter as long as the references to the issues and pull requests (and the references inside them) stay valid.
after changing the build system (and nothing else!) that would provide the legacy ATT ksh93u+ shell for distros to use
The problem is that this task is more difficult than one would expect. If you want to keep support for all the obscure HW and SW at the same time (as repeatedly requested here), it is nearly impossible.
and further bug fixes could than happen on master. but with a policy similar to the one observed by the AST team of D. Korn et al.: carefully and one by one and with a very conservative attitude of integrating patches/pull request after thorough discussion and testing
I think you are underestimating the amount of work needed for this. Most of the users report just issues without patches. Someone has to debug them, write the patches, and review the patches. Given the shape that ksh93u+ is currently in, I think it requires at least one full-time employee dedicated to this.
if this is of interest: @jens-maus has signalled principal willingness to look into the build system issue (considering Cmake so far as the way to go). I am confident he would do a good job. I would be able to help in testing (not so much with the Cmake stuff). any opinions whether this would be welcome?
This would certainly be welcome and I wish you good luck with that! Reacted by Gordon Woodhull
MetadataMetadataDevelopmentNo branches or pull requestsIssue actions
|