std.ascii isPunct returns true for several symbols

[jecolon]

According to https://en.wikipedia.org/wiki/ASCII#Character_set , the following characters are symbols, not punctuation:

'`', '~', '$', '^', '=', '+', '|', '<', '>'

It so happens that this is the case in Unicode too. In std.ascii, there's no isSymbol so these characters are covered by isPunct which differs from both ASCII and Unicode. I have a working version at https://github.com/jecolon/ziglyph/blob/main/src/ascii.zig but don't know if I should make a PR with it or let it be handled via this issue. That version adds isSymbol, with corresponding symbol table and tIndex.Symbol enum value.

[g-w1]

Feel free to make a pr, or I can.

[jecolon]

Ok, I'll do the PR!

[daurnimator]

ispunct is defined by C and posix as:

checks for any printable character which is not a space or an alphanumeric character.

zig follows this convention, which is inherited by many formats and protocols

[jecolon]

Interesting. So isPunct is C and POSIX compliant , not ASCII compliant, but it's in std.ascii . Maybe there should be an asciiIsPunct or isPunctAscii that's aligned with ASCII then?

[Mouvedia]

If #5019 was accepted you could have a std.posix.isPunct and a std.ascii.isPunct.

[jecolon]

If #5019 was accepted you could have a std.posix.isPunct and a std.ascii.isPunct.

That would definitely be a good thing; compliant, not misleading, less confusing.

[andrewrk]

Counter-proposal: delete ascii.isPunct and also don't define it in std.os or std.posix

[jecolon]

Counter-proposal: delete ascii.isPunct and also don't define it in std.os or std.posix

Yeah, that could be the best option to avoid confusion having multiple isPunct functions in different places.

[wooster0]

As of #12448 this function is deprecated so I think this issue can be closed:

zig/lib/std/ascii.zig

Lines 355 to 358 in 99c3578

	/// DEPRECATED: create your own function based on your needs and what you want to do.
	pub fn isPunct(c: u8) bool {
	return inTable(c, tIndex.Punct);
	}

It will be removed eventually along with the other stuff.

[jecolon]

As @r00ster91 points out, this is now a non-issue.

[mnemnion]

It seems this issue was decided without reference to the standard; Wikipedia is not authoritative.

According to the 2007 revision of the 1986 standard, there is an unambiguous term for characters which are not control, space, or alphanumeric: special characters. I would eat a hat live on camera if the 2017 revision changed that, but don't care to spend $60 to find out for sure.

The C and POSIX function ispunct identifies every character that the standard considers "special", and none which it does not. Therefore, a std.ascii.isPunct function identifying the same characters would also be referring to a cogent and unambiguous group of symbols within the standard. If it were desirable to avoid conflict with Unicode over the punctuation vs. symbol distinction (I can see a case for that), then std.ascii.isSpecial would be apropos, using the precise terminology referenced in the ASCII standard.

Eliminating a source of ambiguity between Unicode and ASCII was and is a reasonable choice, but it's left the predicates in std.ascii with a hole: there is a collection of characters which cannot be affirmatively classified, something users coming from C would certainly expect. I propose std.ascii.isSpecial, recognizing the exact set of characters as ispunct in the C standard, those referred to as special characters in the ASCII standard, be added to fill in that gap.