Treat U+FF5E as CJK punctuation (#16832)

This commit is contained in:
Tatsunori Uchino authored and GitHub committed 2024-12-16 16:26:22 +08:00
1 parent a6cdd05466
commit 88c210e844
5 files changed
+34 -2

No files matched your search

+22
View File
@@ -0,0 +1,22 @@
#### U+FF5E as CJK punctuation (#16832 by @tats-u)
U+FF5E FULLWIDTH TILDE (~) is commonly used as a substitute for U+301C WAVE DASH (〜) in Windows for Japanese. Full width alphabets are less used in Markdown documents comparing to other types of documents (e.g. Microsoft Office documents), and the full width tilde is much less used as this purpose compared to full width alphabets and digits. Therefore, we can assume that the full width tilde in Markdown documents in the wild are a alternative form of the wave dash and a part of CJK punctuation.
<!-- prettier-ignore -->
```markdown
<!-- Input (--prose-wrap=never) -->
a 字 a 字 a 字
60~
100点
60〜
100点
<!-- Prettier stable -->
a 字 a 字 a 字 60~ 100点 60〜10点
<!-- Prettier main -->
a 字 a 字 a 字 60~10点 60〜100点
```
The first symbol between 60 and 100 in the above example is U+FF5E FULLWIDTH TILDE (~) and the second one is U+301C WAVE DASH (〜).
+1
View File
@@ -98,6 +98,7 @@
"Friedly", "Friedly",
"frobble", "frobble",
"ftest", "ftest",
"fullwidth",
"Georgii", "Georgii",
"gettin", "gettin",
"githook", "githook",
@@ -84,6 +84,7 @@ const PUNCTUATION_REGEXP = new RegExp(
new Charset(...asciiPunctuationCharacters).toRegExp("u").source, new Charset(...asciiPunctuationCharacters).toRegExp("u").source,
...unicodePunctuationClasses.map( ...unicodePunctuationClasses.map(
(charset) => `\\p{General_Category=${charset}}`, (charset) => `\\p{General_Category=${charset}}`,
"\u{ff5e}", // Used as a substitute for U+301C in Windows
), ),
].join("|")})`, ].join("|")})`,
"u", "u",
@@ -283,7 +283,6 @@ English
[ウ [ウ
ィキペディア]: https://ja.wikipedia.org/ ィキペディア]: https://ja.wikipedia.org/
C言 C言
語 語
・ ・
@@ -293,6 +292,10 @@ Go
・ ・
Rust Rust
U+301C〜
U+FF5E~
U+1F221🈡
=====================================output===================================== =====================================output=====================================
日本語、にほんご。汉语, 中文. 日本語,にほんご.English 日本語、にほんご。汉语, 中文. 日本語,にほんご.English
words!? 漢字!汉字?「セリフ」(括弧) 文字(括弧)文字【括弧】日本語English words!? 漢字!汉字?「セリフ」(括弧) 文字(括弧)文字【括弧】日本語English
@@ -309,5 +312,7 @@ words!? 漢字!汉字?「セリフ」(括弧) 文字(括弧)文字【括
C言語・C++・Go・Rust C言語・C++・Go・Rust
U+301C〜U+FF5E~U+1F221🈡
================================================================================ ================================================================================
`; `;
@@ -58,7 +58,6 @@ English
[ウ [ウ
ィキペディア]: https://ja.wikipedia.org/ ィキペディア]: https://ja.wikipedia.org/
C言 C言
語 語
・ ・
@@ -67,3 +66,7 @@ C++
Go Go
・ ・
Rust Rust
U+301C〜
U+FF5E~
U+1F221🈡