IME improvement ideas

[ian-h-chamberlain]

Hello!

Copied from the itch.io comments:

One comment I have is that I really struggled with some of the IME-like input since the behavior seems to not be quite the same as what I'm used to. Some examples:

enpi -> えんぴ (game requires ennpi. IME allows n followed by any consonant, I think)
tu -> つ (requires 'tsu')
ti -> ち (requires chi)

There might be a few others as well I didn't notice or which I don't tend to use, (e.g. li -> ぃ, xyo -> ょ).

But, especially since this is a time-sensitive kind of game, it would be nice to support some of these common shortcuts. Maybe it would be useful to use a library like https://docs.rs/wana_kana/

I had a couple thoughts based on your reply. First, I just wanted to share this article I found which is related, but unfortunately their advice is primarily targeted to native applications and I don't think it can really be applied to web apps...

So, other potential options I could see working:

1. Develop an IME for every possible language used in word list (seems... impractical at best, impossible at worst)
2. Accept user input via an HTML box or something like that (basically, players bring their own IME. Might be annoying to glue together and look weird)
3. Handle some basic forms of input and allow word lists to define a mapping for non-ascii characters (basically the current model, but with some changes)

Option 2 probably has the best support for all possible languages, but would require some glue code that can handle shuffling keyboard input from browser to the game, and might cause other input problems if you want to use things like arrow keys, etc.

Option 3 seems like the least work, and would probably entail just using a more robust conversion layer from ascii input to language output (check more possibilities while matching characters). If you wanted to go that route, here are some ideas about ways you could support multiple languages:

• In the word list, include a language designator indicating the target language
  • You could use this to decide whether to enable specific-language IMEs like Japanese/Korean/Pinyin
  • This raises the question of how to deal with "unsupported" languages
• Create mappings from ASCII characters -> target characters, some examples follow. This could easily get messy and be easy to miss things, I think. It also doesn't scale for e.g. Chinese which I believe would require thousands of mappings to be usable
  • e -> é
  • ss -> ß
  • ka -> か | カ

All around, I think this is a tricky issue so it may come down to what you are willing to give up vs how seamless you want the user experience to be. Hopefully this gives you some ideas to play with, and if you're interested I could even try implementing something to see how it is in practice. Let me know what you think!

[rparrett]

Thanks for gathering all this here. It could be that some of my goals aren't completely compatible and some tradeoffs need to be made. Also, it's not a goal to support every language. But I want to try to avoid creating a situation that's less flexible than what we have now.

Option 1, if rephrased as "make it possible for a motivated person to contribute an IME for the language they are interested in" is a little less crazy, but perhaps still a bit out there in the effort/reward dimension. And there's some overlap here with Option 3 depending on what your definition of IME is.

Option 2 seems like a bit of a non-starter to me.

• I think that visual/audible in-game feedback while typing is super important.
• It's a (minor) goal to get native builds working again

Option 3 was the general direction I had in mind, but I hadn't thought it through very far. My thought was to

• extend what's happening here:

  taipo/src/data.rs

  Line 87 in 3c95fb0

  WordListKind::Parenthesized => parse_parenthesized(&word_list.string)?,

  to include a WordListKind::Japanese that uses something like wana_kana to process kana
• keep support for parenthesized ascii for kanji
• update TypingTarget and various systems in typing.rs to support multiple ascii chunks for each rendered chunk

minor note: I am not attached at all to the use of nom in there.

[rparrett]

And definitely feel free to experiment. I'd be happy to merge some code if it's going in a general direction I like.

[rparrett]

Here's some highly experimental/incomplete Japanese parsing related to Option 3.

https://github.com/rparrett/japanese-parser-test/blob/main/src/main.rs

println!("{:?}", japanese("京(kyou)都(to)とかんだとひろしま"));

TypingTarget {
  displayed_chunks: ["京", "都", "と", "か", "ん", "だ", "と", "ひ", "ろ", "し", "ま"],
  typed_chunks: [["kyou"], ["to"], ["to"], ["ka"], ["n", "nn"], ["da"], ["to"], ["hi"], ["ro"], ["shi", "si"], ["ma"]]
}

This was somewhat inspired by https://github.com/uzimith/romaji, but went off the rails a bit. It might be hard to get things like standalone sutegana working with this exact scheme.

edit: implemented this in Taipo in the PR below.