Page MenuHomePhabricator

VisualEditor: CE eats up syllables except a last syllable of a word in Korean IME
Closed, ResolvedPublic8 Estimated Story Points

Description

Author: ryuch

Description:
Korean input is not complete.

I tried to input "한글 시험 합니다.".
But it eats up some letters and produce "글 험 다.".

I tested with Chrome and Safari on Mac OSX. And it is all the same with Chrome on Windows XP.

See Also: T52105: VisualEditor: IME shift+ function for adding diacritical marks in Arabic broken T72353: Cannot type in Korean using Mozilla on Windows 7 with dual-script keyboard

Details

Reference
bz50631

Event Timeline

bzimport raised the priority of this task from to High.Nov 22 2014, 2:05 AM
bzimport set Reference to bz50631.

It also comes to Chrome on Win7.

David and I may take a look at this over a cocktail.

This looks quite a lot like an IME issue (at least with iBus Korean).

Trying to type into an empty/slug paragraph causes all sorts of whacky behaviour and should probably be filed as a separate bug

The latin text in iBus Korean is:

gks rmf [SPACE] tl gja [SPACE] gkq sl ek

Typing just "gksr" the following events are fired:

compositionstart
g,k,s
compositionend
compositionstart
r
compositionend

with the final compositionend causing the second Hangul character to end prematurely.

Need to investigate if this a browser bug related to IME's which allow continuous typing across graphemes.

(In reply to comment #4)

The latin text in iBus Korean is:

gks rmf [SPACE] tl gja [SPACE] gkq sl ek

Typing just "gksr" the following events are fired:

compositionstart
g,k,s
compositionend
compositionstart
r
compositionend

I'm seeing slightly different behaviour on Chromium 25.0.1364.160-0ubuntu0.12.04.1 using ibus 1.4.1-3ubuntu1 and ibus-hangul 1.3.1-3build1.

If I start with an almost blank document (just the letter "A" with the cursor immediately after), then I can type "gksrmf tlgja gkqslek" and get exactly the expected characters ""한글 시험 합니다". The event sequence is as it should be (with the second compositionend only happening when I hit space).

However, if I then type the full stop, it goes *after* the cursor.

ryuch wrote:

This was feed backed to
http://www.mediawiki.org/w/index.php?title=Thread:VisualEditor/Feedback/i_can%27t_type_a_east_asian_character

If you cannot resolve the problem in time, we'd better postpone the deployment on Korean Wikipedia or other Korean projects.

Yes, lack of reliable IME support would be a show-stopper.

I also tried to write some Korean words using visual editor
" 시 각 편 집 기 테 스 트 근데 잘 안되"
but it types
" 시 각 편 집 기 테 스 트 근데 잘 안되 되도안아잘자ㅈ데근ㄱ트스ㅋ크스ㅅ테ㅌㅍ페기집지편펴가" see http://ko.wikipedia.org/w/index.php?title=%EC%82%AC%EC%9A%A9%EC%9E%90%3A%EB%B6%84%EB%8B%B9%EC%84%A0M&diff=11132326&oldid=11087937 for details

ryuch wrote:

I think VE team is working on this issue. Did you change the code related this?
I think so.
DangSunM's report is not what I got when I made the first report.

Now I tested on Chome of iPad. It is different also.
I input '한글 시험 합니다.', same as previous.
And now I have '한글 싷험 합닏다.', like the think
http://en.wikipedia.org/w/index.php?title=User:Ryuch&curid=284811&diff=564158229&oldid=554104287
.

It duplicates the begining consonant of a syllable to the place of ending consonant of a syllable just before it in a word when the ending consonant is null.

Now the problem seems something changed, but It is yet hard.

I input '한글 결과는 언제 나오나요'

but in the en wikipedia's visual editor @ chrome ,
it have 'ㄱㅡㄹ ㅏㄴㄴ 제 오요나나제언ㅇㅏ결겨ㄱㅡ'.

See http://en.wikipedia.org/w/index.php?title=User:Galadrien/sandbox&diff=564998163&oldid=549762292 .

It also works on Chrome for WinXP/Win7/Android(though it is not supported).

@Ryu not deliberately! Come to the VE lounge tomorrow (hackathon room) and speak to me and David.

ryuch wrote:

awesome. anyway.
Please make it opt-in on KOWP.

I will catch you guys at the lounge.

ryuch wrote:

Test Cases

글험 합니다. 다닏ㄴ합하험허시ㅅ (2013.8.14, Chrome of Windows XP)

글 험 다. (2013.8.14, Chrome of Mac OS X)

한글 시험 합니다. (2013.8.14, Chrome of Ubuntu, iBus 1.4.1)

I believe the following patch resolves the problem on those platforms where there still was one: https://gerrit.wikimedia.org/r/#/c/79451 . At least, the patch seems to fix the Korean ibus method on Ubuntu+Chromium. However it's hard to be sure because of the problems reproducing the bug.

Change 80080 had a related patch set uploaded by Jforrester:
WIP:Don't emit Surface changes back to the Surface

https://gerrit.wikimedia.org/r/80080

Change 80080 merged by jenkins-bot:
Don't emit Surface changes back to the Surface

https://gerrit.wikimedia.org/r/80080

Given that this is now merged, I'm going to mark this as fixed. However, this is provisional - please re-open if you think that this has not worked!

Sorry, but If you applied it on en.wikipedia. it yet not fixed.

No, it hasn't been deployed to en.wiki yet.

We can check on en.wikipedia or ko.wikipedia, only,
if the problem fixed.

So can you apply it on there?

Yes. All code in master is eventually deployed during our scheduled deployments. If you want to test the code ahead of the deployment you will have to check it out locally.

(In reply to comment #22)

We can check on en.wikipedia or ko.wikipedia, only,
if the problem fixed.

So can you apply it on there?

The code will be applied as part of the normal MediaWiki release cycle, which you can see here: [[mw:MediaWiki_1.22/Roadmap#Schedule_for_the_deployments]].

This code is tagged against the release that happened this morning ("wmf14") - this means that the code is available now on MediaWiki.org, and will be available on all Wikipedias (including the English and Korean Wikipedias) next Thursday, 29 August, at approximately 18:00 UTC.

If you want to test and confirm the code, you can edit pages on MediaWiki.org like [[mw:VisualEditor:Test]]. Also, all code is immediately available on the "Beta labs" testing site, which can also be edited (note that it uses a different account system to production "real" Wikipedias): http://en.wikipedia.beta.wmflabs.org/wiki/VisualEditor

Hope this helps!

ryuch wrote:

(In reply to comment #16)

I believe the following patch resolves the problem on those platforms where
there still was one: https://gerrit.wikimedia.org/r/#/c/79451 . At least, the
patch seems to fix the Korean ibus method on Ubuntu+Chromium. However it's
hard
to be sure because of the problems reproducing the bug.

David,

https://gerrit.wikimedia.org/r/#/c/79451 did not fix the problem even Chrome on Ubuntu, if the code is applied to MediaWiki.org as James said.

I found another thing, when the cursor is after Korean letters backspace does not work.

(In reply to comment #25)

https://gerrit.wikimedia.org/r/#/c/79451 did not fix the problem

Just to avoid misunderstandings: How did you test this? Do you run a private wiki with recent VisualEditor from git master?

I found another thing, when the cursor is after Korean letters backspace does
not work.

Please file separate bug reports for separate issues. Thanks! :)

ryuch wrote:

(In reply to comment #26)

(In reply to comment #25)

https://gerrit.wikimedia.org/r/#/c/79451 did not fix the problem

Just to avoid misunderstandings: How did you test this? Do you run a private
wiki with recent VisualEditor from git master?

Not on a private one, I ran on MediaWiki.Org, specifically on [[mw:VisualEditor:Test]].

I found another thing, when the cursor is after Korean letters backspace does
not work.

Please file separate bug reports for separate issues. Thanks! :)

I think the space problem is caused by this bug. I suppose we need to fix error of synchronization between DM and CE.

Thank you, too.

There's code to address this bug in the following patch, which is due to go live by mediawiki.org on 13 September 2013:

https://gerrit.wikimedia.org/r/#/c/82858/

Please let us know whether it fixes the bug!

Marking this as "FIXED" on the expectation that it's fixed - please re-open if you find that it is still occurring.

ryuch wrote:

(In reply to comment #29)

Marking this as "FIXED" on the expectation that it's fixed - please re-open
if
you find that it is still occurring.

I am sorry. It is not fixed. I tested on mediawiki.org.

I open it again.

(In reply to comment #30)

(In reply to comment #29)

Marking this as "FIXED" on the expectation that it's fixed - please re-open
if
you find that it is still occurring.

I am sorry. It is not fixed. I tested on mediawiki.org.

:-(

I open it again.

OK, we'll look at it urgently. Sorry for this.

heesub wrote:

I am suffering from this issue, tested on the latest version from mediawiki's core.git: 217fd43. The odd thing is that it seems to work properly on Firefox which comes with Ubuntu 13.04 LTS distribution(, but not perfectly).

I've tried:

  • IE on Windows 7 : not working
  • Google Chrome on Windows 7 : not working
  • Google Chrome on Ubuntu 13.04 : not working
  • Safari on Mac OSX 10.8.5: not working
  • Google Chrome on Mac OSX 10.8.5: not working
  • Firefox on Ubuntu 13.04 : WORKING

I hope this information help you guys so that it gets fixed.

heesub wrote:

(In reply to comment #32)

I am suffering from this issue, tested on the latest version from mediawiki's
core.git: 217fd43. The odd thing is that it seems to work properly on Firefox
which comes with Ubuntu 13.04 LTS distribution(, but not perfectly).

I'd tested on VisualEditor.git: cba2935 and also with patchset v16 by David Chan. Not fixed.

Would it be possible to test against current master? We believe that we have fixed this (or, at least, change the behaviour) since you tested it with cba2935 (thank you!).

heesub wrote:

(In reply to comment #34)

Would it be possible to test against current master? We believe that we have
fixed this (or, at least, change the behaviour) since you tested it with
cba2935 (thank you!).

I've tested the latest version and uploaded a video clip which shows what is happening on VisualEditor with Korean characters. You can find the video on this link: http://youtu.be/-Q8n4vNONm0

I am not a JavaScript or web programming expert, it seems that every browser out there has different behaviors on firing and processing compositionStart/End and keyDown events. Take a look at this document: http://joone4u.blogspot.kr/2010/07/composition-events-are-handled_27.html

After digging this issue, I found some combinations which avoid this issue:

  • Safari, Google Chrome (maybe every other browsers, but not tested) on Mac OS X. and change preference settings of Korean IME: set 'Input by' to 'Word' not 'Syllable' which is default.
  • Firefox on Ubuntu

Hope this helps!

Thanks very much for the video -- that's a great help!

I agree about the events. Actually I think it's every browser/OS/IME combination, so it's really troublesome.

As far as I can tell, there are actually two issues at the moment:

(1) Entering Korean text into an empty paragraph, and
(2) Entering Korean text into a non-empty paragraph.

Your video shows that case (1) is still broken. In order to test (2), would it be possible to try typing "hello", and then start typing Korean immediately after the "o" (i.e. without starting a new line/paragraph)?

I have done these sorts of tests, but I can't reproduce the behaviour you show in the video, presumably because of some slight difference between our test platforms.

Thanks again!

I tried VE in [[en:User:Hym411/VETest]] with VE betafeatures enabled;

Input: "구글 크롬 안드로이드용 최신 버전"

Output: See the page above. - "국글 클롬 안들롱읻등용 쵯신 벚전"

I used latest release of Google Chrome for Android.

Also, comment 25 's 'backspace does not work' still happens to me.

Hmm, while reviewing kowiki VE tag , I find most edits have no problem.

In T52631#836024, @revi wrote:

Hmm, while reviewing kowiki VE tag , I find most edits have no problem.

Yes, it's unclear right now which IMEs people are using and what their different impacts in VisualEditor are. It's possible that by now Korean works pretty well for the majority of users. We should follow-up with detailed automatic tests of each specific IME's functionalities so we can assess whether or not it works.

Helo. It seems that finally most of Korean problems are solved.
But one more problem is seen in VE.

In the Windows 7 Chrome 64bit, when Hanguls first inputed and then spaced, the first word disappears.
And this seems not results from IME.

I input it Korean nalgaeset IME:
Input : 날개셋 입력기로도 비슷한 결과가 나오는걸 보니 IME 문제는 애시당초 아니었겠습네다
Output : 입력기로도 비슷한 결과가 나오는걸 보니 IME 문제는 애시당초 아니었겠습네다

Anyway, thank you for your continuous effort.

Same here.

구글 크롬 윈도 7 39.0.2171.71 m

resulted

크롬 윈도 7 39.0.2171.71 m

It omits first word (구글).
Tested on Win7 32bit Chrome version 39.0.2171.71 m.

Jdforrester-WMF lowered the priority of this task from High to Medium.Jan 15 2015, 12:16 AM
browserversioninputoutputdiff
IE11.0.9600.17959인터넷 익스플로러 테스트인터넷 익스플로러 테스트14721416
Firefox40.0.3파이어폭스 테스트파이어폭스 테스트14721436
Chrome44.0.2403.157구글 크롬 테스트구글 크롬 테스트14721446

OS: Win7, 32bit. IME: MS default one.

For diff, use Special:Diff.

Hi, typing seems to work for me in on en.wikipedia.org and on VE-standalone master, using:

  • Windows 8, MS or Nalgaeset IME, Firefox 40.0.3 or Chrome 45.0.2454.99 m
  • Ubuntu 15.04, Anthy IME, Firefox 40.0.3 or Chromium 44.0.2403.89 Ubuntu 15.04 (64-bit)

In particular, I'm not seeing the "first word disappears" behaviour. Can we close this bug, or is anyone still seeing corruption?

Seem to be resolved (idk which patch solved it).

It's good to see that the #1 reason I hate/blame VE is finally resolved.

Jdforrester-WMF set the point value for this task to 8.