SimpleStreamReader cannot read multibyte characters on buffer boundary

When reading character by character using UTF-8, SimpleStreamReader returns -1 (EOF) when it encounters an emoticon. The problem exists even when reading a lot of characters at once but is harder to reproduce (the four bytes of the emoticon must get misaligned with the buffer).

input.txt

The following code:

		try (Reader r = new SimpleStreamReader(new FileInputStream("input.txt"), StandardCharsets.UTF_8))
		{
			int c;
			while ((c = r.read()) != -1)
			{
				System.out.print((char)c);
			}
		}

does not create the same output as the input. I cannot simplify the example for the moment.

Good find, thanks!

I updated it so that when in this condition (a surrogate pair is straddling the buffer, and the requested len is not enough to read both characters), returns read length of zero. And handle that in the CharacterReader.

This saves having to track the last emitted high/low surrogate etc.

Note that reads of length 1 then may return 0 characters, and that r.read() is not implemented and won't work for surrogates correctly.

If a surrogate is on the buffer boundary, break buffering

@gbrdead

@gbrdead can you validate this?

Yes, it works for all the example data I have.

Navigation Menu

Repository navigation

SimpleStreamReader cannot read multibyte characters on buffer boundary #2353

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

SimpleStreamReader cannot read multibyte characters on buffer boundary #2353

Description

Activity

gbrdead commented on Jun 29, 2025

jhy commented on Jun 30, 2025

jhy commented on Jul 4, 2025

gbrdead commented on Jul 5, 2025

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions