🐛 biome breaks characters in stdin mode

Environment information

CLI:
  Version:                      1.7.3
  Color support:                true

Platform:
  CPU Architecture:             aarch64
  OS:                           macos

Environment:
  BIOME_LOG_DIR:                unset
  NO_COLOR:                     unset
  TERM:                         "screen-256color"
  JS_RUNTIME_VERSION:           "v20.12.2"
  JS_RUNTIME_NAME:              "node"
  NODE_PACKAGE_MANAGER:         "yarn/4.1.0"

Biome Configuration:
  Status:                       Loaded successfully
  Formatter disabled:           false
  Linter disabled:              false
  Organize imports disabled:    false
  VCS disabled:                 false

Workspace:
  Open Documents:               0

What happened?

When using the stdin mode in combination with “--colors off", some characters are displayed with broken characters.

I'm not a rustacean but have investigated the code.

The broken characters are Combining character which is represented by multiple code points.

fn main() {
    println!("{}", "⚠️".escape_unicode()); // \u{26a0}\u{fe0f}
    println!("{}", "â".escape_unicode()); // \u{61}\u{302}
    println!("{}", "ｶﾞ".escape_unicode()); // \u{ff76}\u{ff9e}
    println!("{}", "👨🏻‍🦱".escape_unicode()); // \u{1f468}\u{1f3fb}\u{200d}\u{1f9b1}
}

--colors off or not using terminal will result in ColorChoice::Never.

biome/crates/biome_console/src/lib.rs

Lines 94 to 114 in fec262f

    
           fn compute_color(colors: ColorMode) -> (ColorChoice, ColorChoice) { 
        
               match colors { 
        
                   ColorMode::Enabled => (ColorChoice::Always, ColorChoice::Always), 
        
                   ColorMode::Disabled => (ColorChoice::Never, ColorChoice::Never), 
        
                   ColorMode::Auto => { 
        
                       let stdout = if io::stdout().is_terminal() { 
        
                           ColorChoice::Auto 
        
                       } else { 
        
                           ColorChoice::Never 
        
                       }; 
        
                       let stderr = if io::stderr().is_terminal() { 
        
                           ColorChoice::Auto 
        
                       } else { 
        
                           ColorChoice::Never 
        
                       }; 
        
                       (stdout, stderr) 
        
                   } 
        
               } 
        
           }

If you run biome format in stdin mode, the entire code is output here.

biome/crates/biome_cli/src/execute/std_in.rs

Lines 66 to 68 in fec262f

    
           console.append(markup! { 
        
               {output} 
        
           });

Finally, this write_str is called.

biome/crates/biome_console/src/write/termcolor.rs

Line 146 in fec262f

fn write_str(&mut self, content: &str) -> fmt::Result {

In the write_str method, convert characters in the case of Windows or ColorChoice::Never.
By grapheme.chars().nth(0), all combining characters output only the first code point.

biome/crates/biome_console/src/write/termcolor.rs

Lines 167 to 173 in fec262f

    
           if cfg!(windows) || !self.writer.supports_color() { 
        
               let is_ascii = grapheme_is_ascii(grapheme); 
        
               if !is_ascii { 
        
                   let replacement = unicode_to_ascii(grapheme.chars().nth(0).unwrap()); 
        
                   replacement.encode_utf8(&mut buffer);

codesandbox

https://codesandbox.io/p/devbox/serene-swirles-6z4jfd?embed=1&file=%2Fsrc%2Findex.js%3A5%2C1&workspaceId=6833e942-7d98-4ed2-8f40-9a265aa14215

related issues

Expected result

The output of the code should not be converted anything.

Code of Conduct

I agree to follow Biome's Code of Conduct

Your investigation will be VERY USEFUL! Thank you!

As a workaround, you can use --colors=force.

@ematipico

@ematipico I usually run biome check --apply --stdin-file-path $FILENAME in vim.
When I add --colors force, the characters are not broken, but the terminal format code are inserted at the beginning and end of the file.

I'm experiencing the terminal code issue too now on neovim/macos

The formatter I use, conform, use these args, and in my options I'm adding --colors=force to this default set

@ematipico

@ematipico we are also running into this issue at Notion, where we use biome in a node process to auto-format and lint --fix template-generated strings of typescript code via child_process.spawn before we write them to file. Our "workaround" so far has been to write the code to temp files and use biome's normal --write mode instead of its stdin approach, but this is slower and also means we have to put the temp file in a similar location to the final source so that the right lint rules are applied, which is more painful to manage.

The core issue seems not to be the color handling choice from termcolor but rather the conflation of colors and emoji in these two parts of the code

biome/crates/biome_console/src/write/termcolor.rs

Lines 167 to 182 in 020c0ed

    
           if cfg!(windows) || !self.writer.supports_color() { 
        
               let is_ascii = grapheme.is_ascii(); 
        
               if !is_ascii { 
        
                   let replacement = unicode_to_ascii(grapheme.chars().nth(0).unwrap()); 
        
                   replacement.encode_utf8(&mut buffer); 
        
                   if let Err(err) = self.writer.write_all(&buffer[..replacement.len_utf8()]) { 
        
                       self.error = Err(err); 
        
                       return Err(fmt::Error); 
        
                   } 
        
                   continue; 
        
               } 
        
           };

biome/crates/biome_console/src/write/termcolor.rs

Lines 204 to 214 in 020c0ed

    
           /// Replace emoji characters with similar but more widely supported ASCII 
        
           /// characters 
        
           fn unicode_to_ascii(c: char) -> char { 
        
               match c { 
        
                   '\u{2714}' => '\u{221a}', 
        
                   '\u{2139}' => 'i', 
        
                   '\u{26a0}' => '!', 
        
                   '\u{2716}' => '\u{00d7}', 
        
                   _ => c, 
        
               } 
        
           }

Colors are generally terminal-only escape characters and will not appear in a source-code file, so they're fine to retain or strip based on a terminal's color support.

Emoji though have semantic meaning and are real parts of the source - by mutating emoji biome is changing the original source in a way that breaks the intention of the code's author. Whether or not a terminal will correctly display the emoji codepoints isn't really relevant, since what we care about is staying true to the content of the source file, not making the output easier to sight-read. It also means that the end-result output is different between the file-handling side of biome and the stdin side of biome, which leads to some confusion.

Just to get a sense of y'all's thinking on this, how would you feel about a PR that removes the emoji-mutating code from the stdout processor? Or that adds some option to retain emoji codepoints while still making other color decisions per termcolor?

@walkerdb

Thank you, @walkerdb, for chiming in! Please send a PR; everyone would appreciate it. As for the solution, I don't have strong opinions (or knowledge).

Considering the use case, I believe it's mostly likely to have characters that would break than emojis that would break, so maybe an option to keep emoji codepoints should be fine? Again, I don't have much knowledge, so feel free to take the initiative and go for a solution that suits your needs

I didn't know that biome's LSP supports code formatting, if you want to use Biome in NeoVim, you can just use the formatter from the LSP, you don't need something like Conform that uses stdin to format the file. If you are using Conform, you can use the lsp_format = "prefer" or lsp_format = "first" for the languages that are using Biome.
I couldn't get the organize-imports to work with LSP though.

Can confirm, diacritics are gone too:

echo "användarvillkoren" | tee /tmp/test.txt |  npx @biomejs/biome format --colors off --stdin-file-path=/tmp/test.txt

I have the same issue. Any workarounds?

I vibe-coded a possible solution. I might create a preview release so you can test it #8410

This issue should be reopened, the PR does not solve the whole problem, characters like ⚠ are still incorrectly replaced when they appear in the source code.
Biome should only tweak its own output depending on the terminal, source code should appear as-is unless there is a lint or formatting rule to change it.

PRs are welcome then. I tried with my limited knowledge but it didn't work

I also still have the same issue with biome 2.3.10

biome/crates/biome_console/src/write/termcolor.rs

Lines 182 to 200 in 922b04f

    
           } else if !self.writer.supports_color() { 
        
               // On non-Windows with colors disabled: 
        
               // Only convert single-codepoint graphemes (diagnostic symbols) 
        
               // Multi-codepoint graphemes (like emoji with modifiers) are preserved for source code fidelity 
        
               let chars: Vec<char> = grapheme.chars().collect(); 
        
               if chars.len() == 1 { 
        
                   let replacement = unicode_to_ascii(chars[0]); 
        
                   replacement.encode_utf8(&mut buffer); 
        
                   if let Err(err) = self.writer.write_all(&buffer[..replacement.len_utf8()]) { 
        
                       self.error = Err(err); 
        
                       return Err(fmt::Error); 
        
                   } 
        
                   continue; 
        
               } 
        
               // Multi-codepoint graphemes fall through to be written as-is below 
        
           }

This is clearly the problem. In the PR #8410, you modified certain emojis because they can be problematic in some terminals. The solution is that don't do it just because color is disabled, or rather, do it on actual diagnostic symbols and not the source code if they can be separated.

I would just remove that block and it should be good. (at least for non-Windows)

Closing. The issues has been fixed in one of latest versions of Biome.

Navigation Menu

Repository navigation

🐛 biome breaks characters in stdin mode #2988

Environment information

What happened?

codesandbox

related issues

Expected result

Code of Conduct

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

	fn compute_color(colors: ColorMode) -> (ColorChoice, ColorChoice) {
	match colors {
	ColorMode::Enabled => (ColorChoice::Always, ColorChoice::Always),
	ColorMode::Disabled => (ColorChoice::Never, ColorChoice::Never),
	ColorMode::Auto => {
	let stdout = if io::stdout().is_terminal() {
	ColorChoice::Auto
	} else {
	ColorChoice::Never
	};

	let stderr = if io::stderr().is_terminal() {
	ColorChoice::Auto
	} else {
	ColorChoice::Never
	};

	(stdout, stderr)
	}
	}
	}

	if cfg!(windows) \|\| !self.writer.supports_color() {
	let is_ascii = grapheme_is_ascii(grapheme);

	if !is_ascii {
	let replacement = unicode_to_ascii(grapheme.chars().nth(0).unwrap());

	replacement.encode_utf8(&mut buffer);

	if cfg!(windows) \|\| !self.writer.supports_color() {
	let is_ascii = grapheme.is_ascii();

	if !is_ascii {
	let replacement = unicode_to_ascii(grapheme.chars().nth(0).unwrap());

	replacement.encode_utf8(&mut buffer);

	if let Err(err) = self.writer.write_all(&buffer[..replacement.len_utf8()]) {
	self.error = Err(err);
	return Err(fmt::Error);
	}

	continue;
	}
	};

	/// Replace emoji characters with similar but more widely supported ASCII
	/// characters
	fn unicode_to_ascii(c: char) -> char {
	match c {
	'\u{2714}' => '\u{221a}',
	'\u{2139}' => 'i',
	'\u{26a0}' => '!',
	'\u{2716}' => '\u{00d7}',
	_ => c,
	}
	}

	} else if !self.writer.supports_color() {
	// On non-Windows with colors disabled:
	// Only convert single-codepoint graphemes (diagnostic symbols)
	// Multi-codepoint graphemes (like emoji with modifiers) are preserved for source code fidelity
	let chars: Vec<char> = grapheme.chars().collect();
	if chars.len() == 1 {
	let replacement = unicode_to_ascii(chars[0]);

	replacement.encode_utf8(&mut buffer);

	if let Err(err) = self.writer.write_all(&buffer[..replacement.len_utf8()]) {
	self.error = Err(err);
	return Err(fmt::Error);
	}

	continue;
	}
	// Multi-codepoint graphemes fall through to be written as-is below
	}

🐛 biome breaks characters in stdin mode #2988

Description

Environment information

What happened?

codesandbox

related issues

Expected result

Code of Conduct

Activity

ematipico commented on May 26, 2024

ton1517 commented on May 26, 2024

melMass commented on Nov 18, 2024

walkerdb commented on Apr 24, 2025

ematipico commented on Apr 25, 2025

ahmafi commented on Sep 15, 2025

lagomcoder commented on Oct 22, 2025

kiyoon commented on Dec 9, 2025

ematipico commented on Dec 9, 2025

Pajn commented on Dec 17, 2025

ematipico commented on Dec 17, 2025

kiyoon commented on Dec 18, 2025

kiyoon commented on Dec 18, 2025

ematipico commented on Mar 21, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions