Process doesn't exit / hangs at the end on Windows

I tried examples/nllb.rs and it works. but when the translate finish, the process hangs for few minutes, and ctrl + c doesn't exit it.
Is there some cleanup? can we speed it up?

And another question:
Is there a place I can download the needed folder for translate that will work for mac / windows / linux? I use facebook/nllb-200-distilled-600M
I tried to zip the folder created on mac and use it in windows but it didn't worked. I had to use transformers to create it on each platform.
The goal is to have something I can use right away for cross platform desktop app for offline translations.

Thanks for this amazing library!

Update

Looks like it hangs on drop(t) where t is the translator instance

Thank you for reporting this issue. I was able to reproduce it and will investigate what is blocking the termination.

Regarding model files, I am using Hugging Face and have uploaded some models for CTranslate2. You can create an account and upload your model files there.

This code snippet downloads the model files and returns the directory path:

let api = hf_hub::api::sync::Api::new()?;
let repo = api.model("<your account name>/<repo>");

let mut res = None;
for f in repo.info()?.siblings {
    let path = repo.get(&f.rfilename)?;
    if res.is_none() {
        res = path.parent().map(PathBuf::from);
    }
}

// path to the directory that contains the model file
res.ok_or_else(|| anyhow!("no model files are found"))

Thank you for reporting this issue. I was able to reproduce it and will investigate what is blocking the termination.

Let me know if you have any ideas about where it might be. I tried to debug it but couldn't find a good way to do so in Windows. If you find a way, I'd appreciate it if you could share your insights.

Regarding model files, I am using Hugging Face and have uploaded some models for CTranslate2. You can create an account and upload your model files there.

This code snippet downloads the model files and returns the directory path:

Thanks for the code. eventually I created custom downloader since I needed to get progress callbacks

It looks like the join method here blocks forever:

https://github.com/OpenNMT/CTranslate2/blob/master/src/thread_pool.cc#L106

However, I’m not sure why this happens, since the worker threads appear to end correctly.

I've identified the root cause of the issue. As I initially suspected, cxx or the FFI wasn't freeing the model class properly at the endend (It doesn't call the destructor). Typically, I implement Drop for FFI objects using bindgen. While I'm not entirely sure how to do this with cxx, adding the following snippet at the end of the Whisper example resolved the issue:

    unsafe {
        std::ptr::drop_in_place(model.ptr.into_raw());
    }

Make sure to expose the ptr in whisper.rs like:

pub struct Whisper {
    model: OsString,
    pub ptr: UniquePtr<ffi::Whisper>,
}

Thank you for investigating this issue. Your suggestion works for me and resolves the hang, although bypassing the release of UniquePtr might fail to release resources. I’m wondering if your VRAM is being freed correctly.

I opened PR #74 and will merge it if there appear to be no resource leaks.

After checking it again I think that it doesn't release the memory.
As you said the join() of the thread hangs although the worker getting into the last line (and maybe truly return the function and exit)
I suspect that once the worker almost returning, the destructors of the objects in the worker invoked and some of them hanging for some reason.

Thank you for testing. So, it seems we need to solve the thread issue.

Maybe these are related, even though we don’t use dynamic libraries:

With the workaround implemented in #74, VRAM/RAM are not released even though Translator/Generator/Whisper are dropped. However, the RAM is released when the main process is terminated. I think this is still better than having the process blocked forever. So, I’ll merge the PR.

@jkawamoto

Hi @jkawamoto

Any news about that issue?

If I use CUDA, the process will not get blocked for me. But if I use CPU, it still gets blocked.

It’s still a mystery why joining threads gets blocked in some cases.

Don't link gomp on MSVC (openmp-runtime-comp)

For reference, OpenNMT/CTranslate2#2076

@jkawamoto

@jkawamoto Hi! I saw CTranslate2#2076 landed upstream and that you’ve opened #153. If you need any help on Windows, I’m happy to test it on my side.

@timo9378 Thanks for OpenNMT/CTranslate2#2076 — much appreciated.

From the CI runs it looks like the hang on Windows when using ruy has been addressed. Could you try the remove-drop branch on your Windows machine (with ruy enabled) and see whether the translator still hangs on drop? If you can share the result (and any logs if it still hangs) that would help confirm the fix.

Thanks!

@jkawamoto

@jkawamoto Tested the remove-drop branch (7821222, CTranslate2 v4.8.3) on Windows with ruy. The translator no longer hangs on drop, and memory is released.

Setup: Windows 11 Pro (build 26200), i9-13900, MSVC 14.44 (VS 2022 Build Tools), CMake 4.4.0, rustc 1.98.1, +crt-static. Model: NLLB-200-distilled-600M (CTranslate2), Device::CPU, ComputeType::INT8.

The test loads a Translator, translates one sentence, and drops it. It does this 5 times on the main thread, then once more on a spawned thread, and then returns from main. I ran it under a 10-minute timeout to catch a hang at process exit.

features	drop time (6 drops)	RSS loaded → after drop	process exit
`ruy`, `tokenizers`	0.12–0.14 s	~945 MB → 13–17 MB	clean, 61.7 s total
`ruy`, `tokenizers`, `openmp-runtime-comp`	0.12–0.14 s	~945 MB → 13–17 MB	clean, 59.8 s total

The total time is just the 6 load + translate rounds, so there's no stall at exit. Translations were identical every round. For comparison, with the old Drop workaround we measured about 870 MB leaked per load (the whole model) on the same setup.

One FYI from setting this up: depending on the branch as a git dependency failed on Windows. Cargo's libgit2 checkout hit MAX_PATH on CTranslate2/third_party/thrust/libcudacxx/test/.... I worked around it by cloning with core.longpaths=true and using a path dependency. The crates.io package may not have this problem, but anyone pulling it from git on Windows probably will.

Thanks for picking this up! Happy to test the release once it's out.

@timo9378 Thanks a lot for testing the remove-drop branch on Windows and sharing such a detailed report. Your validation is extremely helpful, and it gives us a lot of confidence that the hang on drop is indeed resolved in the Windows + ruy case.

Also, thank you for pointing out the Windows MAX_PATH issue when using a git dependency. That’s a useful catch, and it will be important to document for anyone building from a git checkout on Windows. I’ll make sure to note that in the README so it’s easier to avoid.

Thanks again for taking the time to test this!

Navigation Menu

Repository navigation

Process doesn't exit / hangs at the end on Windows #64

Update

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Process doesn't exit / hangs at the end on Windows #64

Description

Update

Activity

jkawamoto commented on Jun 21, 2024

thewh1teagle commented on Jun 21, 2024

jkawamoto commented on Jun 22, 2024

thewh1teagle commented on Jul 18, 2024

jkawamoto commented on Jul 19, 2024

thewh1teagle commented on Jul 19, 2024

jkawamoto commented on Jul 20, 2024

jkawamoto commented on Jul 22, 2024

thewh1teagle commented on Oct 15, 2024

jkawamoto commented on Oct 15, 2024

jkawamoto commented on Jul 18, 2026

Koimsurai commented on Oct 17, 2026

jkawamoto commented on Oct 17, 2026

Koimsurai commented on Oct 18, 2026

jkawamoto commented on Oct 20, 2026

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions