[a2d8a4v]

The current median filter operator has a CPU version alone. Adding a GPU implementation to satisfy the support for GPU computing.

[a2d8a4v]

The code for testing the GPU implementation is here

void test_median_filter() {

    StorageView input({1, 2, 10}, DataType::FLOAT32);
    auto* in_data = input.data<float>();
    for (size_t i = 0; i < 20; ++i) in_data[i] = float(i + 1);
    std::cout << "Input:\n";
    for (size_t i = 0; i < 20; ++i) {
        std::cout << in_data[i] << ' ';
        if ((i + 1) % 10 == 0) std::cout << '\n';
    }

    StorageView output;
    ops::MedianFilter median(7);
    median(input, output);

    std::cout << "Output:\n";
    auto* out_data = output.data<float>();
    for (size_t i = 0; i < 20; ++i) {
        std::cout << out_data[i] << ' ';
        if ((i + 1) % 10 == 0) std::cout << '\n';
    }

    StorageView input_gpu = input.to(Device::CUDA);
    StorageView output_gpu(input_gpu.shape(), input.dtype(), Device::CUDA);
    output_gpu.zero();
    ops::MedianFilter(7)(input_gpu, output_gpu);
    std::cout << "Output (GPU -> CPU copy):\n";
    StorageView output_gpu_host = output_gpu.to(Device::CPU);
    out_data = output_gpu_host.data<float>();
    for (size_t i = 0; i < 20; ++i) {
        std::cout << out_data[i] << ' ';
        if ((i + 1) % 10 == 0) std::cout << '\n';
    }
    
    std::cout << "Input GPU size: " << input_gpu.size() << '\n';
    std::cout << "Output GPU size: " << output_gpu.size() << '\n';
}

The result is like:

Input:
1 2 3 4 5 6 7 8 9 10
11 12 13 14 15 16 17 18 19 20
Output:
3 3 3 4 5 6 7 8 8 8
13 13 13 14 15 16 17 18 18 18
Output (GPU -> CPU copy):
3 3 3 4 5 6 7 8 8 8
13 13 13 14 15 16 17 18 18 18
Input GPU size: 20
Output GPU size: 20  

I hope this tiny experiment proves that it can work.

[a2d8a4v]

Hi, @minhthuc2502
Can you help me review this pull req?

Sincerely,

[jordimas]

Thanks for your work

Will be possible to update the current MedianFilter then executes also in CUDA and test that the test pass?

[a2d8a4v]

Sure!

[a2d8a4v]

Wow, thank you for helping me
Sorry for my delay...

[a2d8a4v]

I think the implementation in src/models/whisper.cc can be updated as well, cause the GPU implementation is now provided.

[jordimas]

Thanks for your work, it is really good.
In order for Whisper to use in GPU the median filter it will be great to complete this first:

#1954

If you can help it will be great.

[a2d8a4v]

Gotcha, let me see this issue~