Repository navigation
Choosing the background set #391
Description
Hi,
I am using kernel SHAP to explain deep neural network.
My data is all related to the same field but it is very sparse since it contains records from different subfields (for example if it was a medical dataset then I have head related problems, lungs related problems etc. it is sparse since you have different body parts, different treatments...)
If I want to explain a prediction of one record, then I believe that a random background and even k-means background don't represent the neighborhood of the record I want to explain.
Have you tried to choose the background set in another way? Do you think it can be a good idea, for example, to choose the background set using records from the same subfield?
Thanks a lot,
Liat
Activity
slundberg commentedon Jan 19, 2019
You can do that, and DeepExplainer even has a way to generate dynamic background samples. The thing to keep in mind is that if you use a background from the same field you will remove any credit for information that is specific to that field.
For example if the "lung" field of samples always has a feature
is_lung=1and that feature is used by the model then you would see that feature as important if you used a global background, but not if you used a background where all the samples are from the "lung" field.Liatant1 commentedon Jan 23, 2019
Thank you for your answer!
I have another question - I have a sparse dataset with 1000 features. How can I know how many subsamples I should choose?
From my experiments it looks like choosing the background set in a different way (like I asked you) has much more effect than running kernel SHAP with more samples. Before choosing the background set in a different way, I almost didn't get any shap values.
Do you have any advise about the number of subsamples?
I use kernel SHAP because I want it to be model agnostic.
slundberg commentedon Feb 5, 2019
@Liatant1 That's a tricky question that probably depends on the dataset. Choosing different background sets represents how you define "missing" (because it determines what values you plug in to approximate a feature being missing during the toggling process). Using just a single reference is nice because then you know that "missing" means being that reference value. But using many references gets closer to actually integrating out that feature. I would just keep adding more and more random background references, and then see how things change. Then compare that to summaries using things like k-means or medians.
github-actions commentedon Dec 12, 2023
This issue has been inactive for two years, so it's been automatically marked as 'stale'.
We value your input! If this issue is still relevant, please leave a comment below. This will remove the 'stale' label and keep it open.
If there's no activity in the next 90 days the issue will be closed.
github-actions commentedon Mar 12, 2024
This issue has been automatically closed due to lack of recent activity.
Your input is important to us! Please feel free to open a new issue if the problem persists or becomes relevant again.