Stabilizing importDurations (previously printImportBreakdown)

[sheremet-va]

experimental.importDurations

• Introduced in 4.0.15
• Docs:
  • Config option
  • Guide
• Related
  • Enhance importBreakdown support #9141
  • feat(experimental): add print and fail thresholds for importDurations #9533
• Rational:
  • We want to provide an easy way to analyze the performance of your tests. Specifically the "import" time (previously reported as "collected"). It is the time that took Vitest to import the test file, so it includes all the static imports.
  • Import breakdown shows a list of modules that took the longest to load, but it also provides a nice visualization in Vitest UI and the VSCode extension by showing a number next to the import (this is enabled by default even without this feature).
  • Our hope is that this information can help you (or your AI agent) debug the performance of your tests and refactor if necessary.

[mdubourg001]

Hey! First I want to say that this is a great idea to help debugging test performance issues. I myself tried to get such information for my project's test suite using a custom reporter and the importDurations object from the module given as parameter of the onTestModuleCollected hook. See the Gist here.

I computed average and total import times for each module in a rather simplistic way, and so I'm sure using the built-in printImportBreakdown will be much more precise, but I'd like to suggest to allow passing some configuration instead of just a boolean, for example:

export default defineConfig({
  // ...
  experimental: {
    printImportBreakdown: {
      top: 50, // show the Top 50 imports instead of the default 10
      onlyExternal: true // show only imports from `node_modules`
      // and maybe more ?
    }
  }
  // ...
})

[sheremet-va]

There are several improvements planned with the configuration: #9141

You can leave your ideas there too

[sheremet-va]

This is now stable in Vitest 5.

[uthbees]

Not sure if the site just needs to be updated since this discussion is closed, but it still links here for feedback, so: This was extremely helpful, thank you! Between this and some information from the performance guides, I was able to more than quadruple our testing speed (the problem was barrel files).

I mostly just wanted to leave some positive feedback here since it's a little empty, but I'll also mention that I'm not entirely clear on what "self" and "total" mean. The docs say:

Self: the time it took to import the module, excluding static imports;
Total: the time it took to import the module, including static imports. Note that this does not include transform time of the current module.

Going off of that, I would think "self" would be the time to read the module file and "total" would be the time to read both the module file and all the module files it imports recursively. But I'm not sure this is right because the self time seems to vary a lot more than I would expect, up to more than a second in some cases (if I remember correctly). Maybe it just has to do with file caching or some other magic happening in the background, but regardless, it might be nice for the docs to be a little more clear on what these numbers mean. (I don't really need it myself now, but if I'm unclear about something, there are probably others who are as well.)