Feedback on GitHub Pull Request analytics

[mmitrik]

🤔 What types of analytics do you need for your pull requests?

Let us know ideas here!

[elsigh]

I love the leaderboard and being able to slice/dice in PullPanda - this is going to hurt!

[toshimaru]

I'd like to know PR merge time to measure our delivery speed. 🚀

[dimofeev]

I'd like to have overview of all open PRs across my organisation.

[stefanoric]

I use this a lot as technical lead to see what PRs are stuck in review state, when they were opened, etc... It's one my most visited web pages, use it almost every day.

[timjbray]

We use the leaderboard, and find the data invaluable: reviewer & PR's reviewed, turnaround and PR size.

[frenck]

It's the generic overview across the organization it brings, that nothing in GitHub currently can do. Shutting down this tool will have a large impact on how we manage Home Assistant at this point.

So, while it's put out as: "Let us know which analytics you use.", is falling short of what pullpanda currently can do/does (but agreed, it is mainly analytics).

Some key functionality we miss; and will harm our project when pullpanda is taken offline before a proper replacement is provided:

• Being able to monitor organization-wide PR throughput. This is a quick and fast insight into how are review queues are currently holding up. It's a key factor in steering focus for an open-source project. Our queues are already large, but they should definitely not grow.
  

• Leaderboards are nice too. The top ones are the usuals and don't matter that much, but it helps to find code reviewers in the org that are more active than usual (or the ones that are less active than usual). This can help with reaching out, or, motivating people that are potential/likely to become more active reviewers. Again, this is stats across the organization as well.

• The top authors are a nice view too, while it is full of usual suspects, it provides a cross-repository/organization view on active contributors and helps find new members for the open-source project.

• The PRs tab. And this IMHO isn't analytics. The PR tabs give insight into all PRs in an org, and allow for quickly filtering it by a team, repository, and sorting them by stallest/oldest and similar. This can be done with GitHub Search, but GitHub search isn't really providing an overview.

  The difference in the PRs overview pullreminder provides, is that is shows viable information for these purposes at a glance. How stale? How old? How many approvals already? How large is the PR? (Since PR size and staleness tend to be related, those two are helpful).

  

There is more, but I think the above is the main driver for the Home Assistant project at this point and is a functionality GitHub cannot provide. The biggest issue GitHub has to solve IMHO: Is organization overview, that is the gap pullpanda is closing right now.

We are sad to see pullpanda go, and would love to see it being kept online until alternatives are provided. 🙏

../Frenck

[GuillemRoca]

All the analytics provided inside of Pull Panda are nice and insightful, but for me, the analytics and leaderboards are an invaluable source of information to write down the impact and reflect on work done in performance reviews. At a glance, you can see your and your team's performance at a quarterly level and create actionable items to improve.

For this reason, as has been said previously, I would love to see PullPanda available until there is a new tool available with at least the same functionality. I hope the current decision can be reversed.

[ppurcell]

Many of Pull Panda's features have been re-created across the platform in piecemeal form. I think one of the original application's strengths is the ease in which one could see a high level view of many important operational efficiency metrics in a very easy and cohesive manner.

• Leaderboard
  Use Case: Who is contributing(or under contributing) and potentially needs recognition or follow-up. Sliced by Team, Repository, or time scale.
  Potential Improvement: Move more metrics inside this view to provide a more comprehensive picture on contribution. (Avg PR size, PR cadence, PRs cut, ect.)

• Operational Metrics/Graphs (Avg Age, Thoroughput, Success Rate)
  Use Case: Can I identify any long term-trends or identify impact of organizational events that need addressing?
  Potential Improvement: More comprehensive tooling to slice and generate trackable metrics/reporting.

• PR list sliced by group and/or staleness.
  Use Case: How do I quickly check tickets through different slices that are relevant to me?
  Potential Improvement: Github has some of this functionality in house, PullPanda's dropdowns help you select your slice easier than hand building a filter and PullPanda (while opinionated) displays more information(Number of +1s, Days Stale, Size) for analysis and responding to PRs based on need or size. ..

It's been a disappointment that this feature has been "closed" to new signups during this long integration period. It feels too soon to close it down with much of the functionality seemingly missing from the main application.

[mkql]

I would like to see review turn-around time from the reviewee's point of view. (I know that turn-around time from the reviewer's point of view is currently available, which is great feature.)

The point is that it would provide a metric for evaluating how PRs are easy to review. I think PR size can give similar information, but some PRs contains auto-generated codes, which make the meaning of the metric unclear.

[caroleolivier]

Having the insights below is useful to understand team's ways of working (inspired by us using pullreminders)
It'd be brilliant to have these integrated in GitHub 🙏

Time to first review
Why? To see whether we could be blocking each others with long wait times.

Merge Time
Why? Because long merge time could be an indication that our work is not well defined enough or PRs are raised too early or we have a code quality problem when PRs are created.

PR size
Why? To make sure we're creating reasonably sized PR, i.e. easy and quick to review.

Number merged PRs
Why? A variation in amount of PRs we raise could be an indicator of something happening in the team (working on too much non dev work for instance)

For these ☝️ a view per GitHub team would be great.
My team contributes and owns quite a few repos so being able to filter per team rather than repo is very handy.

Insight on Individuals
Very similar to Pullreminder leaderboard.
Why? Because it helps understand our own behaviours (e.g. heavy commenter, rapid approver).

Thanks!

[jordanholliday]

We're making prq.app as a drop-in replacement for Pull Reminders' functionality.

Currently it only has the PR queue for open pulls, but an analytics page at parity with what Pull Reminders / Pull Panda once offered is in the works and should be released soon. We were originally targeting June for release, but now that Pull Reminders is completely shutdown, we'll have to push that up.

My company and I were very heavy users of Pull Reminders, until last week. While I truly appreciate the MS Github team soliciting feedback in this thread, realistically we need a replacement now and cannot wait for MS Github to reimplement the feature set at some unknown future date.

If anyone here as feedback / suggestions / questions about the replacement we are working on, I'm very happy to hear them at jordan@prq.app.

[piersmacdonald]

More variety in notifications. For example, we use Slack notifications but many people, myself included, mute the channel because my regular workflow is to go to github and view my requested PRs.

However I'd love for exceptions to be brought to my attention. e.g. when a PR exceeds a time threshold if we could ping #my_eng_team with an overdue notification to remind people to participate that'd be great.

Then building off that overdue, would be great to know who's doing a good job of participating within our desired threshold (e.g. 2 business days)

[styfle]

@mmitrik If you're still looking for feedback, the link on this blog post is broken.

It links to https://github.com/github-community/community/discussions/categories/pull-requests-feedback

But it should probably link to https://github.com/github-community/community/discussions/categories/pull-requests

[thibaudcolas]

Here is a related thread I had shared about pull request analytics, not sure if it got picked up for consideration already or not: #13037.

Long story short, we’re already attempting to identify bottlenecks in our PR review funnel. Lead time for changes in particular would be a big one for us, and being able to measure how long PRs stay opened would help a lot (already discussed further up as PR merge time).

[VinceChow]

I'm currently building CodeShoutout to address this issue. Although it's not ready for production yet, pretty sure it will be beneficial to some of us here.

Some of our daily pain points that CodeShoutout will be addressing:

1. Long review times slowing down the development process.
2. Lack of clear metrics to measure review efficiency.
3. Unidentified bottlenecks causing workflow issues.
4. Difficulties in setting and achieving review goals.

And the benefits for us:

1. Improved efficiency of the review process.
2. Clear metrics to track and improve performance.
3. Identification and elimination of bottlenecks.
4. Empowering teams with targeted review goals.

Here I'm also sharing one of the chart screenshots that CodeShoutout will be offering when it goes live. Hope it helps.

Happy code reviewing!

[alfvali]

"Time to first review" is something I'm very interested in.

[glebignatieff]

Is it still alive?

[jonathanpruvost]

Good morning, Is the project still current? how to join the beta?

[sergiinvestor]

This discussion raises excellent points about organization visibility on PRs, merge times, and reviewer activity – metrics that are essential for managing both team workflows and open-source projects.

In my experience, combining GitHub's native tools with external solutions can provide deeper insights. For example, tools like Gitlights help explore PR analytics in detail, offering visualizations like PR throughput, reviewer rankings, and temporal trends of PR activity.

Here’s an example of how Gitlights tracks comments, reviews and conversations over a given period:

Having these kinds of analytics alongside GitHub's native insights can make it easier to identify bottlenecks, encourage reviewer engagement, and ensure smoother collaboration across repositories.