GitHub replaces stargazer scraping with privacy-safe history API

Paxful
Blockonomics


News


Ledger

GitHub launched a star-history endpoint that returns aggregate weekly and daily counts, cutting requests and reducing database load.



GitHub replaces stargazer scraping with privacy-safe history API

GitHub said it has replaced a stargazer-based workaround with a purpose-built history endpoint that returns aggregate star trends without exposing individual identities. The change matters because maintainers use star history to gauge whether a release, launch or event changed a project’s trajectory, while GitHub said the older approach was being abused for data collection and was also expensive to serve.

What GitHub changed

In an Oct. 14, 2026 engineering post, GitHub said the new repository star history API can return up to 30 weeks of daily star counts in a single request, with no identities attached. GitHub said the endpoint filters stars by time before counting them, rather than counting first and trimming the response later, which it said reduces database work.

The company said the previous GET /repos/{owner}/{repo}/stargazers endpoint was designed to list current stargazers, not to act as a historical event log. Because stars are removed when users unstar a repository, the model does not preserve a complete history of past starring and unstarring activity. GitHub said tools had been reading pages of stargazer records and discarding the usernames to reconstruct a curve from timestamps alone.

Why the old approach became a problem

GitHub said the original endpoint created two issues. First, identities were available for bulk collection, which it said led bad actors to scrape public stargazer lists for spam and phishing campaigns. Second, highly starred repositories could force the database to process many thousands of entries to return a single page.

For react/react, which GitHub said had about 251,000 stars, reconstructing the full curve took about 2,511 requests under the old approach. GitHub also said a pagination bug meant every request counted all stars to determine the number of pages, even when only up to 100 people were returned. In the week before access was restricted, GitHub said nearly 37,000 slow count queries were logged, averaging nearly 3 seconds each, and database safeguards killed more than 1,000 of them. After access was restricted, GitHub said stargazer requests fell by about 60% and total time spent on the count query dropped by about 80%.

How the new endpoint works

GitHub said it evaluated several designs before settling on a time-range approach. One option would have returned stargazer timestamps without identities, but GitHub said that would still have required API consumers to page through large numbers of individual records. Another option would have aggregated stars first and paginated later, but GitHub said a standard SQL plan would still have scanned and grouped all matching rows before applying LIMIT and OFFSET.

The shipped design instead maps page and per_page to an indexed created_at range, filters stars to that range first and then groups the results into daily counts. GitHub said that means one request can cover up to 30 weeks of history, and increasing page moves backward through a repository’s history. In the react/react example, GitHub said the new approach requires about 24 requests instead of about 2,511, while examining each star about once across the full history.

Limits and remaining trade-offs

GitHub said the new endpoint is safer and more predictable, but it is not free of constraints. Calendar boundaries affect how history is sliced, removed stars are not preserved in the model, and unusually dense weeks can still produce slow or expensive queries. GitHub also said dense weeks remain a scaling limitation even though the normal request path no longer queries every current star.

The company said its review of the star-history problem also led to a separate fix on GitHub.com, where the repository stargazers page was updated to use cached star counts rather than counting stars on demand. GitHub said that change eliminated hundreds of slow queries each week from that page.



Source link

Bybit

Be the first to comment

Leave a Reply

Your email address will not be published.


*