Data source for opensource frontend. This repo scrape data hourly and publish it as JSON files.
Data sources are available at: developersdo.github.io/opensource-data.
You will need to:
- Clone this repo.
- Install dependencies:
npm i - Run migrations:
npm run migrations:run - Copy
.env.defaultto.env. - Create a personal GitHub access token (with no scopes) and add the value to
OSD_GH_TOKENin your.env.- Note: You can skip that if you don't need to run the scraper locally.
| Script | Description |
|---|---|
npm run migrations:run |
Run all migrations. |
npm run migrations:add [name] |
Add a new migration. |
npm run scrape |
Scrape all data. |
npm run scrape -- --only=users |
Scrape users data only. |
npm run scrape -- --only=repos |
Scrape repos data only. |
npm run generate |
Generate public site and JSON files. |
Prepend any commmand with DEBUG=osd:*. For example:
DEBUG=osd:* npm run scrape