Vinifera - Monitor Internal Leaks on Github
Github Monitoring Tool
We have been using Vinifera in production since Dec 2019 and has helped us prevent security incidents. Vinifera started out as an internal project to ensure Security hygiene of our public contributions and monitor potential leaks on Github. We believe this will help other companies to strengthen their security hygiene when it comes to public sources like Github.
What is Vinifera?
Vinifera allows Companies/Organizations to monitor public assets to find references to internal code leaks and potential breaches.
Sometimes developers might leak internal code and credentials by accident. Vinifera aims to help companies detect those breaches in due time and respond to the incident.
How does it work?
Vinifera monitors developers belonging to the organization, monitors and scans public contributions to look for potential violations and breach of internal/secret/proprietary code by looking for references defined.
- Vinifera works by syncing org users.
- For each user, all the public assets are registered (if not already tracked).
- Each asset (Repo, Gist) is then scanned for any differences.
- Each difference is then scanned, stored, and reported (if contains any leaks)
What does the name mean?
During the development and inception of the tool, the Security team consumed lots of grapes, so we named it after the fruit we love :)
Vinifera is inspired from the Bionomial name of Grapes
Setup and Usage.
Vinifera requires the installation of the following tools:
- Ruby (Install via rbenv/rvm )
Github Access Token
To scan your organization members, Vinifera requires a token with the ability to read Organization members.
Generate a new token https://github.com/settings/tokens/new with no special scope
You would want to use the token of an admin user (with no special scope), since the admin can list all users of an organization.
If the authenticated user is also a member of this organization then both concealed and public members will be returned.
.docker_envwith needed variables
GITHUB_ACCESS_TOKEN=<REDACTED> VINIFERA_ORG_NAME=<Your_org_name> RAILS_MAX_THREADS=60 # This also controls DB pool RAILS_MASTER_KEY=<REDACTED> SLACK_UPDATES_GROUP_URL=https://hooks.slack.com/services/<YOUR_CONFIG_HERE> SLACK_TARGETS_GROUP_URL=https://hooks.slack.com/services/<YOUR_CONFIG_HERE> SLACK_USER_TRACKING_GROUP_URL=https://hooks.slack.com/services/<YOUR_CONFIG_HERE> SLACK_VINIFERA_VIOLATION_GROUP_URL=https://hooks.slack.com/services/<YOUR_CONFIG_HERE> SLACK_ERROR_GROUP_URL=https://hooks.slack.com/services/<YOUR_CONFIG_HERE> # By default fork and big fork scanning is disabled VINIFERA_ENABLE_FORK_SCANNING=false VINIFERA_ENABLE_BIG_FORK_SCANNING=false
- Re-building after any changes
docker-compose up --build
- Install required dependencies
cd <location_of_cloned_repo> bundle install
- Setup DB and migrations
bundle exec rails db:create bundle exec rails db:migrate
- Setup Environment Variables and Slack WebHook Sample env file is available at
GITHUB_ACCESS_TOKEN=<REDACTED> VINIFERA_ORG_NAME=<Your_org_name> # Name of org for which token was generated. VINIFERA_DATABASE_HOST=<localhost> VINIFERA_DATABASE_PASSWORD=<YOUR_PASSWORD> RAILS_MAX_THREADS=60 # This also controls DB pool RAILS_MASTER_KEY=<REDACTED> SLACK_UPDATES_GROUP_URL=https://hooks.slack.com/services/<YOUR_CONFIG_HERE> SLACK_TARGETS_GROUP_URL=https://hooks.slack.com/services/<YOUR_CONFIG_HERE> SLACK_USER_TRACKING_GROUP_URL=https://hooks.slack.com/services/<YOUR_CONFIG_HERE> SLACK_VINIFERA_VIOLATION_GROUP_URL=https://hooks.slack.com/services/<YOUR_CONFIG_HERE> SLACK_ERROR_GROUP_URL=https://hooks.slack.com/services/<YOUR_CONFIG_HERE> # Add this only if you are using Docker over TLS, recommended way, if, on the same host as vinifera, you may skip it DOCKER_CLIENT_CERT_PATH=/home/deployer/.docker DOCKER_HOST=tcp://<IP>:<PORT> # By default fork and big fork scanning is disabled VINIFERA_ENABLE_FORK_SCANNING=false VINIFERA_ENABLE_BIG_FORK_SCANNING=false REDIS_URL=redis://<REDIS_URI> SIDEKIQ_REDIS_URL=redis://<REDIS_URI>
- Setup Cron Jobs
bundle exec whenever --update-crontab
- Setup your custom toml rules for Gitleaks. For more info
[[rules]] # Insert your rules here # description = "Internal References" # regex = '''(?i)((.*)<COMPANY_INTERNAL_REFERENCES>(.*))''' # tags = ["internal", "company","references"]
- Start Sidekiq
bundle exec sidekiq
Additionally, to get the metrics on Datadog like in the above screenshot, you can use the DataDog agent - https://docs.datadoghq.com/agent/
We are open to contributions/bug fixes/performance improvements to our project :)
If you appreciate the tool we have built, feel free to contribute/donate to the projects on the top of which Vinifera was built :)
Vinifera is built on top of other open-source software:
- Rails (Our Favourite Web Framework)
- Sidekiq (Handles Job LifeCycle, Scheduling, and Retries)
- Gitleaks (Gitleaks powers the code scanning via Docker Images)
- Docker (For running Scans in an isolated environment)
- Docker-api (Ruby Client to interact with Docker Remote API)
- Sidekiq Throttled (For Throttling workers)
- Octokit (Ruby Toolkit for Github API)
- Whenever (Cron Jobs in Ruby)
.... (list will go on ..., you get the gist ;) )
You can also donate to Feeding India
Neither Zomato nor the developers of this tool are responsible for any damage caused by this tool or usage of the same. Use responsibly. Refer to LICENSE for more details.
Vinifera is licensed under the Apache License, Version 2.0. See LICENSE for the full license text.