Home / Linux / Miscellaneous

benchmark tool for Docker, BuildKit, img, Buildah, and Kaniko

Buildbench: benchmark tool for Docker, BuildKit, img, Buildah, and Kaniko Requirements bash realpath Docker Usage $ ./buildbench ./examples/ex01 out.csv 5 To disable specific builder, plea

Buildbench: benchmark tool for Docker, BuildKit, img, Buildah, and Kaniko

Build Status

Requirements

  • bash
  • realpath
  • bc
  • docker

Usage

$ ./buildbench ./examples/ex01 out.csv 5
  • To disable specific builder, please set DISABLE_DOCKER, DISABLE_BUILDKIT, and so on.
4 Open More issues
Closed

Update to latest versions as of 2019-04-18 and enable Kaniko cache

Updates all the docker images to the latest I could find on Docker Hub. Buildkit is definitely out of date as the only version found was 0.3.0 and we're now up to 0.4.0. Buildah may also be

opened Apr 18, 2019 by samschlegel 7
Closed

Update kaniko with support for cache

help wanted 
opened Nov 18, 2018 by AkihiroSuda 1
Closed

Add makisu

https://github.com/uber/makisu

help wanted 
opened Nov 18, 2018 by AkihiroSuda 1
Closed

Misc improvements

  • use float time
  • support ONLY_${builder} mode
opened Jun 8, 2018 by AkihiroSuda 0
Closed

pin builder images

Signed-off-by: Akihiro Suda [email protected]

opened Jun 19, 2018 by AkihiroSuda 0
Open

Add gnuplot/matplotlib script

enhancement help wanted good first issue 
opened Oct 3, 2018 by AkihiroSuda 0
Open

Benchmark result (June 19, 2018)

Full logs and version info: https://travis-ci.org/AkihiroSuda/buildbench/builds/393967682?utm_source=github_status&utm_medium=notification ( https://github.com/AkihiroSuda/buildbench/commit/80e6341b62965e28a143ee66788918f7e23909eb )

# May, 2018
DOCKER_IMAGE="docker:18.05-dind"
# BuildKit does not have tagged images; git: moby/[email protected] (June 16, 2018)
BUILDKIT_IMAGE="tonistiigi/[email protected]:38d17109b217c2894063f0bd4b844554901de65add1f9d7cc3b5b73da4f6ab4e"
# June 13, 2018
IMG_IMAGE="r.j3ss.co/img:v0.4.6"
# Buildah does not have tagged images; git: projectatomic/[email protected] (June 15, 2018)
BUILDAH_IMAGE="containerbuilding/[email protected]:b03b04c2480521c9ae05c87f28cc2568a347d009de47f4153d30b41f3bf0ccb8"
# May, 2018
KANIKO_IMAGE="gcr.io/kaniko-project/executor:v0.1.0"

Description about workloads: #1

CSV columns: ID, took, begun, finished

ex01.csv

docker-1,18.165849907,1529391592.263071609,1529391610.428921516
docker-2,1.235037047,1529391610.434916377,1529391611.669953424
docker-1,14.818404800,1529391612.367617690,1529391627.186022490
docker-2,1.287132571,1529391627.190408360,1529391628.477540931
docker-1,14.802825026,1529391629.158332039,1529391643.961157065
docker-2,1.228064991,1529391643.965079234,1529391645.193144225
docker-1,15.078891824,1529391645.875215472,1529391660.954107296
docker-2,1.234751728,1529391660.957918448,1529391662.192670176
docker-1,15.178132443,1529391662.856697985,1529391678.034830428
docker-2,1.249452815,1529391678.038773656,1529391679.288226471
buildkit-1,7.861676981,1529391683.423117905,1529391691.284794886
buildkit-2,1.132106418,1529391691.288556542,1529391692.420662960
buildkit-1,7.333715405,1529391693.139716291,1529391700.473431696
buildkit-2,1.153705816,1529391700.477532735,1529391701.631238551
buildkit-1,7.319216463,1529391702.341668182,1529391709.660884645
buildkit-2,1.108104800,1529391709.664805940,1529391710.772910740
buildkit-1,7.638978042,1529391711.533288749,1529391719.172266791
buildkit-2,1.136491177,1529391719.176583838,1529391720.313075015
buildkit-1,9.102700778,1529391721.029759107,1529391730.132459885
buildkit-2,1.183218361,1529391730.136429357,1529391731.319647718
img-1,10.564293780,1529391734.808084287,1529391745.372378067
img-2,1.835848179,1529391745.376254356,1529391747.212102535
img-1,9.769529824,1529391747.431877654,1529391757.201407478
img-2,1.952110230,1529391757.205537429,1529391759.157647659
img-1,9.723758768,1529391759.379980697,1529391769.103739465
img-2,1.817143219,1529391769.107704175,1529391770.924847394
img-1,10.408545686,1529391771.151449561,1529391781.559995247
img-2,1.843239252,1529391781.564312723,1529391783.407551975
img-1,9.514438987,1529391783.633510686,1529391793.147949673
img-2,1.894057609,1529391793.151888472,1529391795.045946081
buildah-1,15.500970782,1529391817.194548251,1529391832.695519033
buildah-2,13.619950535,1529391832.699490829,1529391846.319441364
buildah-1,14.968011827,1529391846.369269058,1529391861.337280885
buildah-2,13.369802949,1529391861.341253218,1529391874.711056167
buildah-1,15.547745947,1529391874.760378952,1529391890.308124899
buildah-2,13.435780378,1529391890.312092187,1529391903.747872565
buildah-1,15.260446171,1529391903.799704630,1529391919.060150801
buildah-2,13.683330997,1529391919.064120785,1529391932.747451782
buildah-1,15.078477477,1529391932.797529318,1529391947.876006795
buildah-2,13.539169131,1529391947.880278848,1529391961.419447979
kaniko-1,14.243428057,1529391963.082480443,1529391977.325908500
kaniko-2,13.409059837,1529391977.330233443,1529391990.739293280
kaniko-1,12.146617865,1529391990.744615692,1529392002.891233557
kaniko-2,12.328629475,1529392002.895359127,1529392015.223988602
kaniko-1,13.466581014,1529392015.228922142,1529392028.695503156
kaniko-2,13.449305711,1529392028.699544314,1529392042.148850025
kaniko-1,13.317310198,1529392042.153942771,1529392055.471252969
kaniko-2,12.848505995,1529392055.475368000,1529392068.323873995
kaniko-1,13.035854787,1529392068.328808640,1529392081.364663427
kaniko-2,13.319246937,1529392081.368680907,1529392094.687927844

ex01.txt

moby.csv

docker-1,353.787756456,1529392007.999389134,1529392361.787145590
docker-2,5.904878890,1529392361.793445013,1529392367.698323903
docker-1,356.152896153,1529392370.380126094,1529392726.533022247
docker-2,5.756538996,1529392726.540641159,1529392732.297180155
docker-1,351.236617027,1529392734.749708553,1529393085.986325580
docker-2,5.694913895,1529393085.990213594,1529393091.685127489
docker-1,346.027762206,1529393094.139444521,1529393440.167206727
docker-2,8.714977267,1529393440.171129941,1529393448.886107208
docker-1,351.950024279,1529393451.371205967,1529393803.321230246
docker-2,5.825282377,1529393803.325100829,1529393809.150383206
buildkit-1,279.197551929,1529392095.673257046,1529392374.870808975
buildkit-2,1.883717697,1529392374.879647411,1529392376.763365108
buildkit-1,279.243992842,1529392379.512312725,1529392658.756305567
buildkit-2,1.844081306,1529392658.760409468,1529392660.604490774
buildkit-1,279.479077391,1529392663.435859295,1529392942.914936686
buildkit-2,1.859363537,1529392942.918906463,1529392944.778270000
buildkit-1,278.989010640,1529392947.566908645,1529393226.555919285
buildkit-2,1.898166498,1529393226.559961109,1529393228.458127607
buildkit-1,277.165037488,1529393231.221658213,1529393508.386695701
buildkit-2,1.872406282,1529393508.390886640,1529393510.263292922
img-1,442.317527554,1529392155.120938513,1529392597.438466067
img-2,7.521936376,1529392597.445835650,1529392604.967772026
img-1,451.095576198,1529392607.323904690,1529393058.419480888
img-2,7.424161858,1529393058.423472851,1529393065.847634709
img-1,447.124890511,1529393068.196446102,1529393515.321336613
img-2,7.677913305,1529393515.325266950,1529393523.003180255
img-1,447.304128181,1529393525.338733986,1529393972.642862167
img-2,7.472302153,1529393972.646785396,1529393980.119087549
img-1,447.636093349,1529393982.401350000,1529394430.037443349
img-2,7.685351348,1529394430.041214689,1529394437.726566037

moby_docker.txt moby_buildkit.txt moby_img.txt

benchmark-result 
opened Jun 19, 2018 by AkihiroSuda 0
Open

Benchmark result (June 9, 2018)

Full logs and version info: https://travis-ci.org/AkihiroSuda/buildbench/builds/389792477?utm_source=github_status&utm_medium=notification

Description about workloads: #1

CSV columns: ID, took, begun, finished

ex01.csv

docker-1,15.088948494,1528516883.067540964,1528516898.156489458
docker-2,1.264920067,1528516898.162242410,1528516899.427162477
docker-1,14.851893349,1528516900.139559065,1528516914.991452414
docker-2,1.296145993,1528516914.995536683,1528516916.291682676
docker-1,14.634571302,1528516916.982249664,1528516931.616820966
docker-2,1.335567962,1528516931.620468566,1528516932.956036528
docker-1,14.938568557,1528516933.646126876,1528516948.584695433
docker-2,1.339203861,1528516948.588702272,1528516949.927906133
docker-1,15.008774077,1528516950.595051276,1528516965.603825353
docker-2,1.287587080,1528516965.608013027,1528516966.895600107
buildkit-1,7.676590955,1528516970.952694997,1528516978.629285952
buildkit-2,1.220504264,1528516978.633374484,1528516979.853878748
buildkit-1,7.520999153,1528516980.576517629,1528516988.097516782
buildkit-2,1.124634502,1528516988.101464458,1528516989.226098960
buildkit-1,7.678619715,1528516989.962819048,1528516997.641438763
buildkit-2,1.172230003,1528516997.645713939,1528516998.817943942
buildkit-1,8.343999825,1528516999.540903954,1528517007.884903779
buildkit-2,1.136848518,1528517007.892358192,1528517009.029206710
buildkit-1,7.540258448,1528517009.767991265,1528517017.308249713
buildkit-2,1.133150257,1528517017.312111999,1528517018.445262256
img-1,9.916585799,1528517021.603791267,1528517031.520377066
img-2,1.895819467,1528517031.524145433,1528517033.419964900
img-1,9.896674246,1528517033.647922903,1528517043.544597149
img-2,1.972287355,1528517043.548718848,1528517045.521006203
img-1,9.927664649,1528517045.753273162,1528517055.680937811
img-2,1.830990818,1528517055.685087238,1528517057.516078056
img-1,10.094313643,1528517057.753391160,1528517067.847704803
img-2,1.972982689,1528517067.851263908,1528517069.824246597
img-1,10.522893321,1528517070.050599802,1528517080.573493123
img-2,1.912381080,1528517080.579451096,1528517082.491832176
buildah-1,13.357741251,1528517102.141753277,1528517115.499494528
buildah-2,12.167797793,1528517115.503435666,1528517127.671233459
buildah-1,12.777023598,1528517127.720550188,1528517140.497573786
buildah-2,10.884054123,1528517140.501773347,1528517151.385827470
buildah-1,14.075376815,1528517151.443113080,1528517165.518489895
buildah-2,11.159878164,1528517165.522856817,1528517176.682734981
buildah-1,12.498024832,1528517176.736068416,1528517189.234093248
buildah-2,11.252820026,1528517189.238772689,1528517200.491592715
buildah-1,13.086742903,1528517200.543874630,1528517213.630617533
buildah-2,11.105947072,1528517213.634928288,1528517224.740875360
kaniko-1,13.949581683,1528517226.325083879,1528517240.274665562
kaniko-2,13.708101622,1528517240.279425567,1528517253.987527189
kaniko-1,12.888729452,1528517253.992463459,1528517266.881192911
kaniko-2,14.101354493,1528517266.885450815,1528517280.986805308
kaniko-1,13.230798628,1528517280.993682640,1528517294.224481268
kaniko-2,13.271245080,1528517294.228877218,1528517307.500122298
kaniko-1,15.104304591,1528517307.512061163,1528517322.616365754
kaniko-2,13.208975221,1528517322.624875939,1528517335.833851160
kaniko-1,13.893086794,1528517335.839334127,1528517349.732420921
kaniko-2,12.690004360,1528517349.739682189,1528517362.429686549

ex01.txt

moby.csv

docker-1,368.328064583,1528516894.627995325,1528517262.956059908
docker-2,5.913627327,1528517262.963389876,1528517268.877017203
docker-1,346.701621722,1528517271.244334087,1528517617.945955809
docker-2,5.887373814,1528517617.950057810,1528517623.837431624
docker-1,354.015341011,1528517626.326424942,1528517980.341765953
docker-2,5.862704967,1528517980.345517145,1528517986.208222112
buildkit-1,281.376391784,1528516969.148981670,1528517250.525373454
buildkit-2,1.772235548,1528517250.531774709,1528517252.304010257
buildkit-1,281.301812854,1528517255.114552131,1528517536.416364985
buildkit-2,1.822073199,1528517536.420459090,1528517538.242532289
buildkit-1,280.564748838,1528517540.955174297,1528517821.519923135
buildkit-2,1.852769957,1528517821.524235519,1528517823.377005476
img-1,430.453649374,1528516883.837080020,1528517314.290729394
img-2,7.088786477,1528517314.303254456,1528517321.392040933
img-1,431.462889460,1528517323.572703808,1528517755.035593268
img-2,7.318575166,1528517755.039628785,1528517762.358203951
img-1,433.546605852,1528517764.603786101,1528518198.150391953
img-2,7.159197898,1528518198.154567821,1528518205.313765719

moby-docker.txt moby-buildkit.txt moby-img.txt

benchmark-result 
opened Jun 9, 2018 by AkihiroSuda 0
Open

Benchmark result (June 6, 2018)

Travis log available: https://travis-ci.org/AkihiroSuda/buildbench/builds/388656138

Buildbench version

https://github.com/AkihiroSuda/buildbench/commit/70c88896bacbd23752195e715339e9d2c48b04d0

Env

TravisCI Ubuntu Trusty ( https://docs.travis-ci.com/user/reference/overview/ )

  • Ext4
  • 7.5GB RAM
  • 2 Cores, bursted

Tested tools

Benchmark: examples/ex01

  • foo-1: clean environment
  • foo-2: keep foo-1 cache storage, touch buildContext/build-bench-dummy, and then build the image again
  • Tested 5 times

ID | Took (seconds) --------|------------- docker-1 | 20,15,18,16,17 (avg: 17.2) docker-2 | 1,2,2,1,1 (avg: 1.4) buildkit-1 | 12, 13, 8, 9, 8 (avg: 10.0) buildkit-2 | 2, 1, 1, 1, 1 (avg: 1.2) img-1 | 11,11,10,13,14 (avg: 11.8) img-2 | 2,2,2,1,2 (avg: 1.8) buildah-1 | 15, 15, 13, 21, 14 (avg: 15.6) buildah-2 | 11, 11, 15, 14, 17 (avg: 13.6) kaniko-1 | 21, 18, 16, 16, 15 (avg: 17.2) kaniko-2 | 15, 18, 18 26, 16 (avg: 18.6)

image

20180606-ex01.txt

Benchmark: moby/[email protected]

  • The image could not be built with Buildah and Kaniko, due to lack of support for FROM anotherstage (EDIT: fixed typo s/anotherimage/anotherstage/)
  • Tested single time only

ID | Took (seconds) --------|------------- docker-1 | 387 docker-2 | 8 buildkit-1 | 266 buildkit-2 | 2 img-1 | 450 img-2 | 6

image

20180606-moby.txt

benchmark-result 
opened Jun 6, 2018 by AkihiroSuda 1
Information
Category: Linux / Miscellaneous
Watchers: 5
Star: 71
Fork: 11
Last update: Oct 13, 2021

Contents
    Resource links
    https://github.com/AkihiroSuda/buildbench
    Share this repo
    Latest Repos
    garutilorenzo Full stack (LEMP) Wordpress docker environment for dev and production use
    Weeb-Linux quickly deploy archlinux on your android (aarch64)
    mrjones2014 A Lua implementation of lighthaus-theme/vim-lighthaus
    nikp123 Run an Android desktop environment off of a device using ONLY scrcpy
    hackclub proxy.hackclub.com - An nginx redirector and proxier for all of our needs!
    BlackWolfed This is a Recon & Inoformation Garhering Methodology In Bug Hunting Process
    mrpippy A hack allowing the use of Xcode 9's toolchain on macOS Catalina, Big Sur, and Monterey.
    chentau A better user experience for interacting with and manipulating Vim marks
    Ls4ss Apache HTTP Server 2.4.49, 2.4.50 - Path Traversal & RCE
    wmitsuda Ethereum event log signatures database
    im-hanzou Apache (Linux) CVE-2021-41773/2021-42013 Mass Vulnerability Checker
    MrLutik Tools for validators
    hamadmarri This is a very basic/lightweight yet a very performant CPU scheduler.
    unkn0w Kolekcja skryptów do szybkiego stawiania usług na serwerach Mikrusa
    YouGina ReconMaster contest - scripts used and a write-up

    Related Repos


    Miscellaneous
    20
    Build Dockerfiles directly in your Kubernetes cluster.
    kvaps kubectl-kaniko Kaniko plugin for kubectl. (inspired by GoogleContainerTools/kaniko#1289) The Kaniko wrapper for kubectl mimics the kaniko executor, bu
     
    Miscellaneous
    242
    Container Builder Interface for Kubernetes with support for several backends (Docker, Buil...
    containerbuilding ⚠️ CBI is no longer under active development, in favor of Tekton. See tektoncd/pipeline and tektoncd/catalog. CBI: Container Builder Interface for Kubernetes CBI provides a vendor-neutral interface for buildin
     
    Popular
    Miscellaneous
    9k
    Build Container Images In Kubernetes
    GoogleContainerTools kaniko - Build Images In Kubernetes kaniko is a tool to build container images from a Dockerfile, inside a container or Kubernetes cluster. kaniko doesn't depend on a Docker daemon and executes each command within a Dockerf
     
    Deep Learning
    32
    Boxes and masks visualization tools
    lufficc Vizer Boxes and masks visualization tools. Install pip install Vizer Usage img = Image.open(path) # draw masks img = draw_masks(img, data['masks'], data['labels']) # draw boxes img = draw_boxes(img, b
     
    Command Line
    1.4k
    Docker CLI plugin for extended build capabilities with BuildKit
    docker Docker CLI plugin for extended build capabilities with BuildKit
     
    Overlay
    31
    vue-img-view - a plugin for Vue.js, you can drag / view / rotate pictures anywhere
    la7ender vue-img-view a plugin for Vue.js, you can drag / view / rotate pictures anywhere DEMO DEMO Install # install npm install vue-img-view Setup import img
     
    Utilities
    4.7k
    Buildah - a tool that facilitates building Open Container Initiative (OCI) container image...
    containers Buildah - a tool that facilitates building Open Container Initiative (OCI) container images
     
    Command Line
    112
    BuildKit CLI for kubectl is a tool for building container images with your Kubernetes clus...
    vmware-tanzu BuildKit CLI for kubectl is a tool for building container images with your Kubernetes cluster
     
    Popular
    Miscellaneous
    4.3k
    concurrent, cache-efficient, and Dockerfile-agnostic builder toolkit
    moby BuildKit BuildKit is a toolkit for converting source code to build artifacts in an efficient, expressive and repeatable manner. Key features: Automatic garbage collection Extendable frontend formats Concurrent depe
     
    Miscellaneous
    74
    :whale:dockerdot shows dockerfile dependenciy graph. This is useful to understand how buil...
    po3rin dockerdot dockerdot shows dockerfile dependenciy graph. This is useful to understand how build dockerfile. This uses Go WebAssembly + BuildKit package. 🐳 https://po3rin.github.io/dockerdot/ 🐳 (not support sma
     
    Deep Learning
    10
    Ultra-lightweight human body posture key point CNN model. ModelSize:2.3MB HUAWEI P40 NCNN...
    dog-qiuqiu Ultra-lightweight human body posture key point CNN model. ModelSize:2.3MB HUAWEI P40 NCNN benchmark: 6ms/img,
     
    Miscellaneous
    712
    A swift library to benchmark code snippets.
    google swift-benchmark A Swift library for benchmarking code snippets, similar to google/benchmark. Example: import Benchmark benchmark("add string reserved capacity") { var x: String = "" x.reserveCapacity(2000) for _
     
    Data Visualization
    3k
    A visualizer for Docker Swarm Mode using the Docker Remote API, Node.JS, and D3
    dockersamples Docker Swarm Visualizer *** note *** This only works with Docker Swarm Mode in Docker Engine 1.12.0 and later. It does not work with the separate Docker Swarm project Also this is a sample app meant for learning Docker. Run
     
    Miscellaneous
    39
    Docker Explorer for Visual Studio Code
    formulahendry Docker Explorer Features Manage Docker Containers, Docker Images, Docker Hub and Azure Container Registry Prerequisites Docker is installed For Linux user, make sure you could manage Docker
     
    Miscellaneous
    411
    Scripts and docker compose files to start flow.ci services from docker
    FlowCI Install from Docker within 5 mins Per-requirements Mac or Linux (Windows not supported yet) Docker installed Docker-Compose installed Clone this 'docker' repo (make sure the scripts and docker compose f
     
    Featured
    Command Line
    3.1k
    The Docker CLI
    docker docker/cli This repository is the home of the cli used in the Docker CE and Docker EE products. Development docker/cli is developed using Docker. Build a linux binary: $ make -f docker.Makefile binary Build bin
     
    Vue.js
    410
    v-img is a plugin for Vue.js that allows you to show images in full-screen gallery
    crowdbotics v-img is a plugin for Vue.js that allows you to show images in full-screen gallery by adding only one directive to the <img> tag. Demonstration page Browser support Firefox Chrome Safari
     
    Overlay
    410
    v-img - Easy to install image gallery.
    crowdbotics v-img is a plugin for Vue.js that allows you to show images in full-screen gallery by adding only one directive to the <img> tag. Demonstration page B
     
    Type Select
    559
    vue-img-inputer A graceful, highly customizable img type input for Vue 2
    waynecz 点击查看中文文档 | Have a glance at Demo Intro <input type="file" /> like, only support single image auto-upload support drop-select / preview / use as <img /
     
    HTTP
    279
    Display images in iTerm
    sindresorhus term-img Display images in your terminal Even animated gifs! Currently only supported on iTerm &gt;=3. Install $ npm install --save term-img Usage const termImg = require('term-img'); functio
     
    Image Processing
    10
    jQuery plugin to convert internal image url to blob url & add watermark.
    bachors Img2Blob.js Lightweight javascript-based. Convert internal image url to blob url &amp; add watermark. HTML &lt;h3&gt;Default&lt;/h3&gt; &lt;img class="sample1" data-img2blob="img/1.png" /&gt; &lt;img class="sample1" d