Home / Linux / Miscellaneous

benchmark tool for Docker, BuildKit, img, Buildah, and Kaniko

Buildbench: benchmark tool for Docker, BuildKit, img, Buildah, and Kaniko Requirements bash realpath Docker Usage $ ./buildbench ./examples/ex01 out.csv 5 To disable specific builder, plea

Buildbench: benchmark tool for Docker, BuildKit, img, Buildah, and Kaniko

Build Status

Requirements

  • bash
  • realpath
  • bc
  • docker

Usage

$ ./buildbench ./examples/ex01 out.csv 5
  • To disable specific builder, please set DISABLE_DOCKER, DISABLE_BUILDKIT, and so on.
5 Open More issues
Closed

Update to latest versions as of 2019-04-18 and enable Kaniko cache

Updates all the docker images to the latest I could find on Docker Hub. Buildkit is definitely out of date as the only version found was 0.3.0 and we're now up to 0.4.0. Buildah may also be

opened Apr 18, 2019 by samschlegel 7
Closed

Update kaniko with support for cache

help wanted 
opened Nov 18, 2018 by AkihiroSuda 1
Closed

Misc improvements

  • use float time
  • support ONLY_${builder} mode
opened Jun 8, 2018 by AkihiroSuda 0
Closed

pin builder images

Signed-off-by: Akihiro Suda [email protected]

opened Jun 19, 2018 by AkihiroSuda 0
Open

Add makisu

https://github.com/uber/makisu

help wanted 
opened Nov 18, 2018 by AkihiroSuda 0
Open

Add gnuplot/matplotlib script

enhancement good first issue help wanted 
opened Oct 3, 2018 by AkihiroSuda 0
Open

Benchmark result (June 19, 2018)

Full logs and version info: https://travis-ci.org/AkihiroSuda/buildbench/builds/393967682?utm_source=github_status&utm_medium=notification ( https://github.com/AkihiroSuda/buildbench/commit/80e6341b62965e28a143ee66788918f7e23909eb )

# May, 2018
DOCKER_IMAGE="docker:18.05-dind"
# BuildKit does not have tagged images; git: moby/[email protected] (June 16, 2018)
BUILDKIT_IMAGE="tonistiigi/[email protected]:38d17109b217c2894063f0bd4b844554901de65add1f9d7cc3b5b73da4f6ab4e"
# June 13, 2018
IMG_IMAGE="r.j3ss.co/img:v0.4.6"
# Buildah does not have tagged images; git: projectatomic/[email protected] (June 15, 2018)
BUILDAH_IMAGE="containerbuilding/[email protected]:b03b04c2480521c9ae05c87f28cc2568a347d009de47f4153d30b41f3bf0ccb8"
# May, 2018
KANIKO_IMAGE="gcr.io/kaniko-project/executor:v0.1.0"

Description about workloads: #1

CSV columns: ID, took, begun, finished

ex01.csv

docker-1,18.165849907,1529391592.263071609,1529391610.428921516
docker-2,1.235037047,1529391610.434916377,1529391611.669953424
docker-1,14.818404800,1529391612.367617690,1529391627.186022490
docker-2,1.287132571,1529391627.190408360,1529391628.477540931
docker-1,14.802825026,1529391629.158332039,1529391643.961157065
docker-2,1.228064991,1529391643.965079234,1529391645.193144225
docker-1,15.078891824,1529391645.875215472,1529391660.954107296
docker-2,1.234751728,1529391660.957918448,1529391662.192670176
docker-1,15.178132443,1529391662.856697985,1529391678.034830428
docker-2,1.249452815,1529391678.038773656,1529391679.288226471
buildkit-1,7.861676981,1529391683.423117905,1529391691.284794886
buildkit-2,1.132106418,1529391691.288556542,1529391692.420662960
buildkit-1,7.333715405,1529391693.139716291,1529391700.473431696
buildkit-2,1.153705816,1529391700.477532735,1529391701.631238551
buildkit-1,7.319216463,1529391702.341668182,1529391709.660884645
buildkit-2,1.108104800,1529391709.664805940,1529391710.772910740
buildkit-1,7.638978042,1529391711.533288749,1529391719.172266791
buildkit-2,1.136491177,1529391719.176583838,1529391720.313075015
buildkit-1,9.102700778,1529391721.029759107,1529391730.132459885
buildkit-2,1.183218361,1529391730.136429357,1529391731.319647718
img-1,10.564293780,1529391734.808084287,1529391745.372378067
img-2,1.835848179,1529391745.376254356,1529391747.212102535
img-1,9.769529824,1529391747.431877654,1529391757.201407478
img-2,1.952110230,1529391757.205537429,1529391759.157647659
img-1,9.723758768,1529391759.379980697,1529391769.103739465
img-2,1.817143219,1529391769.107704175,1529391770.924847394
img-1,10.408545686,1529391771.151449561,1529391781.559995247
img-2,1.843239252,1529391781.564312723,1529391783.407551975
img-1,9.514438987,1529391783.633510686,1529391793.147949673
img-2,1.894057609,1529391793.151888472,1529391795.045946081
buildah-1,15.500970782,1529391817.194548251,1529391832.695519033
buildah-2,13.619950535,1529391832.699490829,1529391846.319441364
buildah-1,14.968011827,1529391846.369269058,1529391861.337280885
buildah-2,13.369802949,1529391861.341253218,1529391874.711056167
buildah-1,15.547745947,1529391874.760378952,1529391890.308124899
buildah-2,13.435780378,1529391890.312092187,1529391903.747872565
buildah-1,15.260446171,1529391903.799704630,1529391919.060150801
buildah-2,13.683330997,1529391919.064120785,1529391932.747451782
buildah-1,15.078477477,1529391932.797529318,1529391947.876006795
buildah-2,13.539169131,1529391947.880278848,1529391961.419447979
kaniko-1,14.243428057,1529391963.082480443,1529391977.325908500
kaniko-2,13.409059837,1529391977.330233443,1529391990.739293280
kaniko-1,12.146617865,1529391990.744615692,1529392002.891233557
kaniko-2,12.328629475,1529392002.895359127,1529392015.223988602
kaniko-1,13.466581014,1529392015.228922142,1529392028.695503156
kaniko-2,13.449305711,1529392028.699544314,1529392042.148850025
kaniko-1,13.317310198,1529392042.153942771,1529392055.471252969
kaniko-2,12.848505995,1529392055.475368000,1529392068.323873995
kaniko-1,13.035854787,1529392068.328808640,1529392081.364663427
kaniko-2,13.319246937,1529392081.368680907,1529392094.687927844

ex01.txt

moby.csv

docker-1,353.787756456,1529392007.999389134,1529392361.787145590
docker-2,5.904878890,1529392361.793445013,1529392367.698323903
docker-1,356.152896153,1529392370.380126094,1529392726.533022247
docker-2,5.756538996,1529392726.540641159,1529392732.297180155
docker-1,351.236617027,1529392734.749708553,1529393085.986325580
docker-2,5.694913895,1529393085.990213594,1529393091.685127489
docker-1,346.027762206,1529393094.139444521,1529393440.167206727
docker-2,8.714977267,1529393440.171129941,1529393448.886107208
docker-1,351.950024279,1529393451.371205967,1529393803.321230246
docker-2,5.825282377,1529393803.325100829,1529393809.150383206
buildkit-1,279.197551929,1529392095.673257046,1529392374.870808975
buildkit-2,1.883717697,1529392374.879647411,1529392376.763365108
buildkit-1,279.243992842,1529392379.512312725,1529392658.756305567
buildkit-2,1.844081306,1529392658.760409468,1529392660.604490774
buildkit-1,279.479077391,1529392663.435859295,1529392942.914936686
buildkit-2,1.859363537,1529392942.918906463,1529392944.778270000
buildkit-1,278.989010640,1529392947.566908645,1529393226.555919285
buildkit-2,1.898166498,1529393226.559961109,1529393228.458127607
buildkit-1,277.165037488,1529393231.221658213,1529393508.386695701
buildkit-2,1.872406282,1529393508.390886640,1529393510.263292922
img-1,442.317527554,1529392155.120938513,1529392597.438466067
img-2,7.521936376,1529392597.445835650,1529392604.967772026
img-1,451.095576198,1529392607.323904690,1529393058.419480888
img-2,7.424161858,1529393058.423472851,1529393065.847634709
img-1,447.124890511,1529393068.196446102,1529393515.321336613
img-2,7.677913305,1529393515.325266950,1529393523.003180255
img-1,447.304128181,1529393525.338733986,1529393972.642862167
img-2,7.472302153,1529393972.646785396,1529393980.119087549
img-1,447.636093349,1529393982.401350000,1529394430.037443349
img-2,7.685351348,1529394430.041214689,1529394437.726566037

moby_docker.txt moby_buildkit.txt moby_img.txt

benchmark-result 
opened Jun 19, 2018 by AkihiroSuda 0
Open

Benchmark result (June 9, 2018)

Full logs and version info: https://travis-ci.org/AkihiroSuda/buildbench/builds/389792477?utm_source=github_status&utm_medium=notification

Description about workloads: #1

CSV columns: ID, took, begun, finished

ex01.csv

docker-1,15.088948494,1528516883.067540964,1528516898.156489458
docker-2,1.264920067,1528516898.162242410,1528516899.427162477
docker-1,14.851893349,1528516900.139559065,1528516914.991452414
docker-2,1.296145993,1528516914.995536683,1528516916.291682676
docker-1,14.634571302,1528516916.982249664,1528516931.616820966
docker-2,1.335567962,1528516931.620468566,1528516932.956036528
docker-1,14.938568557,1528516933.646126876,1528516948.584695433
docker-2,1.339203861,1528516948.588702272,1528516949.927906133
docker-1,15.008774077,1528516950.595051276,1528516965.603825353
docker-2,1.287587080,1528516965.608013027,1528516966.895600107
buildkit-1,7.676590955,1528516970.952694997,1528516978.629285952
buildkit-2,1.220504264,1528516978.633374484,1528516979.853878748
buildkit-1,7.520999153,1528516980.576517629,1528516988.097516782
buildkit-2,1.124634502,1528516988.101464458,1528516989.226098960
buildkit-1,7.678619715,1528516989.962819048,1528516997.641438763
buildkit-2,1.172230003,1528516997.645713939,1528516998.817943942
buildkit-1,8.343999825,1528516999.540903954,1528517007.884903779
buildkit-2,1.136848518,1528517007.892358192,1528517009.029206710
buildkit-1,7.540258448,1528517009.767991265,1528517017.308249713
buildkit-2,1.133150257,1528517017.312111999,1528517018.445262256
img-1,9.916585799,1528517021.603791267,1528517031.520377066
img-2,1.895819467,1528517031.524145433,1528517033.419964900
img-1,9.896674246,1528517033.647922903,1528517043.544597149
img-2,1.972287355,1528517043.548718848,1528517045.521006203
img-1,9.927664649,1528517045.753273162,1528517055.680937811
img-2,1.830990818,1528517055.685087238,1528517057.516078056
img-1,10.094313643,1528517057.753391160,1528517067.847704803
img-2,1.972982689,1528517067.851263908,1528517069.824246597
img-1,10.522893321,1528517070.050599802,1528517080.573493123
img-2,1.912381080,1528517080.579451096,1528517082.491832176
buildah-1,13.357741251,1528517102.141753277,1528517115.499494528
buildah-2,12.167797793,1528517115.503435666,1528517127.671233459
buildah-1,12.777023598,1528517127.720550188,1528517140.497573786
buildah-2,10.884054123,1528517140.501773347,1528517151.385827470
buildah-1,14.075376815,1528517151.443113080,1528517165.518489895
buildah-2,11.159878164,1528517165.522856817,1528517176.682734981
buildah-1,12.498024832,1528517176.736068416,1528517189.234093248
buildah-2,11.252820026,1528517189.238772689,1528517200.491592715
buildah-1,13.086742903,1528517200.543874630,1528517213.630617533
buildah-2,11.105947072,1528517213.634928288,1528517224.740875360
kaniko-1,13.949581683,1528517226.325083879,1528517240.274665562
kaniko-2,13.708101622,1528517240.279425567,1528517253.987527189
kaniko-1,12.888729452,1528517253.992463459,1528517266.881192911
kaniko-2,14.101354493,1528517266.885450815,1528517280.986805308
kaniko-1,13.230798628,1528517280.993682640,1528517294.224481268
kaniko-2,13.271245080,1528517294.228877218,1528517307.500122298
kaniko-1,15.104304591,1528517307.512061163,1528517322.616365754
kaniko-2,13.208975221,1528517322.624875939,1528517335.833851160
kaniko-1,13.893086794,1528517335.839334127,1528517349.732420921
kaniko-2,12.690004360,1528517349.739682189,1528517362.429686549

ex01.txt

moby.csv

docker-1,368.328064583,1528516894.627995325,1528517262.956059908
docker-2,5.913627327,1528517262.963389876,1528517268.877017203
docker-1,346.701621722,1528517271.244334087,1528517617.945955809
docker-2,5.887373814,1528517617.950057810,1528517623.837431624
docker-1,354.015341011,1528517626.326424942,1528517980.341765953
docker-2,5.862704967,1528517980.345517145,1528517986.208222112
buildkit-1,281.376391784,1528516969.148981670,1528517250.525373454
buildkit-2,1.772235548,1528517250.531774709,1528517252.304010257
buildkit-1,281.301812854,1528517255.114552131,1528517536.416364985
buildkit-2,1.822073199,1528517536.420459090,1528517538.242532289
buildkit-1,280.564748838,1528517540.955174297,1528517821.519923135
buildkit-2,1.852769957,1528517821.524235519,1528517823.377005476
img-1,430.453649374,1528516883.837080020,1528517314.290729394
img-2,7.088786477,1528517314.303254456,1528517321.392040933
img-1,431.462889460,1528517323.572703808,1528517755.035593268
img-2,7.318575166,1528517755.039628785,1528517762.358203951
img-1,433.546605852,1528517764.603786101,1528518198.150391953
img-2,7.159197898,1528518198.154567821,1528518205.313765719

moby-docker.txt moby-buildkit.txt moby-img.txt

benchmark-result 
opened Jun 9, 2018 by AkihiroSuda 0
Open

Benchmark result (June 6, 2018)

Travis log available: https://travis-ci.org/AkihiroSuda/buildbench/builds/388656138

Buildbench version

https://github.com/AkihiroSuda/buildbench/commit/70c88896bacbd23752195e715339e9d2c48b04d0

Env

TravisCI Ubuntu Trusty ( https://docs.travis-ci.com/user/reference/overview/ )

  • Ext4
  • 7.5GB RAM
  • 2 Cores, bursted

Tested tools

Benchmark: examples/ex01

  • foo-1: clean environment
  • foo-2: keep foo-1 cache storage, touch buildContext/build-bench-dummy, and then build the image again
  • Tested 5 times

ID | Took (seconds) --------|------------- docker-1 | 20,15,18,16,17 (avg: 17.2) docker-2 | 1,2,2,1,1 (avg: 1.4) buildkit-1 | 12, 13, 8, 9, 8 (avg: 10.0) buildkit-2 | 2, 1, 1, 1, 1 (avg: 1.2) img-1 | 11,11,10,13,14 (avg: 11.8) img-2 | 2,2,2,1,2 (avg: 1.8) buildah-1 | 15, 15, 13, 21, 14 (avg: 15.6) buildah-2 | 11, 11, 15, 14, 17 (avg: 13.6) kaniko-1 | 21, 18, 16, 16, 15 (avg: 17.2) kaniko-2 | 15, 18, 18 26, 16 (avg: 18.6)

image

20180606-ex01.txt

Benchmark: moby/[email protected]

  • The image could not be built with Buildah and Kaniko, due to lack of support for FROM anotherstage (EDIT: fixed typo s/anotherimage/anotherstage/)
  • Tested single time only

ID | Took (seconds) --------|------------- docker-1 | 387 docker-2 | 8 buildkit-1 | 266 buildkit-2 | 2 img-1 | 450 img-2 | 6

image

20180606-moby.txt

benchmark-result 
opened Jun 6, 2018 by AkihiroSuda 1
Information
Category: Linux / Miscellaneous
Watchers: 5
Star: 67
Fork: 10
Last update: May 10, 2021

Contents
    Resource links
    https://github.com/AkihiroSuda/buildbench
    Share this repo
    Most popular
    neovim Vim-fork focused on extensibility and usability
    trimstray A collection of inspiring lists, manuals, cheatsheets, blogs, hacks, one-liners, cli/web tools and more.
    alacritty A cross-platform, GPU-accelerated terminal emulator
    immutable-js Immutable persistent data collections for Javascript which increase efficiency and simplicity.
    preactjs ⚛️ Fast 3kB React alternative with the same modern API. Components & Virtual DOM.
    dylanaraps 📖 A collection of pure bash alternatives to external processes.
    JakeWharton Bind Android views and callbacks to fields and methods.
    BurntSushi ripgrep recursively searches directories for a regex pattern
    ycm-core A code-completion engine for Vim
    syl20bnr A community-driven Emacs distribution - The best editor is neither Emacs nor Vim, it's Emacs *and* Vim!

    Related Repos


    Miscellaneous
    2k
    dadbod.vim: Modern database interface for Vim
    tpope dadbod.vim Dadbod is a Vim plugin for interacting with databases. It's a more modern take on dbext.vim, improving on it on the following ways: Connections are specified with a single URL, rather than prompting you for 14 sepa
     
    Miscellaneous
    2k
    dadbod.vim: Modern database interface for Vim
    tpope dadbod.vim Dadbod is a Vim plugin for interacting with databases. It's a more modern take on dbext.vim, improving on it on the following ways: Connections are specified with a single URL, rather than prompting you for 14 sepa
     
    Miscellaneous
    70
    bootstrap and run any project with the same command
    bkeepers a command line tool that provides bootstrap scripts to run all projects Easily switch between projects that are written in different languages without having to remember how each language handles common tasks like installing depende
     
    Miscellaneous
    1.3k
    This script is intended to automate your reconnaissance process in an organized fashion
    nahamsec _ ____ ____ ___ _ ____ _____ ____ ____ _ / \ / _ \/_ \\ \/// __\/ __// _Y _ \/ \ /| | | | / \| / / \ / | \/|| \ | / | / \|| |\ || | |_/\| |-||/ /_ / / | /| /_ | \_| \_/|| | \|| \____/\_/ \|\____//_
     
    Miscellaneous
    43
    Private Docker registry protected with Let's Encrypt SSL
    fzuleta Five-step setup for private Docker repository with Let's Encrypt SSL Make sure you have a domain name and SSH access to an instance. This works well under a DigitalOcean's Ubuntu with Docker and Docker Compose installed.
     
    Miscellaneous
    489
    Zsh plugin for Yarn autocompletions.
    g-plane zsh-yarn-autocompletions What can it do? This Zsh plugin can read the package.json file in current directory and show you what scripts you can run and what packages you can remove. Installation
     
    Miscellaneous
    1
    linux sub_task scheduler based on msg drive
    FondFish usage：linux task-sub task second schduler，based on msg diver。 compile：gcc sche.h config.h sche.c config.c main.c -Wl,-lpthread -lrt -o test . Or exec build script，need install cmake，elf file will be genrated in output。 [email protected]:~/SCHE$ ./test
     
    Popular
    Miscellaneous
    3.8k
    Jenkins X provides automated CI+CD for Kubernetes with Preview Environments on Pull Reques...
    jenkins-x JX JX is a command line tool for installing and using Jenkins X Installing Check out how to install jx Getting Help To find out the available commands type: jx Or to get help on a specific