Home / Linux / Miscellaneous

benchmark tool for Docker, BuildKit, img, Buildah, and Kaniko

Buildbench: benchmark tool for Docker, BuildKit, img, Buildah, and Kaniko Requirements bash realpath Docker Usage $ ./buildbench ./examples/ex01 out.csv 5 To disable specific builder, plea
Information
Category: Linux / Miscellaneous
Watchers: 6
Star: 75
Fork: 12
Last update: Dec 4, 2023
Resource links
https://github.com/AkihiroSuda/buildbench

Buildbench: benchmark tool for Docker, BuildKit, img, Buildah, and Kaniko

Build Status

Requirements

  • bash
  • realpath
  • bc
  • docker

Usage

$ ./buildbench ./examples/ex01 out.csv 5
  • To disable specific builder, please set DISABLE_DOCKER, DISABLE_BUILDKIT, and so on.
4 Open More issues
Closed

Update to latest versions as of 2019-04-18 and enable Kaniko cache

Updates all the docker images to the latest I could find on Docker Hub. Buildkit is definitely out of date as the only version found was 0.3.0 and we're now up to 0.4.0. Buildah may also be

opened Apr 18, 2019 by samschlegel 7
Closed

Add makisu

https://github.com/uber/makisu

help wanted 
opened Nov 18, 2018 by AkihiroSuda 1
Closed

Update kaniko with support for cache

help wanted 
opened Nov 18, 2018 by AkihiroSuda 1
Closed

pin builder images

Signed-off-by: Akihiro Suda [email protected]

opened Jun 19, 2018 by AkihiroSuda 0
Closed

Misc improvements

  • use float time
  • support ONLY_${builder} mode
opened Jun 8, 2018 by AkihiroSuda 0
Open

Add gnuplot/matplotlib script

enhancement help wanted good first issue 
opened Oct 3, 2018 by AkihiroSuda 0
Open

Benchmark result (June 19, 2018)

Full logs and version info: https://travis-ci.org/AkihiroSuda/buildbench/builds/393967682?utm_source=github_status&utm_medium=notification ( https://github.com/AkihiroSuda/buildbench/commit/80e6341b62965e28a143ee66788918f7e23909eb )

# May, 2018
DOCKER_IMAGE="docker:18.05-dind"
# BuildKit does not have tagged images; git: moby/buildkit@00140b79929a046614feec5985c8e8f56f7501e7 (June 16, 2018)
BUILDKIT_IMAGE="tonistiigi/buildkit@sha256:38d17109b217c2894063f0bd4b844554901de65add1f9d7cc3b5b73da4f6ab4e"
# June 13, 2018
IMG_IMAGE="r.j3ss.co/img:v0.4.6"
# Buildah does not have tagged images; git: projectatomic/buildah@fc438bb932e891cbe04109cfae1dfbe3c99307a5 (June 15, 2018)
BUILDAH_IMAGE="containerbuilding/buildah@sha256:b03b04c2480521c9ae05c87f28cc2568a347d009de47f4153d30b41f3bf0ccb8"
# May, 2018
KANIKO_IMAGE="gcr.io/kaniko-project/executor:v0.1.0"

Description about workloads: #1

CSV columns: ID, took, begun, finished

ex01.csv

docker-1,18.165849907,1529391592.263071609,1529391610.428921516
docker-2,1.235037047,1529391610.434916377,1529391611.669953424
docker-1,14.818404800,1529391612.367617690,1529391627.186022490
docker-2,1.287132571,1529391627.190408360,1529391628.477540931
docker-1,14.802825026,1529391629.158332039,1529391643.961157065
docker-2,1.228064991,1529391643.965079234,1529391645.193144225
docker-1,15.078891824,1529391645.875215472,1529391660.954107296
docker-2,1.234751728,1529391660.957918448,1529391662.192670176
docker-1,15.178132443,1529391662.856697985,1529391678.034830428
docker-2,1.249452815,1529391678.038773656,1529391679.288226471
buildkit-1,7.861676981,1529391683.423117905,1529391691.284794886
buildkit-2,1.132106418,1529391691.288556542,1529391692.420662960
buildkit-1,7.333715405,1529391693.139716291,1529391700.473431696
buildkit-2,1.153705816,1529391700.477532735,1529391701.631238551
buildkit-1,7.319216463,1529391702.341668182,1529391709.660884645
buildkit-2,1.108104800,1529391709.664805940,1529391710.772910740
buildkit-1,7.638978042,1529391711.533288749,1529391719.172266791
buildkit-2,1.136491177,1529391719.176583838,1529391720.313075015
buildkit-1,9.102700778,1529391721.029759107,1529391730.132459885
buildkit-2,1.183218361,1529391730.136429357,1529391731.319647718
img-1,10.564293780,1529391734.808084287,1529391745.372378067
img-2,1.835848179,1529391745.376254356,1529391747.212102535
img-1,9.769529824,1529391747.431877654,1529391757.201407478
img-2,1.952110230,1529391757.205537429,1529391759.157647659
img-1,9.723758768,1529391759.379980697,1529391769.103739465
img-2,1.817143219,1529391769.107704175,1529391770.924847394
img-1,10.408545686,1529391771.151449561,1529391781.559995247
img-2,1.843239252,1529391781.564312723,1529391783.407551975
img-1,9.514438987,1529391783.633510686,1529391793.147949673
img-2,1.894057609,1529391793.151888472,1529391795.045946081
buildah-1,15.500970782,1529391817.194548251,1529391832.695519033
buildah-2,13.619950535,1529391832.699490829,1529391846.319441364
buildah-1,14.968011827,1529391846.369269058,1529391861.337280885
buildah-2,13.369802949,1529391861.341253218,1529391874.711056167
buildah-1,15.547745947,1529391874.760378952,1529391890.308124899
buildah-2,13.435780378,1529391890.312092187,1529391903.747872565
buildah-1,15.260446171,1529391903.799704630,1529391919.060150801
buildah-2,13.683330997,1529391919.064120785,1529391932.747451782
buildah-1,15.078477477,1529391932.797529318,1529391947.876006795
buildah-2,13.539169131,1529391947.880278848,1529391961.419447979
kaniko-1,14.243428057,1529391963.082480443,1529391977.325908500
kaniko-2,13.409059837,1529391977.330233443,1529391990.739293280
kaniko-1,12.146617865,1529391990.744615692,1529392002.891233557
kaniko-2,12.328629475,1529392002.895359127,1529392015.223988602
kaniko-1,13.466581014,1529392015.228922142,1529392028.695503156
kaniko-2,13.449305711,1529392028.699544314,1529392042.148850025
kaniko-1,13.317310198,1529392042.153942771,1529392055.471252969
kaniko-2,12.848505995,1529392055.475368000,1529392068.323873995
kaniko-1,13.035854787,1529392068.328808640,1529392081.364663427
kaniko-2,13.319246937,1529392081.368680907,1529392094.687927844

ex01.txt

moby.csv

docker-1,353.787756456,1529392007.999389134,1529392361.787145590
docker-2,5.904878890,1529392361.793445013,1529392367.698323903
docker-1,356.152896153,1529392370.380126094,1529392726.533022247
docker-2,5.756538996,1529392726.540641159,1529392732.297180155
docker-1,351.236617027,1529392734.749708553,1529393085.986325580
docker-2,5.694913895,1529393085.990213594,1529393091.685127489
docker-1,346.027762206,1529393094.139444521,1529393440.167206727
docker-2,8.714977267,1529393440.171129941,1529393448.886107208
docker-1,351.950024279,1529393451.371205967,1529393803.321230246
docker-2,5.825282377,1529393803.325100829,1529393809.150383206
buildkit-1,279.197551929,1529392095.673257046,1529392374.870808975
buildkit-2,1.883717697,1529392374.879647411,1529392376.763365108
buildkit-1,279.243992842,1529392379.512312725,1529392658.756305567
buildkit-2,1.844081306,1529392658.760409468,1529392660.604490774
buildkit-1,279.479077391,1529392663.435859295,1529392942.914936686
buildkit-2,1.859363537,1529392942.918906463,1529392944.778270000
buildkit-1,278.989010640,1529392947.566908645,1529393226.555919285
buildkit-2,1.898166498,1529393226.559961109,1529393228.458127607
buildkit-1,277.165037488,1529393231.221658213,1529393508.386695701
buildkit-2,1.872406282,1529393508.390886640,1529393510.263292922
img-1,442.317527554,1529392155.120938513,1529392597.438466067
img-2,7.521936376,1529392597.445835650,1529392604.967772026
img-1,451.095576198,1529392607.323904690,1529393058.419480888
img-2,7.424161858,1529393058.423472851,1529393065.847634709
img-1,447.124890511,1529393068.196446102,1529393515.321336613
img-2,7.677913305,1529393515.325266950,1529393523.003180255
img-1,447.304128181,1529393525.338733986,1529393972.642862167
img-2,7.472302153,1529393972.646785396,1529393980.119087549
img-1,447.636093349,1529393982.401350000,1529394430.037443349
img-2,7.685351348,1529394430.041214689,1529394437.726566037

moby_docker.txt moby_buildkit.txt moby_img.txt

benchmark-result 
opened Jun 19, 2018 by AkihiroSuda 0
Open

Benchmark result (June 9, 2018)

Full logs and version info: https://travis-ci.org/AkihiroSuda/buildbench/builds/389792477?utm_source=github_status&utm_medium=notification

Description about workloads: #1

CSV columns: ID, took, begun, finished

ex01.csv

docker-1,15.088948494,1528516883.067540964,1528516898.156489458
docker-2,1.264920067,1528516898.162242410,1528516899.427162477
docker-1,14.851893349,1528516900.139559065,1528516914.991452414
docker-2,1.296145993,1528516914.995536683,1528516916.291682676
docker-1,14.634571302,1528516916.982249664,1528516931.616820966
docker-2,1.335567962,1528516931.620468566,1528516932.956036528
docker-1,14.938568557,1528516933.646126876,1528516948.584695433
docker-2,1.339203861,1528516948.588702272,1528516949.927906133
docker-1,15.008774077,1528516950.595051276,1528516965.603825353
docker-2,1.287587080,1528516965.608013027,1528516966.895600107
buildkit-1,7.676590955,1528516970.952694997,1528516978.629285952
buildkit-2,1.220504264,1528516978.633374484,1528516979.853878748
buildkit-1,7.520999153,1528516980.576517629,1528516988.097516782
buildkit-2,1.124634502,1528516988.101464458,1528516989.226098960
buildkit-1,7.678619715,1528516989.962819048,1528516997.641438763
buildkit-2,1.172230003,1528516997.645713939,1528516998.817943942
buildkit-1,8.343999825,1528516999.540903954,1528517007.884903779
buildkit-2,1.136848518,1528517007.892358192,1528517009.029206710
buildkit-1,7.540258448,1528517009.767991265,1528517017.308249713
buildkit-2,1.133150257,1528517017.312111999,1528517018.445262256
img-1,9.916585799,1528517021.603791267,1528517031.520377066
img-2,1.895819467,1528517031.524145433,1528517033.419964900
img-1,9.896674246,1528517033.647922903,1528517043.544597149
img-2,1.972287355,1528517043.548718848,1528517045.521006203
img-1,9.927664649,1528517045.753273162,1528517055.680937811
img-2,1.830990818,1528517055.685087238,1528517057.516078056
img-1,10.094313643,1528517057.753391160,1528517067.847704803
img-2,1.972982689,1528517067.851263908,1528517069.824246597
img-1,10.522893321,1528517070.050599802,1528517080.573493123
img-2,1.912381080,1528517080.579451096,1528517082.491832176
buildah-1,13.357741251,1528517102.141753277,1528517115.499494528
buildah-2,12.167797793,1528517115.503435666,1528517127.671233459
buildah-1,12.777023598,1528517127.720550188,1528517140.497573786
buildah-2,10.884054123,1528517140.501773347,1528517151.385827470
buildah-1,14.075376815,1528517151.443113080,1528517165.518489895
buildah-2,11.159878164,1528517165.522856817,1528517176.682734981
buildah-1,12.498024832,1528517176.736068416,1528517189.234093248
buildah-2,11.252820026,1528517189.238772689,1528517200.491592715
buildah-1,13.086742903,1528517200.543874630,1528517213.630617533
buildah-2,11.105947072,1528517213.634928288,1528517224.740875360
kaniko-1,13.949581683,1528517226.325083879,1528517240.274665562
kaniko-2,13.708101622,1528517240.279425567,1528517253.987527189
kaniko-1,12.888729452,1528517253.992463459,1528517266.881192911
kaniko-2,14.101354493,1528517266.885450815,1528517280.986805308
kaniko-1,13.230798628,1528517280.993682640,1528517294.224481268
kaniko-2,13.271245080,1528517294.228877218,1528517307.500122298
kaniko-1,15.104304591,1528517307.512061163,1528517322.616365754
kaniko-2,13.208975221,1528517322.624875939,1528517335.833851160
kaniko-1,13.893086794,1528517335.839334127,1528517349.732420921
kaniko-2,12.690004360,1528517349.739682189,1528517362.429686549

ex01.txt

moby.csv

docker-1,368.328064583,1528516894.627995325,1528517262.956059908
docker-2,5.913627327,1528517262.963389876,1528517268.877017203
docker-1,346.701621722,1528517271.244334087,1528517617.945955809
docker-2,5.887373814,1528517617.950057810,1528517623.837431624
docker-1,354.015341011,1528517626.326424942,1528517980.341765953
docker-2,5.862704967,1528517980.345517145,1528517986.208222112
buildkit-1,281.376391784,1528516969.148981670,1528517250.525373454
buildkit-2,1.772235548,1528517250.531774709,1528517252.304010257
buildkit-1,281.301812854,1528517255.114552131,1528517536.416364985
buildkit-2,1.822073199,1528517536.420459090,1528517538.242532289
buildkit-1,280.564748838,1528517540.955174297,1528517821.519923135
buildkit-2,1.852769957,1528517821.524235519,1528517823.377005476
img-1,430.453649374,1528516883.837080020,1528517314.290729394
img-2,7.088786477,1528517314.303254456,1528517321.392040933
img-1,431.462889460,1528517323.572703808,1528517755.035593268
img-2,7.318575166,1528517755.039628785,1528517762.358203951
img-1,433.546605852,1528517764.603786101,1528518198.150391953
img-2,7.159197898,1528518198.154567821,1528518205.313765719

moby-docker.txt moby-buildkit.txt moby-img.txt

benchmark-result 
opened Jun 9, 2018 by AkihiroSuda 0
Open

Benchmark result (June 6, 2018)

Travis log available: https://travis-ci.org/AkihiroSuda/buildbench/builds/388656138

Buildbench version

https://github.com/AkihiroSuda/buildbench/commit/70c88896bacbd23752195e715339e9d2c48b04d0

Env

TravisCI Ubuntu Trusty ( https://docs.travis-ci.com/user/reference/overview/ )

  • Ext4
  • 7.5GB RAM
  • 2 Cores, bursted

Tested tools

  • Docker 18.05-ce
  • moby/buildkit@376093d234ca9446a3d4dc9956d2eb3afff54baf
  • genuinetools/img@1a47aed9
  • projectatomic/buildah@f90b6c0
  • gcr.io/kaniko-project/executor@sha256:501056bf52f3a96f151ccbeb028715330d5d5aa6647e7572ce6c6c55f91ab374 (Uploaded time: May 18, 2018 at 5:13:31 AM UTC+9)

Benchmark: examples/ex01

  • foo-1: clean environment
  • foo-2: keep foo-1 cache storage, touch buildContext/build-bench-dummy, and then build the image again
  • Tested 5 times

ID | Took (seconds) --------|------------- docker-1 | 20,15,18,16,17 (avg: 17.2) docker-2 | 1,2,2,1,1 (avg: 1.4) buildkit-1 | 12, 13, 8, 9, 8 (avg: 10.0) buildkit-2 | 2, 1, 1, 1, 1 (avg: 1.2) img-1 | 11,11,10,13,14 (avg: 11.8) img-2 | 2,2,2,1,2 (avg: 1.8) buildah-1 | 15, 15, 13, 21, 14 (avg: 15.6) buildah-2 | 11, 11, 15, 14, 17 (avg: 13.6) kaniko-1 | 21, 18, 16, 16, 15 (avg: 17.2) kaniko-2 | 15, 18, 18 26, 16 (avg: 18.6)

image

20180606-ex01.txt

Benchmark: moby/moby@fd2f2a919e392b96de74795ae9af2dc5e510bc4c

  • The image could not be built with Buildah and Kaniko, due to lack of support for FROM anotherstage (EDIT: fixed typo s/anotherimage/anotherstage/)
  • Tested single time only

ID | Took (seconds) --------|------------- docker-1 | 387 docker-2 | 8 buildkit-1 | 266 buildkit-2 | 2 img-1 | 450 img-2 | 6

image

20180606-moby.txt

benchmark-result 
opened Jun 6, 2018 by AkihiroSuda 1

Contents

    kvaps Build Dockerfiles directly in your Kubernetes cluster.
    containerbuilding Container Builder Interface for Kubernetes with support for several backends (Docker, BuildKit, Buildah, kaniko, img, Google Cloud Container Builder, Azure Container Registry Build, OpenShift Source-to-Image...)
    GoogleContainerTools Build Container Images In Kubernetes
    lufficc Boxes and masks visualization tools
    docker Docker CLI plugin for extended build capabilities with BuildKit
    AkihiroSuda Nix derivations as Dockerfiles (`docker build -f default.nix .`)
    forest8618 vue-img-view - a plugin for Vue.js, you can drag / view / rotate pictures anywhere
    containers A tool that facilitates building OCI images.
    containers Buildah - a tool that facilitates building Open Container Initiative (OCI) container images
    vmware-tanzu BuildKit CLI for kubectl is a tool for building container images with your Kubernetes cluster
    moby concurrent, cache-efficient, and Dockerfile-agnostic builder toolkit
    po3rin :whale:dockerdot shows dockerfile dependenciy graph. This is useful to understand how build dockerfile. This uses Go WebAssembly + BuildKit package.
    developer-guy A proof-of-concept project that makes accessible buildkitd daemon from macOS
    google A swift library to benchmark code snippets.
    dog-qiuqiu Ultra-lightweight human body posture key point CNN model. ModelSize:2.3MB HUAWEI P40 NCNN benchmark: 6ms/img,
    crowdbotics v-img is a plugin for Vue.js that allows you to show images in full-screen gallery
    crowdbotics v-img - Easy to install image gallery.
    waynecz vue-img-inputer A graceful, highly customizable img type input for Vue 2
    sindresorhus Display images in iTerm
    bachors jQuery plugin to convert internal image url to blob url & add watermark.
    nikli2009 vue-img-orientation-changer - A Vue.js directive that automatically adjust your img to correct orientation.
    Share this repo

    Related Repos


    Miscellaneous
    14
    the ultimate compact fzf gh extension
    gennaro-tedesco the ultimate compact fzf gh extension Installation • Usage • Customisation • Feedback gh-f is the ultimate, compact and snappy fzf extension for gh cl
     
    Miscellaneous
    6
    MacOS Wallpapers for linux desktop
    vinceliuice MacOS Wallpapers for linux desktop WhiteSur (Big Sur) Wallpapers Monterey Wallpapers Installation Install Gnome Backgrounds (wallpaper will change acc
     
    Miscellaneous
    3
    A docker bundle with *arr apps ready to deploy on your linux machine.
    bolchisb Plex Home Media integrations This package is created for those who are not experts in linux servers and docker. If you follow these steps you'll be ab
     
    Miscellaneous
    3
    Vim-textobj-string is a Vim plugin to provide text objects to select a string
    rbtnn vim-textobj-string is a Vim plugin to provide text objects (as and is by default) to select a string.
     
    Miscellaneous
    3
    automatically quote arguments to commands like `git commit -m`
    ianthehenry zsh-autoquoter zsh-autoquoter is a zle widget ("zsh plugin") that will automatically put quotes around arguments to certain commands. So instead of ha
     
    Miscellaneous
    34
    PyTorchCV: A PyTorch-Based Framework for Deep Learning in Computer Vision.
    donnyyou PyTorchCV: A PyTorch-Based Framework for Deep Learning in Computer Vision @misc{CV2018, author = {Donny You ([email protected])}, howpubl
     
    Miscellaneous
    276
    Documentation on building a HTTPS stack in AWS with HAProxy
    jvehent Guidelines for HAProxy termination in AWS Document status NOT READY $Revision: $ @ 2022-01-14 19:00 PST Author Julien Vehent Review CloudOps Table of
     
    Miscellaneous
    8.3k
    Spinnaker is an open source, multi-cloud continuous delivery platform for releasing softwa...
    Spinnaker Welcome to the Spinnaker Project Spinnaker is an open-source continuous delivery platform for releasing software changes with high velocity and confid
     
    Miscellaneous
    52
    A must-have configuration for Spacemacs users after defecting to Vim
    yetone 🪐 My Neovim Configuration 🚀 A must-have configuration for Spacemacs users after defecting to Vim If you want the power of VSCode, the interactivity
     
    Miscellaneous
    12
    This repo is for firmware updates, software files, and other important resources for the M...
    Takiiiiiiii Miyoo Mini Firmware and Configured SD This repository contains a debloated and improved SD card image for the Miyoo Mini. Latest Firmware Update Packa
     
    Miscellaneous
    7
    This app is created that using with Shell Script and Zenity Library.
    kaansertel CV Hazırlama (Shell Script - Zenity Kütüphanesi) Linux Araçları ve Kabuk Programlama dersi kapsamında,Shell script ve Zenity Kütüphanesi kullanılarak
     
    Miscellaneous
    7
    Docker container for PlexAutoSkip
    mdhiggins PlexAutoSkip Official Docker Container Docker container for PlexAutoSkip Version Tags Tag Description latest Stable release Usage docker-compose ple
     
    Miscellaneous
    29
    An open source meme???
    KCGD Open source spiderman meme Haha spiderman make a presentation lol Install gimp This meme requires gimp, install through install_deps.sh chmod +x insta
     
    Miscellaneous
    35
    Script to patch iOS app so we can later use it on Apple Silicon devices.
    Co2333 AppleSiliconUIKitPatch Script to patch iOS app so we can later use it on Apple Silicon devices. App should be decrypted if was downloaded from App Sto
     
    Miscellaneous
    20
    dots configured after setting up NixOS on tmpfs
    Sam1431 λ Idempotent Dotfiles SETTING UP NIXOS , MY WAY NixOS with tmpfs (Sway WM) Disclaimer: This is not a community framework or distribution. It's a priva