machinable provides a modular configuration system for machine learning research

machinable provides a system to manage configuration of machine learning projects more effectively. Using straight-forward conventions and a powerful configuration engine, it can help structure your projects in a principled way so you can move quickly while enabling reuse and collaboration.

Information

Category: Python / Deep Learning
Watchers: 4
Star: 30
Fork: 1
Last update: Aug 4, 2022

Resource links

https://github.com/machinable-org/machinable

machinable

machinable provides a system to manage configuration of machine learning projects more effectively. Using straight-forward conventions and a powerful configuration engine, it can help structure your projects in a principled way so you can move quickly while enabling reuse and collaboration.

Read the user guide to get started.

Features

Powerful configuration

YAML-based project-wide configuration files with expressive syntax
Efficient configuration manipulation
Modular code organisation to allow for encapsulation and re-use
Import system to use 3rd party configuration and code without overhead
'Mixins' for horizontal inheritance structure

Efficient execution

Works with existing code
Support for seamless cloud execution
Automatic code backups
Managed randomness and reproducibility
Advanced hyperparameter tuning using Ray Tune

Effective result collection and analysis

Logging, tabular record writer and storage API
File system abstraction (in-memory, AWS S3, and more)
Flat-file result database with convenient query syntax

11 Open More issues

Closed

Bump elliptic from 6.5.3 to 6.5.4

Bumps elliptic from 6.5.3 to 6.5.4.

Commits

43ac7f2 6.5.4
f4bc72b package: bump deps
441b742 ec: validate that a point before deriving keys
e71b2d9 lib: relint using eslint
8421a01 build(deps): bump elliptic from 6.4.1 to 6.5.3 (#231)
See full diff in compare view

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

@dependabot rebase will rebase this PR
@dependabot recreate will recreate this PR, overwriting any edits that have been made to it
@dependabot merge will merge this PR after your CI passes on it
@dependabot squash and merge will squash and merge this PR after your CI passes on it
@dependabot cancel merge will cancel a previously requested merge and block automerging
@dependabot reopen will reopen this PR if it is closed
@dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
@dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
@dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
@dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) You can disable automated security fix PRs for this repo from the Security Alerts page.

dependencies javascript

opened Apr 25, 2021 by dependabot[bot] 2

Closed

:arrow_up: Bump actions/cache from v2 to v2.1.4

Bumps actions/cache from v2 to v2.1.4.

Release notes

Sourced from actions/cache's releases.

v2.1.4

Make caching more verbose #650

Use GNU tar on macOS if available #701

Commits

26968a0 Make save/restore logs akin (#509)
aeaf731 Use @actions/cache version 1.0.6 (#525)
56a8a2f Merge pull request #514 from eregon/recommend-setup-ruby-bundler-cache
1bfe3ac Recommend ruby/setup-ruby's bundler-cache: true option
3543324 Merge pull request #434 from DanielHabenicht/patch-1
3303695 Merge pull request #507 from odin-delrio/patch-1
e64ab30 Improved gradle cache key calculation example
26c48dc Merge pull request #506 from actions/cache-matrix-example
72f66cf Added a cache example when using matrix
9f3a4d3 Merge pull request #443 from guilleijo/pipenv-example
Additional commits viewable in compare view

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

@dependabot rebase will rebase this PR
@dependabot recreate will recreate this PR, overwriting any edits that have been made to it
@dependabot merge will merge this PR after your CI passes on it
@dependabot squash and merge will squash and merge this PR after your CI passes on it
@dependabot cancel merge will cancel a previously requested merge and block automerging
@dependabot reopen will reopen this PR if it is closed
@dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
@dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
@dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
@dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)

dependencies github_actions

opened Feb 8, 2021 by dependabot[bot] 2

Closed

:arrow_up: Bump pyupgrade from 2.34.0 to 2.37.2

Bumps pyupgrade from 2.34.0 to 2.37.2.

Commits

28d3c0b v2.37.2
e157f70 Merge pull request #684 from asottile/update-typing-symbols
46a809f sync import rewrites from reorder-python-imports
11de5f0 Merge pull request #681 from asottile/pre-commit-ci-update-config
ef16c5d [pre-commit.ci] pre-commit autoupdate
d232077 v2.37.1
7b372d1 Merge pull request #680 from asottile/indented-all-removed
80a63c0 fix replacement of entire indented import
6ef17c3 Merge pull request #679 from asottile/typing-callable
44742d8 only rewrite typing.Callable in 3.10+
Additional commits viewable in compare view

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

@dependabot rebase will rebase this PR
@dependabot recreate will recreate this PR, overwriting any edits that have been made to it
@dependabot merge will merge this PR after your CI passes on it
@dependabot squash and merge will squash and merge this PR after your CI passes on it
@dependabot cancel merge will cancel a previously requested merge and block automerging
@dependabot reopen will reopen this PR if it is closed
@dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
@dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
@dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
@dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)

dependencies python

opened Jul 19, 2022 by dependabot[bot] 1

Closed

:arrow_up: Bump pyupgrade from 2.34.0 to 2.37.1

Bumps pyupgrade from 2.34.0 to 2.37.1.

Commits

d232077 v2.37.1
7b372d1 Merge pull request #680 from asottile/indented-all-removed
80a63c0 fix replacement of entire indented import
6ef17c3 Merge pull request #679 from asottile/typing-callable
44742d8 only rewrite typing.Callable in 3.10+
8eb196d v2.37.0
4df7d46 Merge pull request #676 from asottile/double-parse-when-removing
9870f05 avoid double parse when removing imports
d994398 Merge pull request #675 from asottile/imports-indented
b1724dd fix indentation for import-imports being added
Additional commits viewable in compare view

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

@dependabot rebase will rebase this PR
@dependabot recreate will recreate this PR, overwriting any edits that have been made to it
@dependabot merge will merge this PR after your CI passes on it
@dependabot squash and merge will squash and merge this PR after your CI passes on it
@dependabot cancel merge will cancel a previously requested merge and block automerging
@dependabot reopen will reopen this PR if it is closed
@dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
@dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
@dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
@dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)

dependencies python

opened Jul 11, 2022 by dependabot[bot] 1

Closed

:arrow_up: Bump pyupgrade from 2.34.0 to 2.37.0

Bumps pyupgrade from 2.34.0 to 2.37.0.

Commits

8eb196d v2.37.0
4df7d46 Merge pull request #676 from asottile/double-parse-when-removing
9870f05 avoid double parse when removing imports
d994398 Merge pull request #675 from asottile/imports-indented
b1724dd fix indentation for import-imports being added
2635b18 Merge pull request #674 from asottile/regen-reorder-python-imports
ada7e01 rewrite pipes.quote -> shlex.quote
35e82d7 v2.36.0
1f86c30 Merge pull request #673 from asottile/mock-imports
1a904a7 combine mock and imports plugins
Additional commits viewable in compare view

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

@dependabot rebase will rebase this PR
@dependabot recreate will recreate this PR, overwriting any edits that have been made to it
@dependabot merge will merge this PR after your CI passes on it
@dependabot squash and merge will squash and merge this PR after your CI passes on it
@dependabot cancel merge will cancel a previously requested merge and block automerging
@dependabot reopen will reopen this PR if it is closed
@dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
@dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
@dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
@dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)

dependencies python

opened Jul 10, 2022 by dependabot[bot] 1

Closed

:arrow_up: Bump pre-commit from 2.19.0 to 2.20.0

⚠️ Dependabot is rebasing this PR ⚠️

Rebasing might not happen immediately, so don't worry if this takes some time.

Note: if you make any changes to this PR yourself, they will take precedence over the rebase.

Bumps pre-commit from 2.19.0 to 2.20.0.

Release notes

Sourced from pre-commit's releases.

pre-commit v2.20.0

Features

Expose source and object-name (positional args) of prepare-commit-msg hook as PRE_COMMIT_COMIT_MSG_SOURCE and PRE_COMMIT_COMMIT_OBJECT_NAME.

#2407 PR by @M-Whitaker.

#2406 issue by @M-Whitaker.

Fixes

Fix language: ruby installs when --user-install is set in gemrc.

#2394 PR by @narpfel.

#2393 issue by @narpfel.

Adjust pty setup for solaris.

#2390 PR by @gaige.

#2389 issue by @gaige.

Remove unused --config option from gc, sample-config, validate-config, validate-manifest sub-commands.

#2429 PR by @asottile.

Changelog

Sourced from pre-commit's changelog.

2.20.0 - 2022-07-10

Features

Expose source and object-name (positional args) of prepare-commit-msg hook as PRE_COMMIT_COMIT_MSG_SOURCE and PRE_COMMIT_COMMIT_OBJECT_NAME.

#2407 PR by @M-Whitaker.

#2406 issue by @M-Whitaker.

Fixes

Fix language: ruby installs when --user-install is set in gemrc.

#2394 PR by @narpfel.

#2393 issue by @narpfel.

Adjust pty setup for solaris.

#2390 PR by @gaige.

#2389 issue by @gaige.

Remove unused --config option from gc, sample-config, validate-config, validate-manifest sub-commands.

#2429 PR by @asottile.

Commits

78a2d86 v2.20.0
e3dc5b7 Merge pull request #2454 from pre-commit/asottile-patch-1
ebce88c remove warnings checks
d6cc8a1 Merge pull request #2453 from hroncok/python3.11
901e831 Tests: Adjust traceback regexes to allow Python 3.11+ ^^^^^^^
98bb7e6 Merge pull request #2440 from pre-commit/pre-commit-ci-update-config
706d1e9 Merge pull request #2439 from pre-commit/all-repos_autofix_type-checking
3ebd101 [pre-commit.ci] pre-commit autoupdate
d8b5930 remove imports from TYPE_CHECKING (py37+)
170335c Merge pull request #2429 from pre-commit/remove-config-option-when-unused
Additional commits viewable in compare view

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

@dependabot rebase will rebase this PR
@dependabot recreate will recreate this PR, overwriting any edits that have been made to it
@dependabot merge will merge this PR after your CI passes on it
@dependabot squash and merge will squash and merge this PR after your CI passes on it
@dependabot cancel merge will cancel a previously requested merge and block automerging
@dependabot reopen will reopen this PR if it is closed
@dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
@dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
@dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
@dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)

dependencies python

opened Jul 10, 2022 by dependabot[bot] 1

Open

:arrow_up: Bump pydantic from 1.9.1 to 1.9.2

Bumps pydantic from 1.9.1 to 1.9.2.

Release notes

Sourced from pydantic's releases.

v1.9.2 (2022-08-11)

Revert Breaking Change: v1.9.1 introduced a breaking change where model fields were deep copied by default, this release reverts the default behaviour to match v1.9.0 and before, while also allow deep-copy behaviour via copy_on_model_validation = 'deep'. See #4092 for more information.

Allow for shallow copies of model fields, Config.copy_on_model_validation is now a str which must be 'none', 'deep', or 'shallow' corresponding to not copying, deep copy & shallow copy; default 'shallow', #4093 by @timkpaine

Changelog

Sourced from pydantic's changelog.

v1.9.2 (2022-08-11)

Revert Breaking Change: v1.9.1 introduced a breaking change where model fields were deep copied by default, this release reverts the default behaviour to match v1.9.0 and before, while also allow deep-copy behaviour via copy_on_model_validation = 'deep'. See #4092 for more information.

Allow for shallow copies of model fields, Config.copy_on_model_validation is now a str which must be 'none', 'deep', or 'shallow' corresponding to not copying, deep copy & shallow copy; default 'shallow', #4093 by @timkpaine

Commits

1d37ffd uprev and prepare for release
1d6a6e6 allow for shallow copies (#4093)
5774756 comments mostly from @PrettyWood (#4226)
80ec177 Pydantic V2 blog (#4218)
See full diff in compare view

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

@dependabot rebase will rebase this PR
@dependabot recreate will recreate this PR, overwriting any edits that have been made to it
@dependabot merge will merge this PR after your CI passes on it
@dependabot squash and merge will squash and merge this PR after your CI passes on it
@dependabot cancel merge will cancel a previously requested merge and block automerging
@dependabot reopen will reopen this PR if it is closed
@dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
@dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
@dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
@dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)

dependencies python

opened Aug 11, 2022 by dependabot[bot] 0

Open

Improve multiprocessing execution

[ ] allow creation of follow up execution from within processes (dependencies)
[ ] improve stack trace communication
[ ] #316

opened Jun 7, 2022 by frthjf 0

Open

Support output redirection in multiprocessing execution

This feature was previously removed due to conflicts with the pytest output capturing mechanism.

opened Jun 7, 2022 by frthjf 0

Open

Allow to express settings in settings.toml

Once Python 3.11 becomes available, use the stdlib toml parser to make settings more readable.

opened Jun 7, 2022 by frthjf 0

Open

Enable doc string linter

Example setup

opened Dec 30, 2020 by frthjf 0

Open

Enable mypy and increase type annotation coverage

Example setup

opened Dec 30, 2020 by frthjf 0

v3.0(Jul 28, 2021)

Source code(tar.gz)
Source code(zip)

v3.0b2(Jul 27, 2021)

Source code(tar.gz)
Source code(zip)

v3.0b1(Jul 26, 2021)

v3 preview release

Source code(tar.gz)
Source code(zip)

v2.14.0(Feb 4, 2021)

Nest submissions under individual components to simplify storage retrival

Source code(tar.gz)
Source code(zip)

v2.13.0(Dec 19, 2020)

Add URL to submission serialization
Simplify view registration methods

Source code(tar.gz)
Source code(zip)

v2.12.2(Dec 17, 2020)

Allow registration overwrite in Execution.set_schedule

Source code(tar.gz)
Source code(zip)

A modular toolbox for meta-learning research with a focus on speed and reproducibility.

A modular framework for vision & language multimodal research from Facebook AI Research (FAIR)

NitroML is a modular, portable, and scalable model-quality benchmarking framework for Machine Learning and Automated Machine Learning (AutoML) pipelines.

MMFa modular framework for vision and language multimodal research from Facebook AI Research.

Easy Machine Learning is a general-purpose dataflow-based system for easing the process of applying machine learning algorithms to real world tasks.

Tagged Modular Configuration

Reverb is an efficient and easy-to-use data storage and transport system designed for machine learning research

🔥 Cogitare - A Modern, Fast, and Modular Deep Learning and Machine Learning framework for Python

Vowpal Wabbit is a machine learning system which pushes the frontier of machine learning

FedML: A Research Library and Benchmark for Federated Machine Learning

A high-level machine learning and deep learning library for the PHP language.

Accompanying source code for Machine Learning with TensorFlow. Refer to the book for step-by-step explanations.

ML.NET is an open source and cross-platform machine learning framework for .NET.

Rumale is a machine learning library in Ruby

MOCA (Modular Object-Centric Approach) addresses the task of long horizon instruction following with a modular architecture that decouples a task into visual perception and action policy prediction.

Machine Learning Tutorials in Python

Datumbox is an open-source Machine Learning framework written in Java which allows the rapid development of Machine Learning and Statistical applications.

Machine Learning Research

Spatio-temporal datasets collected for network science, deep learning and general machine learning research.

The Research Tree - A playground for research at the intersection of Continual, Reinforcement, and Self-Supervised Learning.

A mindmap summarising Machine Learning concepts, from Data Analysis to Deep Learning.

Share this repo

Related Repos

Deep Learning

837

An implementation of chunked, compressed, N-dimensional arrays for Python.

Zarr Latest Release Package Status License Build Status Coverage Downloads Gitter Citation What is it? Zarr is a Python package providing an implement

Deep Learning

317

Serverless proxy for Spark cluster

Hydrosphere Mist Hydrosphere Mist is a serverless proxy for Spark cluster. Mist provides a new functional programming framework and deployment model f

Deep Learning

2.5k

Distributed Tensorflow, Keras and PyTorch on Apache Spark/Flink & Ray

Distributed TensorFlow, PyTorch, Keras and BigDL on Apache Spark & Ray Analytics Zoo is an open source Big Data AI platform, and includes the followin

Deep Learning

Black-Scholes StarkNet Library

Cairo Black-Scholes Library Black-Scholes library implemented as a Cairo smart contract. All inputs, outputs, and internal calculations use 27-digit f

Deep Learning

Motoko library to help create an append-only logger actor

IC Logger This motoko library provides a module to help create an append-only logger actor. Usage You can use this library with the vessel package man

Deep Learning

LeetCode_Solution

LeetCode_Solution LeetCode: https://leetcode.com/problemset/all/ # Title Difficulty Colab Code 1 Two Sum Easy 2 Add Two Numbers Medium 3 Longest Subst

Deep Learning

OCR-D wrapper for detectron2 based segmentation models

ocrd_detectron2 OCR-D wrapper for detectron2 based segmentation models Introduction Installation Usage OCR-D processor interface ocrd-detectron2-segm

Deep Learning

134

Check for LDAP protections regarding the relay of NTLM authentication

LDAP Relay Scan A tool to check Domain Controllers for LDAP server protections regarding the relay of NTLM authentication. If you're interested in the

Deep Learning

Fuzzware's main repository. Start here to install.

Fuzzware Fuzzware is a project for automated, self-configuring fuzzing of firmware images. The idea of this project is to configure the memory ranges

Deep Learning

Time Weighted Asset Mints

TWAM • Time Weighted Asset Mints A minting harness enabling time-weighted assets to determine minting prices. How it works Requirements: maxMintingAmo

Deep Learning

A complete, self-contained example for training ImageNet at state-of-the-art speed with FF...

ffcv ImageNet Training A minimal, single-file PyTorch ImageNet training script designed for hackability. Run train_imagenet.py to get... ...high accur

Deep Learning

เอกสารประกอบการสอนทุกเนื้อหาในช่องยูทูป KongRuksiam Official

📖 เอกสารประกอบการสอน (PDF) 🔏 Blockchain & Smart Contract Blockchain เบื้องต้น Smart Contract & Solidity เบื้องต้น 💻 Programming เขียนโปรแกรมภาษา Ja

Deep Learning

Omnivore A Single Model for Many Visual Modalities

Omnivore: A Single Model for Many Visual Modalities [paper][website] OMNIVORE is a single vision model for many different visual modalities. It learns

Deep Learning

Python implementation of the Lox language from Robert Nystrom's Crafting Interpreters

pylox Python implementation of the Lox language from Robert Nystrom's Crafting Interpreters. https://craftinginterpreters.com. This only implements th

Deep Learning

ParaGen is a PyTorch deep learning framework for parallel sequence generation.

ParaGen ParaGen is a PyTorch deep learning framework for parallel sequence generation. Apart from sequence generation, ParaGen also enhances various N