Home / Ruby / Miscellaneous

A pure Ruby assembler

This is a pure Ruby assembler (I guess). I'm not 100% sure if it counts as pure Ruby because it just reads a JSON file and metaprograms most of it. Anyway, you can use it to write assembly in Ruby, then have it assembled.
Information
Category: Ruby / Miscellaneous
Watchers: 3
Star: 26
Fork: 0
Last update: Feb 27, 2021
Resource links
https://github.com/tenderlove/fisk

Fisk - A Pure Ruby Assembler

This is a pure Ruby assembler (I guess). I'm not 100% sure if it counts as pure Ruby because it just reads a JSON file and metaprograms most of it. Anyway, you can use it to write assembly in Ruby, then have it assembled.

I named it after Wilson Fisk mainly because it reminds me of this project and because I've been playing lots of Spider-Man.

If you want to hack on this project make sure to init the git submodules!

Usage

Here is an example of assembling something:

fisk = Fisk.new

binary = fisk.asm do
  push rbp
  mov rbp, rsp
  int lit(3)
  pop rbp
  ret
end

Though, it's not very fun to assemble something unless you can execute it. So here is an example of how to execute the above assembly. This assembly code will send an interrupt and tell the debugger to stop. So lets write the machine code to some executable memory, and call it from a Ruby program that we will start in lldb.

require "fisk"
require "fisk/helpers"

module Break
  fisk = Fisk.new

  jitbuf = Fisk::Helpers.jitbuffer 4096

  fisk.asm(jitbuf) do
    push rbp
    mov rbp, rsp
    int lit(3)
    pop rbp
    ret
  end

  define_singleton_method :dance!, &jitbuf.to_function([], Fiddle::TYPE_VOID)
end

def deep i = 2
  if i == 0
    Break.dance!
  else
    deep(i - 1)
  end
end

deep

If we launch this script under lldb, the debugger will halt the process when we call the dance! method:

[[email protected] ~/g/fisk (master)]$ lldb ~/git/ruby/ruby -- -I lib fun.rb
error: module importing failed: invalid pathname
(lldb) target create "/Users/aaron/git/ruby/ruby"
procCurrent executable set to '/Users/aaron/git/ruby/ruby' (x86_64).
(lldb) settings set -- target.run-args  "-I" "lib" "fun.rb"
(lldb) process launch
Process 33042 launched: '/Users/aaron/git/ruby/ruby' (x86_64)
Process 33042 stopped
* thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BREAKPOINT (code=EXC_I386_BPT, subcode=0x0)
    frame #0: 0x00000001007f4005
->  0x1007f4005: popq   %rbp
    0x1007f4006: retq   
    0x1007f4007: addb   %al, (%rax)
    0x1007f4009: addb   %al, (%rax)
Target 0: (ruby) stopped.
(lldb) bt
* thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BREAKPOINT (code=EXC_I386_BPT, subcode=0x0)
  * frame #0: 0x00000001007f4005
    frame #1: 0x00007fff2db538e5 libffi.dylib`ffi_call_unix64 + 85
    frame #2: 0x00007fff2db5322a libffi.dylib`ffi_call_int + 692
    frame #3: 0x0000000107458e3d fiddle.bundle`nogvl_ffi_call(ptr=0x00007ffeefbf9bb0) at function.c:204:5
    frame #4: 0x00000001002e704f ruby`rb_nogvl(func=(fiddle.bundle`nogvl_ffi_call at function.c:201), data1=0x00007ffeefbf9bb0, ubf=0x0000000000000000, data2=0x0000000000000000, flags=0) at thread.c:1671:5
    frame #5: 0x00000001002e7570 ruby`rb_thread_call_without_gvl(func=(fiddle.bundle`nogvl_ffi_call at function.c:201), data1=0x00007ffeefbf9bb0, ubf=0x0000000000000000, data2=0x0000000000000000) at thread.c:1787:12
    frame #6: 0x00000001074585a6 fiddle.bundle`function_call(argc=0, argv=0x00000001080280c8, self=0x0000000128f85fd8) at function.c:375:15
    frame #7: 0x0000000100383ca7 ruby`call_cfunc_m1(recv=0x0000000128f85fd8, argc=0, argv=0x00000001080280c8, func=(fiddle.bundle`function_call at function.c:211)) at vm_insnhelper.c:2594:12
    frame #8: 0x000000010037ef99 ruby`vm_call_cfunc_with_frame(ec=0x0000000100b069b0, reg_cfp=0x0000000108127eb0, calling=0x00007ffeefbfa030) at vm_insnhelper.c:2924:11
    frame #9: 0x0000000100377603 ruby`vm_call_cfunc(ec=0x0000000100b069b0, reg_cfp=0x0000000108127eb0, calling=0x00007ffeefbfa030) at vm_insnhelper.c:2945:12
    frame #10: 0x0000000100376f2b ruby`vm_call_method_each_type(ec=0x0000000100b069b0, cfp=0x0000000108127eb0, calling=0x00007ffeefbfa030) at vm_insnhelper.c:3414:16
    frame #11: 0x00000001003769e9 ruby`vm_call_method(ec=0x0000000100b069b0, cfp=0x0000000108127eb0, calling=0x00007ffeefbfa030) at vm_insnhelper.c:3507:20
    frame #12: 0x00000001003578f5 ruby`vm_call_general(ec=0x0000000100b069b0, reg_cfp=0x0000000108127eb0, calling=0x00007ffeefbfa030) at vm_insnhelper.c:3550:12
    frame #13: 0x000000010036d688 ruby`vm_sendish(ec=0x0000000100b069b0, reg_cfp=0x0000000108127eb0, cd=0x0000000100e2be30, block_handler=0x0000000000000000, method_explorer=mexp_search_method) at vm_insnhelper.c:4525:15
    frame #14: 0x000000010033f3de ruby`vm_exec_core(ec=0x0000000100b069b0, initial=0x0000000000000000) at insns.def:789:11
    frame #15: 0x0000000100361d6e ruby`rb_vm_exec(ec=0x0000000100b069b0, mjit_enable_p=true) at vm.c:2162:22
    frame #16: 0x00000001003884c5 ruby`invoke_bmethod(ec=0x0000000100b069b0, iseq=0x000000010781d168, self=0x0000000128f7d180, captured=0x00000001294ef620, me=0x0000000128f85f10, type=0x0000000022220101, opt_pc=0) at vm.c:1292:11
    frame #17: 0x0000000100360321 ruby`rb_vm_invoke_bmethod [inlined] invoke_iseq_block_from_c(ec=0x0000000100b069b0, captured=0x00000001294ef620, self=0x0000000128f7d180, argc=0, argv=0x00007ffeefbfcab0, kw_splat=0, passed_block_handler=0x0000000000000000, cref=0x0000000000000000, is_lambda=1, me=0x0000000128f85f10) at vm.c:1337:9
    frame #18: 0x000000010036017d ruby`rb_vm_invoke_bmethod [inlined] invoke_block_from_c_proc(ec=0x0000000100b069b0, proc=0x00000001294ef620, self=0x0000000128f7d180, argc=0, argv=0x00007ffeefbfcab0, kw_splat=0, passed_block_handler=0x0000000000000000, is_lambda=1, me=0x0000000128f85f10) at vm.c:1434
    frame #19: 0x00000001003600cf ruby`rb_vm_invoke_bmethod(ec=0x0000000100b069b0, proc=0x00000001294ef620, self=0x0000000128f7d180, argc=0, argv=0x00007ffeefbfcab0, kw_splat=0, block_handler=0x0000000000000000, me=0x0000000128f85f10) at vm.c:1470
    frame #20: 0x000000010037f3e7 ruby`vm_call_bmethod_body(ec=0x0000000100b069b0, calling=0x00007ffeefbfcd20, argv=0x00007ffeefbfcab0) at vm_insnhelper.c:2983:11
    frame #21: 0x0000000100377e8f ruby`vm_call_bmethod(ec=0x0000000100b069b0, cfp=0x0000000108127ee8, calling=0x00007ffeefbfcd20) at vm_insnhelper.c:3003:12
    frame #22: 0x00000001003770c9 ruby`vm_call_method_each_type(ec=0x0000000100b069b0, cfp=0x0000000108127ee8, calling=0x00007ffeefbfcd20) at vm_insnhelper.c:3440:16
    frame #23: 0x00000001003769e9 ruby`vm_call_method(ec=0x0000000100b069b0, cfp=0x0000000108127ee8, calling=0x00007ffeefbfcd20) at vm_insnhelper.c:3507:20
    frame #24: 0x00000001003578f5 ruby`vm_call_general(ec=0x0000000100b069b0, reg_cfp=0x0000000108127ee8, calling=0x00007ffeefbfcd20) at vm_insnhelper.c:3550:12
    frame #25: 0x000000010036d688 ruby`vm_sendish(ec=0x0000000100b069b0, reg_cfp=0x0000000108127ee8, cd=0x0000000100e2bcb0, block_handler=0x0000000000000000, method_explorer=mexp_search_method) at vm_insnhelper.c:4525:15
    frame #26: 0x000000010033f3de ruby`vm_exec_core(ec=0x0000000100b069b0, initial=0x0000000000000000) at insns.def:789:11
    frame #27: 0x0000000100361de7 ruby`rb_vm_exec(ec=0x0000000100b069b0, mjit_enable_p=true) at vm.c:2171:22
    frame #28: 0x0000000100363070 ruby`rb_iseq_eval_main(iseq=0x000000010781da28) at vm.c:2419:11
    frame #29: 0x00000001000dbefb ruby`rb_ec_exec_node(ec=0x0000000100b069b0, n=0x000000010781da28) at eval.c:317:2
    frame #30: 0x00000001000dbd83 ruby`ruby_run_node(n=0x000000010781da28) at eval.c:375:30
    frame #31: 0x00000001000036fc ruby`main(argc=4, argv=0x00007ffeefbff5f8) at main.c:47:9
    frame #32: 0x00007fff20530621 libdyld.dylib`start + 1
    frame #33: 0x00007fff20530621 libdyld.dylib`start + 1
0 Open More issues
Closed

Implement performance checking

Add a performance check option to Fisk#write_to; currently, the only check performed is on MOV across the same register.

Verified (also) by enabling the check by default. This currently causes an error in the build due to test_comment using mov rax, rax.

The design is a draft for discussion. It's the least intrusive possible I could think of (there are a lot of considerations, but I'm leaving the PR description simple :)).
A interesting alternative, with a cleaner design, is to add check_performance directly to the given Fisk::Instructions::Instruction instance (via Instruction.new(...) { def check_performance ...}. The problem with this is that instruction metadata classes are autogenerated, and it's sort of ugly to add manual code (but not necessarily unacceptable). An alternative is to have a companion file that patches the given instruction metadata class, but it can't be done because it's frozen.

End of design discussion! :)

opened Oct 20, 2021 by 64kramsystem 6
Closed

Could you make this able to be installed?

I've written a lot of ruby (>15 years ago) but I've never dealt with building gems... (until now... my first attempt was for fisk). Tried to install fisk but ran into issues, as I did not get the gemspec right and the it did not get installed right. (was able to install all the prerequisites, include capstone dev, etc).

I modified the example so that it should work on Linux, but was not able to try it out...

Fisk would be perfect for a proof of concept prototype new language compiler I'm working on...

opened Jun 2, 2021 by bogen85 5
Closed

Add support for 16- and 8-bit registers

Should fix #8, but doesn't quite work yet - see comments inline.

opened Jul 23, 2021 by chrisseaton 4
Closed

Register allocation needs to be done at the control-flow level

I think you need to do register allocation at the control-flow level, not the lexical instruction level.

For example your allocator will allocate this with just two registers, re-using r9 for both reg1 and reg3, but really reg1 is still alive.

reg1 = 1
reg2 = 1
head:
  goto break if reg1 > 100
  reg1 = reg1 + reg2
  reg3 = 1            # your register allocator thinks reg1 is dead at
                      # this point, so shares a register with reg3, but
                      # it's still alive - it's going to be used in the
                      # next iteration!
  reg2 = reg3
  goto head
break:

reg1 = fisk.register
fisk.mov reg1, fisk.imm32(1)
reg2 = fisk.register
fisk.mov reg2, fisk.imm32(1)
fisk.put_label(:head)
fisk.cmp reg1, fisk.imm32(100)
fisk.jg(fisk.label(:break))
fisk.add reg1, reg2
reg3 = fisk.register
fisk.mov reg3, fisk.imm32(1)
fisk.mov reg2, reg3
fisk.jmp(fisk.label(:head))
fisk.put_label(:break)
fisk.assign_registers([Fisk::Registers::R9, Fisk::Registers::R10])     # should fail!
opened Jul 16, 2021 by chrisseaton 4
Closed

Implement size() for Register and Temp

Required when the Fisk caller needs to know the size of the operand; currently, the TenderJIT automatic cfunc stack alignment (see https://github.com/tenderlove/tenderjit/issues/83).

enhancement 
opened Nov 1, 2021 by 64kramsystem 3
Closed

README: Add note about including the Ruby binary debugging symbols

This may be more or less obvious to the project typical audience, but it may save some headaches to people who are not very familiar with the context.

opened Oct 5, 2021 by 64kramsystem 1
Open

Remove redundant workaround of Crabstone

The Crabstone gem used by Fisk is maintained by me, where the memory leak issue mentioned in https://github.com/bnagy/crabstone/issues/10 is already fixed.

Tested bundle exec rake passed.

opened Mar 27, 2022 by david942j 0
Open

Don't use memory that is both writable and executable

The main reason to do this is that it is more secure, as memory that is both writable and executable is a security risk. It is also portable to systems that do not allow memory that is both writable and executable (OpenBSD by default, NetBSD and Linux with security patches).

Initialize memory as read-execute. While writing, mprotect it to read-write, then back to read-execute afterward.

There may be other places that allow memory writing besides write_to and this would break those cases. I didn't audit the code to look for additional spots where you would need to call allow_writes.

Not sure whether this works. The tests don't run on OpenBSD and the example segfaults with or without this change.

I'm marking this as draft, since I'm guessing there are multiple issues with it, assuming it works at all.

opened Nov 9, 2021 by jeremyevans 7
Open

Add support for REP instruction

I'm trying to encode the REP instruction, e.g. rep movsb, which, as far as I know, it's a valid x64 instruction.

I've checked the instructions list, but oddly, it's entirely missing, so I'm not sure if this is intentional.

opened Nov 2, 2021 by 64kramsystem 0
Open

Implement performance optimizer

Super-simple™ implementation of the performance optimizer.

This is a per-instruction optimizer, which is in turn a simplified optimizer in itself. Future extension to multiple-instruction optimization will surely require a more extensive change.

There is a metric ton of design considerations. This PR can be considered a draft, or not :) The most important considerations are:

  • the Fisk::Instruction fields have been exposed; this is necessary, as the optimizer needs to know the instruction, and will need even more, in case it will work on multiple instructions; the alternative OO-design of adding an optimizer on the specific Fisk::Instructions::Instruction is bound to break down
  • the Optimizer could now absorb the performance check (performance checking is now located separately from performance optimization, which is not Vewy Nice); I think it requires a bit of design agreement (I have ideas about what to do), which can be done either in this PR or in a separate one.

Relates to https://github.com/tenderlove/tenderjit/issues/49.

enhancement 
opened Oct 25, 2021 by 64kramsystem 2

Contents

    chrisseaton Rhizome - a JIT for Ruby, implemented in pure Ruby
    devxoul Pure DI in Swift
    artichoke 💎 Artichoke is a Ruby made with Rust
    ronsoros simple virtual machine and assembler
    vladris Minimal 16-bit toy VM and assembler
    StanfordPL x86-64 assembler library
    francisrstokes 💻 A 16-bit virtual machine, including assembly language with 37 instructions, binary assembler, and a step through debugger
    akyoto :running: An x86-64 assembler written in Go.
    EdouardBERGE RASM powerful Z80 assembler
    andrewchambers A mini assembler for fun
    IchMagBier Simple example to show how x64 Assembler and Raylib work together
    uetchy 🍽 Fast, zero-dependency Protocol Buffers binary message parser and assembler
    francisrstokes TypeScript Embedded GameBoy Macro Assembler
    Plutonomicon An untyped Plutus Core assembler
    Panda381 ZX-80 computer clone with ATmega8 processor and with BASIC interpreter written in AVR assembler.
    bwasti C++ WebAssembly assembler
    ryanmelt An easy to install gem version of the Ruby bindings to Qt
    fullstaq-ruby A server-optimized Ruby distribution: less memory, faster, easy to install and security-patch via APT/YUM
    boazsegev A Pure ruby library to merge PDF files, number pages and maybe more...
    libgit2 ruby bindings to libgit2
    jfelchner Ruby/ProgressBar is a text progress bar library for Ruby.
    Share this repo

    Related Repos


    Miscellaneous
    17
    The Budget Planner App
    Rustamxon7 Budget-Planner The Budget Planner App is going to be helpful to track your expenses and transactions. Built With Ruby Ruby on Rails Live Demo Live Dem
     
    Miscellaneous
    12
    Ruby implementation of Wordle
    JonathanWThom Wordle Ruby implementation of Wordle. Installation gem install wordle Usage Just run wordle from the command line. This version will pull a new word a
     
    Miscellaneous
    10
    Tools for validating the AWS service JSON model files.
    amazon-archives AWS Model Validators Provides validators for the various AWS JSON model formats. Basic Usage require 'aws-model-validators' # validating an API model
     
    Miscellaneous
    4.2k
    Export existing AWS resources to Terraform style (tf, tfstate) / No longer actively mainta...
    dtan4 Terraforming Project Status (2021-12-11): No longer actively maintained Terraforming is no longer actively maintained. If you want to generate Terrafo
     
    Miscellaneous
    315
    Easy-to-use pub/sub built on AWS. Ruby.
    iHiD Propono v3.0.0 has been released with new AWS configuration options. Check out the upgrading doc for more information. Thanks to @dougal for this work
     
    Miscellaneous
    189
    Puppet module for managing AWS resources to build out infrastructure
    puppetlabs Table of Contents Overview Description - What the module does and why it is useful Setup Requirements Installing the aws module Usage - Configuration
     
    Miscellaneous
    4
    Use literate Agda in your Jekyll website!
    paolobrasolin jekyll-agda jekyll-agda is a Jekyll plugin which allows you to use literate Agda for your website and take full advantage of its highlighting and hype
     
    Miscellaneous
    3
    A StimulusReflex demonstration of a reactive map UI
    leastbad Maps Demonstrate reactive map navigation, making use of Stimulus Reflex, Cable Ready, Stimulus and Leaflet/Mapbox. The 💡 idea here is that we use the
     
    Miscellaneous
    12
    Lean, non-intrusive authentication for Rails and Sinatra, with out-of-the-box support for ...
    tomas Entrance Clean, adaptable authentication library for Rails and Sinatra. $ gem install entrance Usage # in an intializer, e.g. config/initializers/ent
     
    Miscellaneous
    10
    Yet another TODO application eXtreme edition
    leandronsp yatax Yet another TODO application eXtreme edition Que isso? O objetivo principal do Yatax é, ao longo do desenvolvimento da aplicação, que deve ser a
     
    Miscellaneous
    7
    A starter blueprint for fast tracking Rails development
    dalezak Rails Blueprint A starter blueprint to help fast track setting up a new Rails application. Getting Started To get started, simply clone this repo into
     
    Miscellaneous
    6
    Some helpful Ruby scripts using Octokit to automate tasks in GitHub
    meg-gutshall Octokit Script This is a WIP repo to house some Octokit Ruby scripts I've created that automate actions I regularly need and mass actions in my GitHub
     
    Miscellaneous
    5
    Mobile app build on Rails that keeps track of a registered users' expenses, which can be d...
    ReshyResh Budget Saver Rails application meant to keep track of your spendings. Users can register an account, which will track the categories created and their
     
    Miscellaneous
    213
    DSL and Gem for building ready-to-launch Docker images
    swipely dockly is a gem made to ease the pain of packaging an application. For this gem to be useful, you will want to use Docker for process isolation. Altho
     
    Miscellaneous
    3
    The Recipe app keeps track of all your recipes, ingredients, and inventory
    Donard97 The Recipe app keeps track of all your recipes, ingredients, and inventory. It will allow you to save ingredients, keep track of what you have, create recipes, and generate a shopping list based on what you have and what you are missing from a recipe.