Family mruby OS: A Multi-VM MicroRuby System Built on FreeRTOS

kishima

2025年12月26日 — 11 min read

Today, I’d like to introduce Family mruby, a personal project I’ve been working on recently, and its evolution into Family mruby OS.

(This post is translated from a previous post)

This is a demo running on the current Linux-based simulation environment. Wouldn’t it be exciting if a UI like this could run on an ESP32 microcontroller using mruby, and you could even develop and execute mruby code directly on it?

Background and Motivation

Family mruby is a project that I originally started in 2019.
I also presented an early prototype at RubyKaigi Takeout 2020.

What is Family mruby?

Long ago, BASIC was often the first programming language that children encountered.
Despite its limitations, there were products like Family BASIC, which allowed BASIC programming not only on PCs but also on platforms such as the MSX or the Famicom(NES). Many programmers discovered the joy of programming through these environments.
Today, development environments for most programming languages are freely available and easily installable on PCs. However, because so much is possible, beginners often don’t know where to start. Even reaching the point where you can make something slightly beyond “Hello World,” such as a simple game, can require a surprisingly high setup cost.
With that in mind, I wanted to create an environment where you could build small games or applications using a scripting language on a single microcontroller board. That idea became Family mruby.

Features of the Original Family mruby

Standalone mruby editor and execution environment running directly on ESP32
Designed to run on ESP32 with PSRAM
Custom-designed dedicated hardware board
VGA video output
Analog audio output
PS/2 keyboard input

Dormant Period

Due to the COVID-19 pandemic and the busyness of my personal life, development stopped for a while.
Still, the idea of rebooting the project always lingered in the back of my mind.

Encounter with PicoRuby / MicroRuby

While I was away from the project, I learned that @hasumikin had developed PicoRuby, which greatly inspired me.

In my original implementation, I ported raw mruby to the ESP32 and built a custom editor on top of it. PicoRuby, however, was much more resource-efficient and implemented many features that significantly improved user experience, such as a filesystem and USB device support.

Inspired by this design philosophy, I decided to rebuild my own ideal mruby development and execution environment from scratch.

PicoRuby is based on the mruby/c VM, but I heard that MicroRuby would be based on the standard mruby VM, which meant I could leverage my previous mruby experience.
By 2025, my personal life had stabilized enough to allow me to fully reboot the project and resume development seriously.

Differences Between the Original and the New Family mruby

During the project’s dormant period, I changed jobs and shifted from a PM-focused role back to hands-on coding. With my development instincts refreshed, I decided to design the new system with a “do everything I want” mindset, partly as a skills refresher.

The architecture of the original Family mruby looked like the diagram below.

The new Family mruby architecture is shown in the next diagram.

The system has been reconstructed as an OS, allowing the world of “developing and running software in mruby” to expand significantly.

The main differences are summarized in the table below.
To address pain points from the original version—such as the cumbersome setup for PS/2 keyboards and VGA output—I switched to USB devices and enabled development on Linux as well.

Differences Between the Original and the New Family mruby

Family mruby OS

In the new Family mruby, USB mice can also be connected. Taking advantage of this, I wanted to provide a simple window-based GUI inspired by Windows 3.1 and FM-TOWNS OS, which I personally grew up with.

Since ESP-IDF is built on FreeRTOS, I realized that it should be possible not only to run mruby code but also to execute multiple VMs concurrently.

I decided to call this new environment Family mruby OS.

Assumptions

Compared to PicoRuby and similar systems, Family mruby OS is expected to require several times more memory.
Therefore, it assumes an ESP32 equipped with several megabytes of PSRAM, such as:

In a sense, this might be the “luxurious microcontroller environment” that mruby originally envisioned.
In the future, I’d also like to support the ESP32-P4.

Key Features

Adoption of MicroRuby

By using MicroRuby as the primary VM, we benefit from its compact yet dynamic compilation capabilities, as well as various picoruby-gems.

Since MicroRuby is under active development and may introduce breaking internal changes, it’s important to track upstream carefully rather than maintaining a fork.

I initially considered PicoRuby for its stability, but since mruby/c relies heavily on global variables, it appeared difficult to isolate memory spaces cleanly, so I decided against it.

MicroRuby’s built-in multitasking features remain useful. Combined with FreeRTOS-level concurrency (machine-code level), we can also retain bytecode-level concurrency.
(Currently, instead of using interrupt handlers, each VM’s tick is driven from a dedicated task.)

Multi-VM Architecture

FreeRTOS is a real-time operating system that provides task functionality.
Each task (similar to a thread) has its own stack and can operate independently.

By placing one VM per task and creating multiple tasks, multiple VMs can run concurrently.
To make this possible, memory isolation and hardware resource separation are essential.

To manage VMs like processes, various supporting mechanisms have been implemented.
FreeRTOS Thread Local Storage (TLS) is used to manage VM contexts.

Each VM is assigned its own memory allocator handle, limiting its accessible memory to a predefined pool. This approach:

Prevents system-wide memory fragmentation
Isolates failures so one VM does not corrupt others
Improves overall system stability

In the worst case, a misbehaving application can be terminated, its memory pool reallocated, and restarted.
(Of course, illegal memory access at the C level or hardware resource conflicts are a different matter.)

To avoid busy loops in a resource-constrained environment, event waiting is implemented using FreeRTOS queues.
When inter-VM events arrive, the corresponding callback method is invoked immediately in the target VM.

As long as memory and hardware resources are properly isolated and size constraints are met, languages other than mruby can also be supported.
Currently, Lua is integrated as a proof of concept, and MicroPython is a future candidate.

Hardware Abstraction

With a system of this scale, development directly on a microcontroller can be challenging.
To address this, all hardware- and OS-dependent functionality is accessed through an abstraction layer.

As a result, the same code can now run on Linux as on the ESP32.
This leverages ESP-IDF’s Linux build support and LovyanGFX, which can also run on Linux via SDL2.

A well-defined abstraction layer improves portability and opens the door to future possibilities such as:

Running via WASM
Supporting new board designs

One challenge is that existing picoruby-gems for peripherals may not align perfectly with Family mruby OS’s abstraction model, especially regarding memory management.
Additionally, many are not designed for multi-VM environments, so some form of mutual exclusion will likely be required.

For GPIO and I2C, I think I should rewrite implementations to be thread-safe while using the hardware abstraction layer (HAL).

Development Location

Development is carried out under the following GitHub organization:

Documentation is still sparse, but I plan to improve it gradually.

Execution Environments

Family mruby is designed to run either on dedicated hardware or on Linux.

Dedicated Hardware

There is no mass-produced board yet, but I designed a development board and had it assembled via JLCPCB’s PCBA service.
Unfortunately, it didn’t work as expected, and debugging is currently underway.

The rough block diagram is shown below.
In a somewhat desperate move, I mounted two ESP32 chips on the board. Someday, with an ESP32-P4 and a dedicated display, I might be able to consolidate this into a single chip.

All hardware design data is publicly available:

Here is a photo of the actual development board. The BOM was correct, and assembly quality was excellent.
This was my first time having through-hole components assembled as well, which was a valuable learning experience.

Linux

By using ESP-IDF’s Linux simulation along with SDK2 and LovyanGFX, fmrb-core runs almost unchanged on Linux.

The secondary ESP32 is represented by a separate process.
Instead of SPI, communication is implemented using a custom protocol over sockets.

Current Demo

I recorded a demo video of Family mruby OS. So far I’ve only verified it in the Linux environment, so my next step is to finish debugging the board and then get the same setup running on the ESP32 in the near future.

The OS management features, the wallpaper/system bar screen, and a pseudo-shell app are each running in their own independent MicroRuby VM. Lua is also running separately in its own VM.

Since there’s no launcher yet, I open the pseudo-shell and Lua windows in response to mouse clicks.

The pseudo-shell app doesn’t implement shell functionality yet, but keyboard input is sent to the MicroRuby VM running the pseudo-shell app and reflected on screen.

Mouse interaction is still incomplete as well, so for now I move the shell window using the arrow keys.

demo

Shell-like Demo Application Code

(The shell functionality itself is not yet implemented; this is display-only for now.)

class ShellApp < FmrbApp
  def initialize
    super()
    @current_line = ""
    @history = []  # Line history
    @cursor_x = 0
    @cursor_y = 0
    @char_width = 6
    @char_height = 8
    @prompt = "> "
    @need_redraw = false
  end

  def on_create()
    @gfx.clear(FmrbGfx::WHITE)
    draw_window_frame
    draw_prompt
    @gfx.present
    puts "[ShellApp] on_create called"
  end

  def on_update()
    if @need_redraw
      redraw_screen
      @need_redraw = false
    end
    33 # msec
  end

  def draw_prompt
    # Draw all history lines
    @history.each_with_index do |line, i|
      x = @user_area_x0 + 2
      y = @user_area_y0 + 2 + (i * @char_height)
      @gfx.draw_text(x, y, line, FmrbGfx::BLACK)
    end

    # Draw current input line
    x = @user_area_x0 + 2
    y = @user_area_y0 + 2 + (@history.length * @char_height)
    @gfx.draw_text(x, y, @prompt + @current_line, FmrbGfx::BLACK)
  end

  def redraw_screen
    # Clear user area
    @gfx.fill_rect(@user_area_x0, @user_area_y0,
                    @user_area_width, @user_area_height, FmrbGfx::WHITE)
    draw_window_frame
    draw_prompt
    @gfx.present
  end

  def on_event(ev)
    puts "on_event: shell app"
    p ev

    if ev[:type] == :key_down
      handle_key_input(ev)
    elsif ev[:type] == :key_up
      handle_key_up(ev)
    end
  end

  def handle_key_up(ev)
    keycode = ev[:keycode]

    # Window position control with arrow keys
    x = 0
    y = 0
    case keycode
    when 82  # up
      y = -10
    when 81  # down
      y = 10
    when 80  # left
      x = -10
    when 79  # right
      x = 10
    end

    if x != 0 || y != 0
      set_window_position(@pos_x + x, @pos_y + y)
    end
  end

  def handle_key_input(ev)
    keycode = ev[:keycode]
    puts "[Shell] keycode=#{keycode}"

    # Enter key (keycode 13 = CR)
    if keycode == 13
      handle_enter
      return
    end

    # Backspace key (keycode 8 = BS)
    if keycode == 8
      handle_backspace
      return
    end

    # Ignore arrow keys (79-82)
    if keycode >= 79 && keycode <= 82
      return
    end

    # Convert keycode to character (ASCII printable range)
    if keycode >= 32 && keycode <= 126
      char = keycode.chr
      @current_line += char
      @need_redraw = true
    end
  end

  def handle_enter
    puts "[Shell] Command: #{@current_line}"

    # Add current line to history
    @history << (@prompt + @current_line)

    # TODO: Execute command here and add output to history

    # Clear current line
    @current_line = ""

    # Check if we need to scroll
    max_lines = @user_area_height / @char_height
    if @history.length >= max_lines - 1
      # Remove oldest line to make room
      @history.shift
    end

    @need_redraw = true
  end

  def handle_backspace
    if @current_line.length > 0
      @current_line = @current_line[0...-1]
      @need_redraw = true
    end
  end

  def on_destroy
    puts "[ShellApp] Destroyed"
  end

end

# Create and start the system GUI app instance
puts "[ShellApp] ShellApp.new"
begin
  app = ShellApp.new
  puts "[ShellApp] ShellApp created successfully"
  app.start
rescue => e
  puts "[ShellApp] Exception caught: #{e.class}"
  puts "[ShellApp] Message: #{e.message}"
  puts "[ShellApp] Backtrace:"
  puts e.backtrace.join("\n") if e.backtrace
end
puts "[ShellApp] Script ended"

What Has Been Implemented So Far

OS Framework
- Implementation of basic functionality
- Launching multiple MicroRuby VMs using FreeRTOS tasks
  - During the process of enabling multitasking, issues were found with memory space isolation and tick behavior, so patches have been applied. How these should ultimately be handled still requires further consideration.
- Launching a guest VM (Lua) using FreeRTOS tasks
- Memory pool management
  - The system is designed so that each VM is assigned an independent memory pool. For memory allocation, the default estalloc allocator is used inside MicroRuby, while TLSF is used elsewhere. estalloc has also been configured to operate correctly in a multitasking environment.
- Foundation of an inter-core communication protocol
  Currently implemented using sockets
- Simple drawing API
  RPC implemented on top of LovyanGFX
For Linux
- Execution environment using SDL2 + LovyanGFX
  - Audio is not yet implemented
For real hardware
- Porting the APU block from an NES emulator to the ESP32
- Proof-of-concept for fmrb-audio-graphics
  - Simultaneous operation of LovyanGFX (NTSC) and the APU has been confirmed
Board
- Design and implementation of a prototype board
  - However, issues remain
Applications
- Only development-time drawing samples so far

Future Development Items

OS framework improvementsHID event handling / STDIO replacementWindow management (z-order, movement)Application lifecycle managementFullscreen modeGUI redesign
LinuxExpanded graphics supportAudio implementation
HardwareInter-core communicationUSB host support (keyboard, mouse)Sub-ESP32 firmware
BoardDebuggingMass-production design
ApplicationsShellFile managerGPIO/I2C sample apps

Looking Ahead

My next step is to submit a CFP to RubyKaigi, then aim for a full presentation.
This project is far too large to complete in a few months, so I plan to continue gradual development throughout the coming year.

Once things stabilize, I’d like to sell the board as a set at events.

Dreams

If this project takes shape, I’d like to explore:

Enhanced inter-VM communication
Integrating MicroPython and reusing its ecosystem
Pixel-art and music editors for game development
GUI-based GPIO and I2C control
Wi-Fi connectivity and app sharing/downloading
Ports to LCD-equipped ESP32-S3 or Tab5 devices
A PicoCalc-style device with LCD and mini keyboard
Browser execution via Emscripten, with sensors handled by companion boards

References

Picotron - Fantasy Desktop

I discovered this during development. While it’s not for microcontrollers, its UI closely matches my ideal vision.
It’s a retro, 8-bit-style game development environment.