Hacking Up an RGB Framebuffer Driver for Wii-Linux

Update: Take Two


Linux was successfully ported to Wii by the gc-linux team as early as Feb 2008, shortly after the release of the Twilight hack. In the following two years, huge progress and improvements were made to the port and some of the work has already made into mainline Linux kernel.

The fact that Wii resembles its predecessor, GameCube, to such a degree that Wii has often been labeled as ‘not next-gen’, played an important part in making the Wii port possible within such a short time frame, because the gc-linux people had already been working on the GC port of linux for many years. One of the many important similarities between GameCube and Wii is the use of YUY2 colour format in the framebuffer (XFB). There might have been valid reasons for Nintendo to choose such a format for their game consoles, but for the purpose of running Linux on these devices, this choice caused serious inconveniences.

In computer world, colour framebuffers on graphic cards have been predominantly using packed RGB formats for ages. Consequently, computer-oriented OS and software have grown to assume as much about the hardware and generally made no attempt of accommodating the possibility of ‘otherwise’. There certainly were many valid reasons for the choice of RGB for computer graphics, probably way more than there could have been for Nintendo’s choice of YUY2. Unfortunately, this ‘disagreement’ over formats between hardware and software meant more work for the porting guys, as they had to implement converting mechanisms so that the right colour could be displayed on the screen.

Analysis of the ‘Problem’

The way the issue is handled in the current Wii port of Linux is like this:

At the kernel level, the framebuffer driver, called ‘gcnfb’, advertises itself to everything else as only capable of displaying 16bit RGB565 format.This is because RGB565 matches the pixel data size of 2 bytes of YUY2, making it easier to handle resolution-related stuff. In other words, the driver fakes an RGB565 framebuffer and all unknowing programs will try to dump RGB565 data into it. The driver maps the physical framebuffer memory into kernel space and makes it available to other kernel modules such as fbcon (framebuffer console). The mapped framebuffer memory can in turn be mapped into user space for direct reading/writing. RGB565 -> YUY2 conversion is implemented in the driver, not only in a hackish way, but also specifically and exclusively for fbcon’s writes to framebuffer, which ensures correct display, most of the time, in text console mode. While using ‘Up’ arrow to access command line history or using ‘Backspace’ in nano editor, garbled colours are sometimes displayed, as a result of such an imperfect solution.

At the user space level, it’s up to individual programs themselves, that are directly accessing the framebuffer, to make sure that data written to the mapped memory are of the correct format and that data read from the mapped memory are properly converted before being manipulated by RGB-based processing. This means that all of such programs have to be patched in order to work. Programs accessing framebuffer through another program, for instance SDL applications running on top of SDL and X applications running on top of X, do not need to be patched if the underlying framebuffer-interfacing programs already incorporate proper YUY2-RGB565 conversions. In X, a special Xorg framebuffer driver called is available, which handles the format conversion issues, and since all X programs automatically use this driver if it is specified in xorg.conf, they will ‘just work’ if ‘cube’ is configured correctly. The issue with ‘cube’ is that it currently supports only one desktop resolution and makes no attempt to accommodate TV overscans, leaving parts of the display area invisible to some users.

In short, various programs at kernel and user space levels have to be patched directly or indirectly to be able to output correct display. This creates a system that is difficult to install/maintain, and leaves lots of console-mode programs not working because no patches have been developed for them and they are simply dumping RGB565 image data directly into the YUY2 framebuffer. For example, although you can get correct text display in a console mode shell, Debian Installer, which also runs in console mode, will give you screens like this one if run on top of Wii-Linux:

Debian Installer running on top of Wii-Linux

Debian Installer running on top of Wii-Linux

Wouldn’t it be nice if the Wii framebuffer driver could fake an RGB framebuffer more competently and handle all RGB-YUY2 conversions internally and transparently as far as everything else concerned? This way, no patching would be required and everything would ‘just work’ as on a ‘normal’ PC. Let me explain how this could be achieved with some simple hacks.

This entry was posted in Linux, Wii and tagged , , , , , . Bookmark the permalink.

63 Responses to Hacking Up an RGB Framebuffer Driver for Wii-Linux

  1. Gabe says:

    so, I ran depmod -va for verbose mode, and the screen was filled with pages and pages of “dependancies”. ex: “/lib/modules/2.6.32.41/kernel/fs/fuse/cube.ke needs “fuse_abort_conn”; /lib/modules/2.6.32.41/kernel/fs/fuse/fuse.ke.” fuse isn’t the only problem. Running lenny from bootmii.

  2. Gabe says:

    nevermind.

  3. Gabe says:

    and trying to get internal wifi, not external

  4. Gabe says:

    ok, i did a fresh install with lenny, but doing iwconfig shows no wireless extensions.

  5. Gabe says:

    I did all this, and my screen is technicolor (in a bad way) and I have no Bluetooth. any suggestions?

  6. joemegaherz says:

    i just wanted to say thank you for your efforts on the wii. development has been slow and its nice to see someone improving the wii linux expierence. keep it up!!

  7. vulpineRyth says:

    I think I figured out the problem. Depmod was installed but for some reason the folder /lib/modules/2.6.32-isobel-wii was never made, thus causing errors in depmod. I was able to fix it and ran depmod. I got USB mounting to work but still no wlan0.

  8. vulpineRyth says:

    MINI though bootmii. However I’ve only been working with Linux for 2 weeks now

    • farter says:

      Extact downloaded packge to root of SD card.

      In bootmii GUI, select SD card, browse to the ‘ppcboot’ folder, select a file corresponding to your TV signal format.

      • vulpineRyth says:

        Okay I got it to work only to have another problem. Wlan0 is missing now.

      • vulpineRyth says:

        Okay I got it to work only to have another problem. Wlan0 is missing now and I can’t mount USB drives. It gives me a wrong fs type message on /dev/sda1.

        • farter says:

          That’s because you updated the kernel without updating the kernel modules.

          You need to extract the modules archive in the downloaded package into your rootfs (usually second partition of SD), so that you have /lib/modules/2.6.41/…

          After booting into linux, do ‘sudo depmod -a’.

  9. vulpineRyth says:

    How do you exactly put this in? I’m still learning but where do I put the files?

  10. > It is impossible for the kernel driver to keep track of such changes

    No it isn’t, you can use the MMU to tell you when changes were made.

    Look at driver/video/fb_defio.c