Hacking Up an RGB Framebuffer Driver for Wii-Linux

Update: Take Two


Linux was successfully ported to Wii by the gc-linux team as early as Feb 2008, shortly after the release of the Twilight hack. In the following two years, huge progress and improvements were made to the port and some of the work has already made into mainline Linux kernel.

The fact that Wii resembles its predecessor, GameCube, to such a degree that Wii has often been labeled as ‘not next-gen’, played an important part in making the Wii port possible within such a short time frame, because the gc-linux people had already been working on the GC port of linux for many years. One of the many important similarities between GameCube and Wii is the use of YUY2 colour format in the framebuffer (XFB). There might have been valid reasons for Nintendo to choose such a format for their game consoles, but for the purpose of running Linux on these devices, this choice caused serious inconveniences.

In computer world, colour framebuffers on graphic cards have been predominantly using packed RGB formats for ages. Consequently, computer-oriented OS and software have grown to assume as much about the hardware and generally made no attempt of accommodating the possibility of ‘otherwise’. There certainly were many valid reasons for the choice of RGB for computer graphics, probably way more than there could have been for Nintendo’s choice of YUY2. Unfortunately, this ‘disagreement’ over formats between hardware and software meant more work for the porting guys, as they had to implement converting mechanisms so that the right colour could be displayed on the screen.

Analysis of the ‘Problem’

The way the issue is handled in the current Wii port of Linux is like this:

At the kernel level, the framebuffer driver, called ‘gcnfb’, advertises itself to everything else as only capable of displaying 16bit RGB565 format.This is because RGB565 matches the pixel data size of 2 bytes of YUY2, making it easier to handle resolution-related stuff. In other words, the driver fakes an RGB565 framebuffer and all unknowing programs will try to dump RGB565 data into it. The driver maps the physical framebuffer memory into kernel space and makes it available to other kernel modules such as fbcon (framebuffer console). The mapped framebuffer memory can in turn be mapped into user space for direct reading/writing. RGB565 -> YUY2 conversion is implemented in the driver, not only in a hackish way, but also specifically and exclusively for fbcon’s writes to framebuffer, which ensures correct display, most of the time, in text console mode. While using ‘Up’ arrow to access command line history or using ‘Backspace’ in nano editor, garbled colours are sometimes displayed, as a result of such an imperfect solution.

At the user space level, it’s up to individual programs themselves, that are directly accessing the framebuffer, to make sure that data written to the mapped memory are of the correct format and that data read from the mapped memory are properly converted before being manipulated by RGB-based processing. This means that all of such programs have to be patched in order to work. Programs accessing framebuffer through another program, for instance SDL applications running on top of SDL and X applications running on top of X, do not need to be patched if the underlying framebuffer-interfacing programs already incorporate proper YUY2-RGB565 conversions. In X, a special Xorg framebuffer driver called is available, which handles the format conversion issues, and since all X programs automatically use this driver if it is specified in xorg.conf, they will ‘just work’ if ‘cube’ is configured correctly. The issue with ‘cube’ is that it currently supports only one desktop resolution and makes no attempt to accommodate TV overscans, leaving parts of the display area invisible to some users.

In short, various programs at kernel and user space levels have to be patched directly or indirectly to be able to output correct display. This creates a system that is difficult to install/maintain, and leaves lots of console-mode programs not working because no patches have been developed for them and they are simply dumping RGB565 image data directly into the YUY2 framebuffer. For example, although you can get correct text display in a console mode shell, Debian Installer, which also runs in console mode, will give you screens like this one if run on top of Wii-Linux:

Debian Installer running on top of Wii-Linux

Debian Installer running on top of Wii-Linux

Wouldn’t it be nice if the Wii framebuffer driver could fake an RGB framebuffer more competently and handle all RGB-YUY2 conversions internally and transparently as far as everything else concerned? This way, no patching would be required and everything would ‘just work’ as on a ‘normal’ PC. Let me explain how this could be achieved with some simple hacks.

This entry was posted in Linux, Wii and tagged , , , , , . Bookmark the permalink.

63 Responses to Hacking Up an RGB Framebuffer Driver for Wii-Linux

  1. fishears says:

    Me again :)
    Do you happen to have a copy of the BFS patch you used? Con Kolivas isn’t hosting anything pre 3.0 now. I’ve looked for one myself but they all seem to rely on different source files than the ones I have (and the ones on kernel.org – which is weird).
    Thanks if have. No worries if you don’t.

  2. fishears says:

    Not sure if you still read these but I’ve just got back into Wii Linux for writing fiction on. I’ve got your 480p kernel running with fbdev and its fantastic having no problems with colour and also having the screen fit my TV. I got b43 loading (add to /etc/modules) and now all that’s missing is swap on RVL-MEM2. I was using it just fine on the previous mikep5 2.6.32 kernel but now its gone. I’ve set up a swap file for now (which is ok but slow). Can you tell me if MEM2 is available as a device in your kernel and if so, how to get at it (it doesn’t list in /proc/devices)?
    THANK YOU

  3. DeltaLink says:

    Which driver would be best for console (TTY) use with a framebuffer (with no X) in your opinion? I’m looking for a driver with YUV colour that requires the least CPU for use with a resource heavy GUI console application. Do any of the drivers allow for resolution or bit depth reduction by default or with little modification?

    • DeltaLink says:

      I mean YUY2 colour space not YUV.

    • farter says:

      Hi. Both drivers are actually quite similar, the real difference being that the unmodified wii linux driver only hardcodes color conversion for text console, while the modified RGB driver converts everything. Neither supports bit depth or resolution change, but you can choose to use centered small area for display.

      The fastest approache IMO is to make your app directly output YUY2 images onto kernel framebuffer. If you need resolution or bit depth reduction, do it in your app too. This is how GeeXboX for Wii output images.

      Alternatively, you could use devkitppc to develop Wii mode app. I’m not sure, but devkit apps seem to be able to harness some GPU capabilities.

  4. -DarkAceZ- says:

    OK, thanks! I had already extracted the modules.zip, but forgot to run depmod.
    Is there anyway to make it automaticly do modprobe b43 on bot? I noticed that it trys to start an internet connection, but because the hardware isn’t started it can’t do anything.