Hacking Up an RGB Framebuffer Driver for Wii-Linux

Update: Take Two


Linux was successfully ported to Wii by the gc-linux team as early as Feb 2008, shortly after the release of the Twilight hack. In the following two years, huge progress and improvements were made to the port and some of the work has already made into mainline Linux kernel.

The fact that Wii resembles its predecessor, GameCube, to such a degree that Wii has often been labeled as ‘not next-gen’, played an important part in making the Wii port possible within such a short time frame, because the gc-linux people had already been working on the GC port of linux for many years. One of the many important similarities between GameCube and Wii is the use of YUY2 colour format in the framebuffer (XFB). There might have been valid reasons for Nintendo to choose such a format for their game consoles, but for the purpose of running Linux on these devices, this choice caused serious inconveniences.

In computer world, colour framebuffers on graphic cards have been predominantly using packed RGB formats for ages. Consequently, computer-oriented OS and software have grown to assume as much about the hardware and generally made no attempt of accommodating the possibility of ‘otherwise’. There certainly were many valid reasons for the choice of RGB for computer graphics, probably way more than there could have been for Nintendo’s choice of YUY2. Unfortunately, this ‘disagreement’ over formats between hardware and software meant more work for the porting guys, as they had to implement converting mechanisms so that the right colour could be displayed on the screen.

Analysis of the ‘Problem’

The way the issue is handled in the current Wii port of Linux is like this:

At the kernel level, the framebuffer driver, called ‘gcnfb’, advertises itself to everything else as only capable of displaying 16bit RGB565 format.This is because RGB565 matches the pixel data size of 2 bytes of YUY2, making it easier to handle resolution-related stuff. In other words, the driver fakes an RGB565 framebuffer and all unknowing programs will try to dump RGB565 data into it. The driver maps the physical framebuffer memory into kernel space and makes it available to other kernel modules such as fbcon (framebuffer console). The mapped framebuffer memory can in turn be mapped into user space for direct reading/writing. RGB565 -> YUY2 conversion is implemented in the driver, not only in a hackish way, but also specifically and exclusively for fbcon’s writes to framebuffer, which ensures correct display, most of the time, in text console mode. While using ‘Up’ arrow to access command line history or using ‘Backspace’ in nano editor, garbled colours are sometimes displayed, as a result of such an imperfect solution.

At the user space level, it’s up to individual programs themselves, that are directly accessing the framebuffer, to make sure that data written to the mapped memory are of the correct format and that data read from the mapped memory are properly converted before being manipulated by RGB-based processing. This means that all of such programs have to be patched in order to work. Programs accessing framebuffer through another program, for instance SDL applications running on top of SDL and X applications running on top of X, do not need to be patched if the underlying framebuffer-interfacing programs already incorporate proper YUY2-RGB565 conversions. In X, a special Xorg framebuffer driver called is available, which handles the format conversion issues, and since all X programs automatically use this driver if it is specified in xorg.conf, they will ‘just work’ if ‘cube’ is configured correctly. The issue with ‘cube’ is that it currently supports only one desktop resolution and makes no attempt to accommodate TV overscans, leaving parts of the display area invisible to some users.

In short, various programs at kernel and user space levels have to be patched directly or indirectly to be able to output correct display. This creates a system that is difficult to install/maintain, and leaves lots of console-mode programs not working because no patches have been developed for them and they are simply dumping RGB565 image data directly into the YUY2 framebuffer. For example, although you can get correct text display in a console mode shell, Debian Installer, which also runs in console mode, will give you screens like this one if run on top of Wii-Linux:

Debian Installer running on top of Wii-Linux

Debian Installer running on top of Wii-Linux

Wouldn’t it be nice if the Wii framebuffer driver could fake an RGB framebuffer more competently and handle all RGB-YUY2 conversions internally and transparently as far as everything else concerned? This way, no patching would be required and everything would ‘just work’ as on a ‘normal’ PC. Let me explain how this could be achieved with some simple hacks.

This entry was posted in Linux, Wii and tagged , , , , , . Bookmark the permalink.

63 Responses to Hacking Up an RGB Framebuffer Driver for Wii-Linux

  1. -DarkAceZ- says:

    Wow, you must have a solution to everything!
    So anyway, I apparently didn’t extract something right, or something like that because my Wifi doesn’t work anymore. (Since the new kernel)
    I posted something about this somewhere else, but the topic is either dead, or no one knew how to fix it. Here’s the exact post:
    “Oh, shoot. I installed farters mess/kernel/screen and now my wifi isn’t working. I’m missing some files. ifup wlan1 says the same thing: no such device and all of that junk. iwconfig says “lo no wireless extensions”. So I did modprobe b43 and it says:
    [CODE]WARNING: Could not open ‘kernel/drivers/leds/led-class.ko’: No such file or directory
    WARNING: Could not open ‘kernel/net/wireless/cfg80211.ko’: No such file or directory
    WARNING: Could not open ‘kernel/net/mac80211/mac80211.ko’: No such file or directory
    FATAL: Could not open ‘kernel/drivers/net/wireless/b43/b43.ko’: No such file or directory[/CODE]
    Heh heh, I think I might not have extracted something right here…”
    What do you think I need to do?

  2. -DarkAceZ- says:

    Maybe I should overclock it… JK!
    I noticed that I can’t fullscreen Supertux anymore… It says something like “[FATAL] /home/[USERNAME]/src/supertux/branches/supertux/0_3_x/src/main.cpp:599 Unexpected exception: Couldn’t set video mode (800×600-0bpp): No video mode large enough for 800×600″. Is there any way to fit the entire game on my screen? It used to work before I used your kernel… On window mode it goes off-screen.

    • farter says:

      Never used supertux, but i’m guessing that because the screen size is now set to smaller than 640×480 (cube driver uses this resolution), it can not find a appropriate resolution for full screen mode.

      You may try disabling ‘TV-safe’ by changing the ‘nostalgic’ option to gcnfb driver in kernel command line.

  3. -DarkAceZ- says:

    Okay, I’m lost at editing the xorg.conf file… I didn’t do anything with it yet, but (I think I) did everything else. Now my entire screen is inverted. What do I need to do?

    • farter says:

      If you haven’t changed Xorg.conf, you are still using cube driver in X, which is not compatible with the modified kernel posted here, so you are seeing incorrect displays.

      I seem to remember I put the needed xorg.conf in the download. Anyway, you basically need to ‘apt-get install xserver-xorg-video-fbdev’ and change xorg.conf to use ‘fbdev’ driver.

      • -DarkAceZ- says:

        Wow! Thanks for helping a noob out man! I have a really nice screen now! Awesome!
        I was wondering though, are those black edges ever going to get used? (On the left and right there is unused screen area.)
        Also, what DE are you using? I’m trying to play avi files, but VLC Media Player won’t do anything, and Totem only once played one (And it was frame skipping a lot), otherwise it’ll just play the audio. And the whole desktop is EXTREMELY slow when playing videos. (MP3s seem to play without much trouble at all.)

        • farter says:

          The kernel driver is configured by default to use only ‘TV-safe’ area of the screen, hence the black borders. If you disable this feature, you may end up with a screen larger than your display with the borders invisible.

          I only ever used LXDE. Don’t expect to play modern day stream video in DE, the cpu is already too stretched as it is.

  4. -DarkAceZ- says:

    OK, and how does Gnome work?
    “Gnome Display Manager Login on Wii”
    Is Gnome fast on here? Or didn’t you try it?

    • farter says:

      Hi, gdm is just a display/login manager, not the whole Gnome desktop environment. I haven’t tried Gnome, but I suspect it will much slower than LXDE etc.

      • -DarkAceZ- says:

        Okay, I think Gnome probably would be pretty slow, I thought you had actually got it on WiiLinux. I was just wondering because I’m used to Gnome.
        Well I think I’ll try this now. I just need to download the corresponding elf/kernel and edit the xorg.conf file, right?
        (I’m running at the moment:
        debian lenny 5 ppc (kernel 2.3.32)
        the bootloader (elf) is MiNI mikep5 )

  5. -DarkAceZ- says:

    Agreed with joemegaherz!!
    I have a question though… Is there an interface/GUI that’s faster than xfce4? I might be expecting too much for the Wii though… (I haven’t installed any of your kernels yet)

  6. Coto says:

    Farter, hi again, just wanted to say, thank you for the effort. I am a C & PHP (although pseudo languages are rather easy) developer too, and since i’ve managed to run this experimental driver I’ll be an active linux user… (see, develop something and don’t know how to use linux is just wrong..)

    If you ever need something, contact me at my mail adress!

    Cheers

    • farter says:

      Although Wii is too underpowered by today’s standards to fully expose the power of linux, it’s still a good start. Wish you all the wonderful experiences! :)

  7. Coto says:

    Ok, figured it out…

    Since I use X as a desktop environment, on lenny had cube driver enabled by default.

    Changing lines in
    Section “Device”
    Identifier “Nintendo Wii Gpu Card” //Or something like that
    “Driver” “cube”
    End Section

    into

    Section “Device”
    Identifier “Configured Video Device”
    Driver “fbdev”
    Option “UseFBDev” “true”
    EndSection

    in /etc/X11/xorg.conf

    Now I have a beautiful screenie! Thank you man, 8)

  8. Coto says:

    Ok, like Gabe I’m having troubles seeing my screen in a old XGA mode (horribly technicolor screen) I use component cables, ran through bootmii (elf arm) your updated booter (mikep5(vfb)-zImage-2.6.32.41.mini.480p(NTSC).elf) (I use a NTSC wii) and the kernel you provided was successfully installed. Through modprobe b43 I am able to initialize wireless, bluethoot as well, but the screen just looks wrong. What’s wrong?

    even if “sudo depmod -a” (I have super user access…)

  9. Jonimoose says:

    Do you have a git repo with your work anywhere? I have made up a few minor patches as well, one of which actually maps the sensorbar and slot led to led devices as well as rebased on later 2.6.32.y kernels but if you are also doing this I see no reason for us to be doing duplicate work such as the rebasing.

    My led patch is here http://jonimoose.net/archii/wii_led.patch if you are interested.

    • farter says:

      Hi, interesting work. I am not working on anything other than the framebuffer, so there is no risk of your work being duplicated by me.

      I don’t have publit git either. I am more familiar with hg, :) .

  10. Ben says:

    Has anyone tried compiling Linux 3.0 mainline for the wii? Is there any reason someone couldn’t do a newer Debian distro also?