As a refinement, since you are satisfied with vesa as a universal driver after the framebuffer phase,
I suggest making the earlier phase just a scan to select a fb appropriate to the video hardware.
I've seen this in Ubuntu 10.04. Seems like one size doesn't fit all in frame buffers in Ubuntu, anyway.
In Ubuntu, one of the options is to suffer thru the framebuffer phase which doesn't last too long;
the subsequent X-session phase proceeds ok, then a brief fb phase on shut-down. I spent a month
on this because the initial Ubuntu 10.04 opted to not include SIS video's fb, which it turns out can
be made to work just as well as what those from what they call 'the big three' video suppliers.
Vintage Apple Macintosh SE M5010 Computer w/ Keyboard & mouse -power up- no OS
$148.52
Vintage Apple IIe Computer A2S2128 (825-1351-A)
$174.99
Vintage Apple Macintosh 512K / Mouse, Keyboard & Extras + Carry Case / Turns On
$260.00
Apple MacIntosh IIcx Vintage Desktop Computer M5650 Bad Power Supply
$119.99
Vintage 1991 Apple Macintosh SE/30 Mobius Grayscale External Video Card RARE
$250.00
Vintage Apple 3.5" Drive Floppy Disk Drive A9M0106 - Untested
$69.99
Vintage 1987 Apple Macintosh SE Radius Full Display External Video Card PARTS
$50.00
Vintage MACINTOSH SE M5011 { Powers On }
$79.99
Vintage Apple A2M2010 Monitor II, Green Phosper CRT
$99.99
Vintage Apple IIe 2e iie Computer A2S2064
$215.00