As a refinement, since you are satisfied with vesa as a universal driver after the framebuffer phase,
I suggest making the earlier phase just a scan to select a fb appropriate to the video hardware.
I've seen this in Ubuntu 10.04. Seems like one size doesn't fit all in frame buffers in Ubuntu, anyway.
In Ubuntu, one of the options is to suffer thru the framebuffer phase which doesn't last too long;
the subsequent X-session phase proceeds ok, then a brief fb phase on shut-down. I spent a month
on this because the initial Ubuntu 10.04 opted to not include SIS video's fb, which it turns out can
be made to work just as well as what those from what they call 'the big three' video suppliers.
DELL PowerEdge R430 8SFF 2x E5-2680v4 2.4GHz =28 Cores 32GB H730 4xRJ45
$226.00
Dell PowerEdge R630 Server 2x E5-2680 V4 = 28 Cores S130 32GB RAM NEW 480GB SSD
$197.99
Dell Poweredge R630 Server 2x E5-2620 V4 =16 Cores | S130 | 32GB RAM | 2x trays
$159.99
Cisco UCS C240 M4 2U Server w/ 2x E5-2680v4, NO RAM
$49.99
Dell PowerEdge R630 Server 2x E5-2640v3 2.60Ghz 16-Core 64GB H330
$182.65
Lenovo ThinkServer TS140 Xeon E3-1225v3 3.20GHz 16GB RAM Tower Server Needs HDD
$99.99
Dell PowerEdge R730 16-Bay 2.5” 2xE5-2695 v3 2.3Ghz 64GB iDRAC8 Ent H730 2x750W
$250.00
Dell PowerEdge R720xd 26HDD 300gb 2.5-inch E5-2697 X 2CPU 384RAM 7.2 Tb HDD 
$180.00
Dell Poweredge R730xd LFF 14-Bay 2U Server | Choose Your CPU & RAM Config
$489.99
1U Supermicro Server 10 Bay 2x Intel Xeon 3.3Ghz 8C 128GB RAM 240GB SSD 2x 10GBE
$259.00