Whether to build the FBdev target is autodetected during ./configure. Read the framebuffer documentation in the kernel sources (Documentation/fb/*) for more information.
If your card doesn't support VBE 2.0 standard (older ISA/PCI cards, such as S3 Trio64), only VBE 1.2 (or older?): Well, VESAfb is still available, but you'll have to load SciTech Display Doctor (formerly UniVBE) before booting Linux. Use a DOS boot disk or whatever. And don't forget to register your UniVBE ;))
The FBdev output takes some additional parameters above the others:
specify the framebuffer device to use (default: /dev/fb0)
mode name to use (according to /etc/fb.modes)
config file of modes (default: /etc/fb.modes)
important values, see example.conf
If you want to change to a specific mode, then use
mplayer -vm -fbmode
-vm alone will choose the most suitable mode from /etc/fb.modes. Can be used together with -x and -y options too. The -flip option is supported only if the movie's pixel format matches the video mode's pixel format. Pay attention to the bpp value, fbdev driver tries to use the current, or if you specify the -bpp option, then that.
-zoom option isn't supported (use -vf scale). You can't use 8bpp (or less) modes.
You possibly want to turn the cursor off:
echo -e '\033[?25l'
setterm -cursor off
and the screen saver:
setterm -blank 0
To turn the cursor back on:
echo -e '\033[?25h'
setterm -cursor on
FBdev video mode changing does not work with the VESA framebuffer, and don't ask for it, since it's not an MPlayer limitation.