MAME 2016 Developer Docs¶
As of 2019, this libretro port of MAME (the libretro "OSD" in MAME parlance) is known to be rather messy. The plan is to rewrite it eventually.
What follows here is a technical explanation of what libretro had to do to MAME's GENie build system in order to compile it as a shared library with the appropriate API exported to be used as a libretro core.
The problem with a static library MAME OSD¶
You're not supposed to put the public API of a shared library into a static library, particularly if those symbols are "unused" by any other part of the shared library. In other words, while the libretro API is contained within the retro OSD, the file containing the public API (libretro.c) must not be stripped by the helpful linker trying to optimize for you.
How you do that varies quite a bit by linker, and some of them such as GNU ld require the options be in a specific order. How to do that with GENie is not clear.
To solve all of these issues at once, we leave libretro.c out of the OSD library and add it to the main MAME library directly. This requires adding an includedirs block for the directory containing libco's header and a files block containing libretro.c. These changes are near the bottom of the GENie project in scripts/src/main.lua.
This is not a clean solution. It just happens to be the one that works across platforms without jumping through hoops for every linker that comes along.
Other libretro GENie mods¶
As mentioned above, near the bottom of scripts/src/main.lua, there's a chunk of libretro-related additions and overrides. You can spot it by searching for "BEGIN libretro overrides to MAME's GENie build". Most of what's there is pretty self-explanatory. It overrides MAME to build a shared library and includes processing for ARCH and platform, the two standard libretro build target variables, and includes the libretro.c file mentioned above. It could probably be split into its own file, but hasn't been thus far for simplicity.
The only other addition is scripts/src/osd/retro.lua and its cfg file which isn't strictly necessary to be a separate file. Pretty standard stuff in there. The real work is in main.lua, and it's all contingent upon use of the libretro OSD to be as minimally invasive into MAME as possible.
That's about it really. Compared to the old system, GENie is both much simpler and more complicated for us. More complicated because stuff is in so many different places. Simpler because modifying GENie rules like we have done makes for an incredibly shallow fork.
Makefile.libretro¶
There is still a pure Makefile named Makefile.libretro. It guesses the ARCH and platform variables if they're not defined, and it passes those (renaming ARCH in the process) along with the OSD and several other useful arguments to GENie. It's basically a major shortcut for the make command line which for our needs is pretty long.
It's been tested building the "arcade" (used to be UME) and tiny targets, just pass SUBTARGET=tiny to Makefile.libretro to get the latter.
Final thoughts¶
MAME is a beast. It compiles to a 137MB .dylib on the Mac and a 172MB .so on Linux. Nonetheless, if its retro OSD weren't such a mess, it would be a good example of how to port something to libretro. Presently it still requires a fork, but the fork is perhaps the shallowest libretro fork we've got. And while not every decision the MAME developers have ever made could be considered optimal, the one thing they have done is insist that every external dependency they have is included in their source tree -- which is exactly what we need.
MAME's GENie setup is complex, but that is because MAME is complex. GENie isn't difficult to figure out if you know a bit of lua. Indeed iKarith does not claim to fully understand GENie, but managed to modify the project to build a libretro core with only a few pointers (thanks balrog!) and a little grepping of 3rdparty/genie.