Skip to content

Databases

RetroArch uses .rdb database format files stored locally by default in folder RetroArch/databases. The .rdb files are compiled from clrmamepro format .dat files stored at the libretro database repository. See the database readme for comprehensive information about the sources and functioning of the repository.

Terminology Note: Game Name

The term Game Name refers to the name displayed within the RetroArch interface and in playlists, not to the filename of the underlying file on the computer or device. Game Name in this document is synonymous with playlist item label, playlist entry, content name, and game title.

Features of RetroArch Database Usage

RetroArch uses the database to provide several automated cataloging functions:

  • Validation. Reject or accept files when using the Import Scanner / Playlist Generator based on whether the ROM checksum (or other key) matches the checksum of a known verified completely intact (aka "properly dumped") file in the database.
  • Game Naming. Assign a definitive and uniform display name for each game in a playlist regardless of filename. RetroArch will look up the name field specified for the file's key in the database.
    • Secondary: Thumbnail Images. Download and display thumbnail images for games based on the uniform name assigned by the database, regardless of filename. (Thumbnails are not directly assigned by the database or by checksum association, but as a secondary effect of databased game name assignment if a matching thumbnail is available on the server. Also see: Flexible Name Matching Algorithm.)
  • Category Search ("Explore"). Allows the user to find/view games that match selected criteria, e.g. by Developer, Release Year, Genre, and other attributes/metadata.
  • Per-Game Information View. Provide an in-app viewable informational screen for each game (Game > Information > Database Entry).

How the Import Scanner Uses the Database

See also: Importing Content and Creating Playlists.

RetroArch's Import Content actions "Directory Scan" and "File Scan" will do the following:

  1. Compute a CRC checksum of the file(s) or scan for the in-game serial number. CRC and serial number are the keys used for matching a game file to the database.
  2. Search for that CRC or serial in the information of the local .rdb files (default location RetroArch/databases). If the key is not found in databases, the file will not be added to a playlist. See Validation & Rejection.
  3. Assign the Game Name (aka display name or playlist item name) that is specified as name within the database entry for the key (CRC/serial). The assigned Game Name will appear in the playlist, instead of the filename.
  4. All other associated metadata collated in the .rdb entry for the given CRC/serial can be viewed in the Information > Database Entry for the game and will be viewable via "Explore".

Validation and Rejection

Validation here refers to checking a file or attribute against a reference, and then accepting or rejecting it based on whether it matches what is specified in the reference data (aka what is "allowed"). RetroArch's "Scan Directory" and "Scan File" automated importers are validation processes, not merely tools for adding all files to a playlist. Part of their function is to reject files, not to import all files. The database is the reference, and the ROM file is the item being validated.

If your file's crc or internal serial data (whichever is the key used for matching, see below) does not exist in the database, the file will be rejected by the automatic scans and will not appear in the playlist.

Bypass validation and rejection. To import your games into a playlist regardless of database matches, or if your files are being rejected by the automatic scan (in other words are not recognized by the database) and you wish to add them to the playlist anyway, use the Manual Scan.

Key Field for Matching

During Playlist / Import Scanning ("Directory Scan" and "Scan File" in menu), RetroArch will identify your files in order to then match your file to a data entry in the database. The key for matching varies by console typical file size (i.e. original media type).

  • CRC checksum for systems with smaller file sizes, e.g. games before the advent of disc-based consoles.
  • Serial Number for larger files like disc-based games, to avoid computing checksums on large files. Found within the ROM file. The serial is not metadata but encoded within the game's binary data, which is scanned (in applicable cases) as a byte array by RetroArch.

Contrary to popular belief, the data used for matching is often the serial number encoded within a disc-game's binary data.

Databases include cryptographic hashes (sha1, etc) for informational purposes to define the item specified, but only CRC checksum (or serial) not hashes are used for matching.

Databases and Thumbnails

Thumbnails are not assigned or retrieved based on checksum, serial, or game database matching. See separate documentation for thumbnail handling and the thumbnail matching algorithm.

Currently there is no automatic process that applies database game name changes/updates to libretro thumbnail repository image filenames. Therefore one of the visible consequences of a Game Name or database problem is the lack of an appropriate thumbnail display in RetroArch whenever the Game Name displayed in the interface doesn't match a repository thumbnail filename.

Troubleshooting

See also: RetroArch documentation for Import Scanning and Playlist Creation/Scanning.

The information below is for Users who are interested in figuring out the cause of a database-related problem and possibly helping to fix the problem in a way that will help all users. RetroArch is a volunteer project, and many problem situations can be improved by interested users with medium technical awareness and no programming skill needed.

Common Problems and Solutions

The most common user problems and solutions related to the database are:

  • Missed files during import scan. I.e. automated Directory Scan or File Scan "misses" some files, meaning the files are not imported and do not appear in the playlist. See Validation & Rejection above.
    • Solution A: Contribute to the database with data for the files/games that are not yet covered by the database.
    • Solution B: Use the Manual Scan option, which will accept all files according to the chosen settings.
  • Game Name error or incorrect information. E.g. A game file receives a wrong title inside the RetroArch playlist/interface.
    • Solution:
    • Follow the investigation steps below to find the .dat file that has the erroneous information, and contribute a correction.
    • Depending on the source of the data, an upstream change within a database group's system may be required, but it is also possible to create ad hoc database coverage on the libretro github.
  • Outdated Local Files. I.e. an error(s) has been fixed in the libretro database but the fix has not yet been downloaded in the user's app install.
    • Solution: Update your RetroArch databases (Main Menu > Online Updater > Update databases). That will apply recent fixes/corrections to your RetroArch install.

Investigating Database Issues

Follow the steps below to find the cause of a database or game/name identification issue:

  • Learn about the factors that might be causing the Problem
    • Understand the multifaceted .dat system and files. Multiple different .dat files may have data for the same game.
    • Understand which key field RetroArch uses for identifying your file and for searching for your file's info in the database.
    • Understand precedence within the dat files in the repository.
  • Verify data "on both sides"
    • Verify your file properties. Verify your game file has the appropriate key ID: compute the crc checksum, or verify the encoded serial number with a hex editor, whichever is applicable.
    • Verify libretro databases on github. Look in the repository databases to find which .dat file might hold incorrect data for the game file at issue. Even if you find a .dat that holds correct data for the game key (CRC or serial) in question, a different dat with precedence may hold incorrect data that is over-ruling the correct data.
  • Verify upstream data to find mismatches or pending corrections. If an upstream database group (No-Intro, Redump, GameTDB, etc) is responsible for the .dat at issue, look on their websites to see whether their current information is correct or incorrect.

Help Fix the Problem

After you've investigated the issue (see above), some possible actions are:

  • Update. Use Main Menu > Online Updater > Update Databases to download new and possibly corrected data from the libretro server.
  • Contribute a correction or addition of data to fix the issue. It may be possible to create an ad hoc database or to make a new entry within an existing ad hoc database.
  • Use the Issue Tracker.
    • Search for existing issues on github that may hold useful advice or solutions for your problem. Adding your new examples or insights to the discussion for the problem may help Members/Contributors create a fix.
    • Open an Issue if a relevant one isn't already open.
    • Open a Database Issue if you observe either of the following:
      • You see a large-scale issue affecting many data entries or entire dats.
      • You found that Upstream Data is correct but libretro or RetroArch doesn't reflect it, and at leat 4 weeks have passed since the Upstream update occurred.
    • Open a RetroArch Issue if: you see a problem with RetroArch's scanning behavior or validation, while the databases appear correct and match your file's properties (crc and serial within the game's binary data viewable with a hex editor).
  • Submit Upstream Changes. Make changes upstream (No-Intro, Redump, GameTDB, etc) by going through the channels of the upstream group responsible for the data at issue if: you found that Upstream Data is Incorrect and has been imported to the libretro database repository. The upstream group must make the correction to "fix it at the source", though it may be possible to create alternative data coverage instead (see below).

How to Contribute to Databases

Like thumbnails and documentation, databases are an area where users who are not programmers can contribute to RetroArch and in a way that benefits all users.

Github Steps Overview

  1. Make an account on github.com and login.
  2. Fork the libretro database repository. Forking means copying the repository into your own working area on github.
  3. Make your changes within your fork. For example, create a new .dat, or add a game entry or correction to an existing dat.
  4. "Commit" (save) your changes with your fork.
  5. Use the "Pull Request" button (or the Contribute > Open Pull Request button) to send and propose your changes to the libretro official team members. They will review your contribution, and in time either accept it or inform you about required changes or a reason why it isn't acceptable.

Small-Scale Corrections

See database repository readme to learn when/where small-scare corrections are appropriate.

Two methods for adding data coverage for a single game or niche of games, via Pull Request proposal on github:

  • Method A: Fix the dat at issue. This is only possible if two conditions are met:
      1. The .dat doesn't originate from an import from upstream and
      1. The .dat won't receive subtractive sync over-writes in the future.
    • If those conditions are not met, the intended "fix" would be deleted by the next bulk import sync.

Or...

  • Method B: Edit a different dat, leaving the erroneous dat intact but moot. This is only advisable when the correction and the error have different keys, or if the edited/corrected database has precedence over the erroneous database. If one of those conditions is not met, then the attempted correction would fail: it would be over-ruled in the .rdb compile by the erroneous dat's information. If one of those conditions is met, you may do one of following:
    • Add a game data entry to an existing and appropriate ad hoc .dat in the repository.
    • Create a new ad hoc .dat. This is often acceptable even for a small number of games because of the multi-faceted nature of the dat system. Some limitations may be enforced by admins, e.g. for the manageability of the build script or the repository.

Large-Scale Additions

See Adding New Database. Contributors are welcome to propose the addition of bulk data from their own build scripts or otherwise, via github Pull Request.