How to Fully and Correctly Use FileMaker’s Recovery Tool

File corruption. It happens. It’s part of software, and sometimes no matter how hard we try to prevent it, it still happens. This guide will detail out how to determine if your FileMaker file is corrupted, what to next, and how to prevent it from corrupting again in the future.

How do I know if my file might be corrupted?

You may suspect your file is corrupted if you are experiencing and of the following:

• Repeated crashing of the FileMaker file.
• Unexpected or unexplained behavior (for example an odd behavior will start happening once in a while and the only way to fix it is to restart FileMaker).
• Any time you shut down your server machine without properly closing down the files in FileMaker Server. If that ever happens, you should verify there is no corruption. For example, if there was a power outage.
• If your OS level backup software on your server (i.e. Time Machine, Crashplan, etc.) is backing up the hosted FileMaker files, they could get corrupted. You should always back up your backup copies, not the hosted files located inside the  FileMaker Server > Data > Databases.
• If you are working in Manage Database on a hosted file and you lose connectivity to the file while Manage Database is open, your file may be corrupted.
• If the file size increases dramatically from one backup to the next, without a large data increase to explain it, it’s possible that corruption is the cause.
• Generally, if something seems fishy, it’s worth ruling out corruption!

How can I verify if my file is corrupted?

FileMaker has a Recovery feature that allows you to determine if your file is safe to use or not. Here is a step by step guide to running a recovery process:

• Grab a recent backup of the file you think may be corrupted.  Ideally, move this file to a non-server machine (it’s best not to take up resources on the server machine)
• Make a compacted copy or clone of the backup. A clone is nice because it will make the recovery faster, but it will not tell you if the corruption is in your data and not your schema.
• Open FileMaker, and from the menu select File > Recover…
• Select the compacted copy/clone and run the recovery (you can optionally check the consistency of the file first, but no matter the result of the consistency check, you should continue with the recovery process)
• You will be asked if you want to rename the old file and select a location of the new recovered file
• Once the recovery is complete, FileMaker will alert you if there were any issues found. If there were issues, your file is corrupted.

My file is corrupted. Now what?

Option 1: Revert to a Backup

• Gather your recent backups and run them through the recovery process until you find the most recent backup that passes the recovery check
• Create a clone of the good backup
• Create a compacted copy of the clone

Note: Be careful, if there has been any code or layout or schema changes in production since the last good backup, you will need to replicate those changes.

• Perform a data import into this new file with most recent data
• Host your updated file as a replacement of your corrupted file

Option 2: Perform Surgery

Before going into these steps, I want to really stress that you need to be saving copies of the file as you go along! This is really important! At every step of the way, you make a copy. You don’t want to start the surgery only to half way through discovering that you will have to redo all the changes you made for whatever reason, or even worse, you don’t have an original copy anymore because you have been making changes to the original. I recommend creating a new folder for each stage you are at and have the folder name be a very specific description of where you are in the surgery.

For example your folders may look like: Original, CF_Removed, CF_And_Relationship_Removed, and so on.

• Open the log. When you perform a recovery, a log file gets generated. You can look through this file to get a little more detail on what has failed.
• The 2nd column is an error code column. For most of the rows, this will be a “0”. Scroll until you find a non-zero number; this is where your corruption is. There may be more than one location.
• Look for the location of the error, and determine if you can “perform surgery” to take it out. For example, if the log determines it’s a specific layout, try deleting all the contents of that layout and running the recovery again (making backup copies of each step along the way!). If it is a script, try deleting the contents of that script. If it’s a custom function, delete the contents of the custom function, etc. I recommend not deleting the script, layout, custom function, etc. in its entirety (at first) because you will then lose the references throughout your file. If the recovery still fails after deleting the contents, and the log says it’s failing at that layout/script/custom function/etc, then I think it’s appropriate to delete the item and run the recovery again.

• Repeat the surgery until you’ve removed all failed items and your file passes a recovery check.
• You will then have to rebuild those items you have modified or deleted. Ideally, do not cut and past anything; rebuild from scratch. You don’t want to risk re-introducing corruption. You may want to copy the corrupt objects into myFMButler’s Clip Manager in order to inspect and repair the XML, which you can then past back into your file.
• If you think it’s the data that might be corrupted, try running recovery on a clone and see if it passes. If you determine the data is corrupted, export the data as .merge files, but rename those files to have a .csv suffix. This should take care of the bad data. Then import that data back into a clone of your file.
• If the surgery fails, you will have to look into other options, such as option 1 above: Revert to a Backup.

What Can I Do to Prevent Corruption?

There’s not a whole lot you can do to prevent corruption, but there are things you can do to minimize the damage:

• If possible, get a backup generator for your server machine in case there’s a power outage. This will prevent unexpected shutdowns of your server machine without proper closing down of the files
• Make sure your OS-level backup software on your server (i.e. Time Machine, Crashplan, etc.) is not backing up the hosted FileMaker files. You should always be backing up your FileMaker backup files, not your hosted files.
• If you are working in Manage Database on a hosted file, try to be as quick as possible. Don’t leave Manage Database open while you go get a cup of coffee. You should be in and out really fast.
• Backup your files as frequently as you can (within reason) and try to keep your backups for as long as you can. When corruption happens, the more backup files you have to work with, the better the options you will have.
• If you have files that seem to get corrupted often, check them on a regular basis. Put a reminder in your calendar if you need to. Doesn’t hurt to be proactive!
• Compact the file on a regular basis or whenever you get the chance. The smaller the file, the fewer blocks available for corruption.

My Experience with Corruption

I have had some interesting corruption causes; I’m sure some of you can relate. Here is a list of issues I have determined to cause corruption in the files I worked on, along with what some of my colleagues have experienced:

• Corrupted text/image on a layout (deleting this one object on all my layouts allowed the file to pass the recovery)
• Corrupted Fonts on a Mac (going to Font Book > Restore Standard Fonts… fixed this)
• Unexplainable constant crashing on a specific layout. I was creating a new report, and whenever I went into browse mode, the table assignment for that layout would disappear and FileMaker would get very slow and eventually crash. When comparing backups we realized this file also jumped drastically in file size, and it wasn’t due to a large record volume increase or container usage increase. Also this file was passing the recovery! I ended up reverting from a backup (before the large file size increase) and it has been working fine since.
• I have had a file fail recovery because of an uninstalled plugin when I ran the recovery. I ran the recovery on the same file with a machine that had the plugin installed and it passed.

If you have any interesting recovery stories or tips I would love to hear them!