FileMaker 21 has been released. Among the new features are a collection of enhancements to the Save as XML function.
Recap of Save as XML
First, let’s have a quick recap in case you’re not yet familiar with the Save as XML. Claris introduced this functionality back in version 18. It saves (almost) all of the code (table definitions, fields, layouts, scripts, etc.) in an XML format. I say ‘almost’ because Claris took the approach of purposely shipping this new functionality before it was fully baked so that they could start collecting and incorporating feedback into future versions. And indeed, every new FileMaker version since then has made improvements to the XML format.
Save as XML (or “SaXML”) is similar to the DDR XML that FileMaker developers are familiar with. Claris wanted to have a format they could use to support new features oriented around deserializing XML into FileMaker files. For example, add-ons use SaXML-style XML to add code to a FileMaker file. The DDR XML was never intended to be used in that way.
SaXML Use Cases
Why bother creating an XML representation of a FileMaker file?
Analysis
The DDR XML has been (and still is, for the time being) used for analyzing a database file. For example, if you want to change a calculation field, you need to know all the places that reference that field so that you can consider how the intended change will affect those places. The goal for the new SaXML format is to eventually include all of the information present in the DDR XML (plus additional information that the DDR XML doesn’t have). It’s not quite there yet, but it’s getting closer with every new FileMaker version.
Add-ons
In addition to the analysis use case, the SaXML format is also being used to create and install add-ons.
Last Modified
Among other information, SaXML includes a UUID child element for many of the building blocks of FileMaker files. This UUID element contains ‘last modified’ information. Here’s an example:
<UUID
modifications="2"
userName="Mislav"
accountName="mkos"
timestamp="2024-04-29T16:58:08"
>
E3292C4A-8C1E-46E2-B01E-429136E2D4A7
</UUID> This makes it possible to see who made what change and when.
Proof+Geist has made a ModLog tool freely available that does just this.
A quick side note: the UUID value is currently not guaranteed to be unique. One way in which the same UUID can be used in different files is if you duplicate a file and then rename it so that it can become part of your overall solution. The FileMaker community is working with Claris to resolve this problem.
Other Use Cases
There are other use cases we can imagine. These are still being fleshed out by Claris and members of the FileMaker community.
- Source control
- Diffing – identifying differences between two versions of a file
- Patching – applying small fixes to a file – see Custom App Upgrade Tool) (currently in developer preview; i.e., not ready for production use)
- Deserializing SaXML into a FileMaker file – reversing the SaXML process by converting an XML file into a fully functional FileMaker file
Generating SaXML
The XML can be created manually (from the Tools menu) or programmatically (either using a script step or the FileMaker Developer Tool CLI).
The script step is supported on Server, so you can create schedules to create SaXMLs on a regular basis or use Perform Script on Server to run the command ad hoc. You can specify a window name to have the script step target a file other than the current file. But beware that if the window isn’t found, FileMaker will quietly target the current file. In other words, you might think you’re creating XML for FileB, but you’re actually creating it for FileA.
The script step operates on an open file (hosted on FileMaker Server or opened locally). If you want to offload the work onto some other computing resource, or if you don’t want to tie up the hosted FileMaker files, you can target closed files (e.g., backups) or even clones using the FileMaker Developer Tool.
Security Warning
As is the case with DDR XMLs, SaXML stores your code in plaintext. If you store any secrets (for example, API keys) in your scripts, custom functions, etc., those will be included in the XML.
New Enhancements in FileMaker 21
Now that we’ve reviewed what SaXML is all about, let’s go over the changes introduced in FileMaker 21.
Split “ValueListCatalog”
It’s useful to think of changes made to a FileMaker file as grouped into categories. Are we making changes to scripts, layouts, or fields? These categories (scripts, layouts, fields, etc.) are generally referred to as catalogs in the SaXML. For example, layout definitions are defined in a “LayoutCatalog” element.
Value lists are tracked in their own catalog. This was true pre-21, but the way they are tracked has changed.
When deploying changes to a file, whether doing this manually or programmatically (using the App Upgrade Tool), the changes need to be introduced in a particular sequence. For example, if you configure a field with validation that references a value list, the value list must be created before the validation is set up.
But it’s also a good idea to group the changes and then deploy the groups in a particular sequence. For example, deploy all value list changes, then deploy all field changes.
This introduces a problem. Consider the following change set:
- Create “Company::CompanyName” field.
- Create “CompanyName” value list based on the “Company::CompanyName” field.
- Create “Contact::CompanyName” field and set up validation that references the value list.
If we deploy value list changes before field changes, the value list will be broken, since “Company::CompanyName” field won’t exist yet.
FileMaker 21 splits the ValueListCatalog into ValueListCatalog and OptionsForValueLists. Now, the change set can be described in the XML as follows:
- Deploy ValueListCatalog changes:
- Create “CompanyName” value list stub.
- Deploy FieldCatalog changes:
- Create “Company::CompanyName” field.
- Create “Contact::CompanyName” field and set up validation that references the value list.
- Deploy OptionsForValueLists changes:
- Update “CompanyName” value list so it’s based on the “Company::CompanyName” field.
Custom Icon Images
Prior to FileMaker 21, if the same custom icon (for example, a custom button glyph – see screenshot) was used multiple times, duplicate copies of the base64-encoded icon image were saved, one for each instance where it was used. This unnecessarily bloated the SaXML file size.
The following “IconData” element was repeated every time that icon was used:
<IconData type="1" size="30">
<BinaryData>
<LibraryReference id="2" key="D3CD5C865EB41FEE71B094F413009974"></LibraryReference>
<StreamList>
<Stream name="MAIN" type="id" size="4">PNGf</Stream>
<Stream name="SIZE" type="Hex" size="4">00580028</Stream>
<Stream name="PNGf" type="Base64" size="6547">LARGE_BASE64_ENCODED_DATA_REMOVED</Stream>
<Stream name="GLPH" type="Hex" size="1">01</Stream>
<Stream name="FNAM" type="Hex" size="28">000000010533373B3D3F00102E3F292E7733373B3D3F776B742A343D</Stream>
<Stream name="DPI_" type="Hex" size="4">00900090</Stream>
</StreamList>
</BinaryData>
</IconData>With FileMaker 21, the base64-encoded data is saved just once.
Now, each “IconData” instance looks like this:
<IconData type="1" size="30">
<BinaryData>
<LibraryReference id="2" key="D3CD5C865EB41FEE71B094F413009974"></LibraryReference>
</BinaryData>
</IconData>And the icon data is stored inside of a LibraryCatalog element.
<LibraryCatalog membercount="1">
<BinaryData>
<LibraryReference id="2" key="D3CD5C865EB41FEE71B094F413009974"></LibraryReference>
<StreamList>
<Stream name="MAIN" type="id">PNGf</Stream>
<Stream name="SIZE" type="Hex" size="4">00580028</Stream>
<Stream name="PNGf" type="Base64" size="6547">LARGE_BASE64_ENCODED_DATA_REMOVED </Stream>
<Stream name="GLPH" type="Hex" size="1">01</Stream>
<Stream name="FNAM" type="Hex" size="28">000000010533373B3D3F00102E3F292E7733373B3D3F776B742A343D</Stream>
<Stream name="DPI_" type="Hex" size="4">00900090</Stream>
</StreamList>
</BinaryData>
</LibraryCatalog>Hashes of Layout Objects and Script Steps
One of the intended uses of SaXML is to compare (a.k.a. “diff”) two versions of the same file. The XML format isn’t well suited for diffing large sets of hierarchical data. Layout objects and script steps now include a “hash” attribute to make these kinds of comparisons easier.
<Step hash="6956823710DDA149C27D9C84FAD5DA69" index="0" id="141" name="Set Variable" enable="True">The exact rules for how a hash is computed haven’t been published. Based on my testing, the hash seems to be based on all of the information included in the relevant XML element.
Consider the following script step:
It will be representing in SaXML as follows:
<Step hash="F79361EFB6BD17FBBED598CD6E8FFF1E" index="0" id="141" name="Set Variable" enable="True">
<UUID>346EC373-2974-4302-A6D3-44B569B7FED5</UUID>
<OwnerID></OwnerID>
<Options>16388</Options>
<ParameterValues membercount="1">
<Parameter type="Variable">
<value>
<Calculation datatype="1" position="1">
<Calculation>
<Text><![CDATA["hello world"]]></Text>
<ChunkList hash="51C76AB7703246BA4FB92CB910320A06">
<Chunk type="NoRef">"hello world"</Chunk>
</ChunkList>
</Calculation>
</Calculation>
</value>
<Name value="$scriptStepHashTest"></Name>
<repetition></repetition>
</Parameter>
</ParameterValues>
</Step>If I make a change to that step – for example, by changing $scriptStepHashTest to $scriptStepHashTestCHANGED – and then generate a new SaXML, the hash for that script step will change.
If I then undo the change (by deleting “CHANGED”) and re-generate the SaXML, the hash will revert to its previous value. (Note that script steps do not include a ‘last modified’ timestamp; otherwise, the hash would have changed.)
But if I add whitespace to the “hello world” calculation, that will cause the hash to change, even though there is no functional change to the script.
Also, if I move the step to a different line (shown using the ‘index’ attribute in SaXML), the hash will change.
Modernizing FileMaker with Save as XML
The Save as XML function is the foundation for the potential to modernize FileMaker DevOps and analysis tools. The FileMaker 21 changes improve on the previous functionality and better position the FileMaker community to create tools for better automation of deployment activities, comparison of different file versions, and other capabilities that will improve DevEx (developer experience) and, as a result, the quality and efficiency of the application maintenance lifecycle.
If you end up spotting an inconsistency in the SaXML structure or grammar, let us know. The format is not fully baked yet, so there is still an opportunity to help shape it. For help leveraging new FileMaker 21 functionality within your application, contact our team to talk with a consultant today.
