If you are using FileMaker Pro or Server 20.3 or later, please read this updated blog post for updated insights and instructions.
Claris FileMaker just announced its first release of 2022. Our team has taken a good look and reviewed the release notes. Here are my thoughts on the most important changes and feature additions.
Introduction
This first release of the Claris FileMaker platform in 2022 is packed with changes, so let’s dive in and review the most important ones.
We will concentrate mostly on the new or changed features, but there are also a lot of bug fixes included with this release. Review the release notes (links required at launch time) for FileMaker Pro, Go, and Server to see if those will affect your solution.
While the version numbering scheme in software (<major>.<minor>.<patch>.<build>) typically carries meaning as to how important the release is, don’t let Claris’ numbering scheme fool you. All releases that bump up the “minor” number are, in fact, major releases. And so is 19.5.
In a recent blog post, we reviewed all the changes that were made to the platform in the last 12 months, which saw us progress from 19.2.2 all the way through 19.4.2.
The next sections will describe what is new for each of the FileMaker platform components: Pro, Go, Server, WebDirect, and the Data Migration Tool.
FileMaker Pro 19.5.1
New function: JSONGetElementType()
Sometimes you just need to figure out whether a particular piece of data is a JSON object or array, something you can parse with the JSON functions.
Up to 19.4, you would typically use the JsonFormatElement() function on the data and check the result for an indication that the data was not something that can be touched with the JSON functions:
The downside of this approach is that it can be slow when the data is substantial; you’d still have to do some text parsing of the result to detect an error or not (by looking for the leading “?”.
With the new function, you can do these checks more easily by verifying the JSON data type, which will work even if the input is an empty JSON object or empty JSON array:
And, of course, you can check any element within a JSON object or array to verify its data type before you set out to work with it:
Where do we use it:
There are some APIs, for instance, that will use an array if there are multiple ‘children’ but not an array if there is only one child element. This is an inconsistent way of dealing with sub-items, but not all APIs are consistent that way. In one such example below, the first order has four order items, but the second order only has one; the API provider chose not to use an array for a single element. This makes the item key a JSON object in one case and a JSON array in another, and we must parse it differently.
In our FileMaker script, we can now detect that very easily and branch our code accordingly.
Perform Script on Server – on server
Perform Script on Server (PSoS) was introduced with FileMaker 13 and has always been a script step you could only use in a client (Pro/Go/WebDirect and the Data API). Scripts that run on the server as part of a PSoS call or from a scheduled script run could not use the PSoS functionality to spawn other server-side sessions. There is some built-in protection with that limitation: if a script that runs on the server can spawn other server-side script sessions, which in turn spawn other ones, especially in loop constructs…you can see how an innocent mistake can lead to a massive server overload.
If you need the ability to speed up processing by running multiple simultaneous server script sessions for your routines, then you can now allow PSoS to be a server-compatible script step by overriding the existing behavior. You’ll need to set a feature flag in the ReleaseDebugOn file on your FileMaker Server. For instructions on how to use feature flags, see this blog post from October 2020 and FileMaker 19.1.
The flag name is SupportNestedPSOS.
A few words of caution here:
1. Prepare for Behavior Changes of PSoS Scripts
Using this feature changes the behavior of all your PsoS scripts in all files hosted on that server. If you were previously relying on scripts silently ignoring the PSoS script step when a script was executed on the server, then that will no longer behave the same and will now spawn a new server-side script session.
As a developer, you will have no indication whatsoever in the Script Workspaces UI as to how your PSoS call will behave when it is inside a script executed on the server. For that reason, we highly encourage defensive coding to avoid surprises by being explicit and not relying on implicit and silent failures.
For instance, if you have a script that might run locally or server-side, then add a few extra lines of code to detect where the code execution happens. With the added code, the server will use the regular Perform Script to keep the subscript execution within the same session, but when the same script runs from a client, it will call on the server to execute it.
Where the custom function _DEVICE_server is:
2. Consider Server Load Capabilities
Just like with the use of regular PSoS itself, do not blindly expect the server to be capable of handling the load. While developing solutions, it is often easy to believe that the liberal use of server-side activity will speed things up. However, if the server is not up to the task, the performance for all connected clients may degrade. In extreme cases with underpowered servers, the overall stability may suffer because of it. When you architect your apps with server-side activity, make sure to pick server deployments that can handle the load.
3. Manage Host’s Set Limit of Simultaneous PSoS Sessions
When you let a PSoS call spawn other PSoS sessions, it becomes a lot more likely that at some point, you will exceed the host’s set limit of simultaneous PSoS sessions. We suggest checking for Get(LastError) = 812 after making each PSoS call and handling the error in the best way for the task at hand.
Security
OpenSSL
Under the hood, all FileMaker clients now use OpenSSL 1.1.1n for their secure communications with FileMaker Server.
Calling remote files through an fmp URL
FileMaker can help protect your solution by disallowing any unexpected outbound fmp:// url calls from within your app to remote locations.
When an fmp URL executes directed at an unrecognized host, a new dialog can prompt you to that event:
You can cancel from here or allow the outbound call, and you can choose to trust all calls to this FileMaker Server.
If you do choose to use Always permit, you can then revoke those permissions at any time in the FileMaker Pro application preferences:
The Permitted section of the FileMaker Pro preferences is also where you will find the new checkbox to enforce this new security setting. Note that it is off by default, and you have to explicitly turn it on. Manually or through the Assisted Install mechanism when deploying FileMaker Pro.
If you find it odd that this setting is governed at the FileMaker Pro level and not at the solution / FileMaker file level: it would be too easy for someone to send you a malicious FileMaker file that would have the setting set to allow this kind of interaction. By controlling it at the FileMaker application level, you are protected in this scenario.
The Assisted Install file has been updated to also allow this setting to be turned on automatically when you install FileMaker Pro:
macOS-only
There are two new features that work only on macOS (and on iOS with FileMaker Go), provided you have at least macOS Monterey and iOS 15
GetLiveText()
This new function can grab any text that is in a picture. The quality of the returned text, of course, depends heavily on the quality of the source picture.
In addition to pointing the function to a picture stored in a container or in a variable, you also specify a language to help accuracy.
Note that using these functions on older macOSes will not generate an error. It will just fail silently, so make sure to do the required OS checks before calling this new functionality.
Also note that this function doesn’t do very well for more complex text structures like multiple columns. For that, you will need to integrate with a service like AWS Textract that returns text plus metadata such as its location on the picture.
The benefit of this new function – if your OS is the most recent macOS – is that you do not need to integrate with any outside service if your text recognition requirements are simple enough.
ReadQRCode()
Along the same lines, and also only available on macOS and iOS, is a new function to read the text embedded in a QR code.
You could, of course, already read a QR code from an iOS device with the Insert from Device script step, but this functionality allows you to read QR codes from both the desktop and iOS. It also allows for easier post-processing of container content.
As long as the source is a picture and the picture contains a readable QR code, this function will work:
NOTE: The function will, however, not work if the container has a PDF in it.
Save a Copy as XML
This feature was introduced with FileMaker 18 and did not bring any client-facing functionality. However, for developers, it is and will continue to be one of the crucial features of the platform in terms of documenting the schema. And later, as we have seen on the Claris roadmap, we will be able to use these XML outputs to diff and patch solutions in an automated fashion.
You can now run the Save a Copy as XML script step in a server-side schedule or PSoS or through any of the APIs. And with that, you can automatically document your solution and drop the XML representation of your solution in your preferred source control management system. We don’t yet have all the benefits of source control because we cannot go from the XML copy back to the actual FileMaker file. But we hope that this is coming soon and the level of attention that this feature gets makes us happy. FileMaker 19.5 enhances the XML output in significant areas to make it a more complete representation of your app.
One of the more visible changes is the removal of the generation timestamp at the top of the document.
Unfortunately, this means that you can no longer read the contents of the file and use this timestamp to know when the XML was generated. You will have to rely on the OS file timestamps.
Small Features
FileMaker 19.5 also has a set of smallish features that are nonetheless more than worthwhile mentioning:
Record Slider
When your record set or found count is bigger than 1,000, you will appreciate that the UI now uses your system’s separators.
Preserve Empty Tab Order for Copied Objects
When you copy an object from a layout where that object was not part of the tab order, then they will remain out of the tab order after you paste them onto another layout. The object will not automatically get the next tab order sequence assigned to them anymore.
License certificate upload or change
When your Claris license expires, FileMaker Pro will now prompt you and give you the option to upload a new license certificate.
You can also manually upload a new license (or change between licensees) directly from the Help menu:
FileMaker Go 19.5.1
First off: do make sure to read the release notes and look at the fixed bugs. There is a very important fix to the Get(UUIDnumber) function, which would, under certain circumstances, generate duplicate numbers. If you are using that function extensively, make sure to review the issue and the fix.
As mentioned in the FileMaker Pro section, the GetLiveText() and ReadQRCode() functions are available to you if your devices run on iOS/iPadOS 15 or later.
Scanning a QR code is something you can, of course, already do as part of the normal barcode scanning functionality with the Insert from Device script step.
There is one more type of barcode that is now being added to the barcodes that FileMaker Go can recognize: GS1 DataBar.
“DataBar barcodes are often used to label fresh foods. These barcodes can hold information like an item’s batch number or expiry date, in addition to other attributes used at the point-of-sale such as the item weight.”
These barcodes look like this, and FileMaker Go can read all these types.
In your solution, configure the Insert from Device options for the barcode to include the new type:
If you hold on to the picture taken by the Insert from Device script step, then you can always retrieve the barcode with GetAsText(). The GetContainerAttribute() function will also tell you what barcode type is in the picture in addition to the barcode text.
WebDirect
Performance
Over the last few releases, there have been significant performance enhancement tweaks made to WebDirect. If you are not on the latest FileMaker Server and use WebDirect extensively, you may want to set up a sandbox and do some testing.
On the topic of performance, also check out the auto-scaling feature for WebDirect worker servers that our Soliant.cloud team introduced back in March.
Login screen
Since 19.2.2, when your FileMaker Server has an OAuth identity provider enabled, all files will show just the OAuth provider login button on the login dialog. We wrote about that change in behavior a year ago in this blog post.
In 19.5.1, your WebDirect login screen will do the same so that the behavior is now consistent across all clients: it will not show the username and password entry fields for regular FileMaker accounts.
As a quick reminder, the toggle to show or not the FileMaker accounts fields is controlled by the file’s settings:
Customizable WebDirect login experience with OAuth
Instead of taking your user to the normal WebDirect login page, you can completely customize the login experience so that you can incorporate it with your own branding.
With this utility, you don’t have to take the user to the file’s WebDirect login page just to click the OAuth button there. You can start the OAuth flow from your own button on your own web page.
With regular internal FileMaker accounts, this has been possible for a while through an HTTP POST request as per the WebDirect documentation here. It is a feature that is not widely known. If you are using regular FileMaker accounts and not an external identity provider, then I would recommend this GitHub project to get started with this.
In a similar fashion, you can now make the WebDirect login experience a lot smoother if you are using externally authenticated accounts.
We will write about this in detail in a separate blog post.
And while the utility is being released now, we have been able to adapt it to work with FileMaker Server 19.4 as well since the authentication flow is exactly the same. More about that in our separate blog post.
FileMaker Server 19.5.1
Performance
Summarizing
FileMaker 19.5 continues the trend of giving us options to tackle performance by allowing functionality to run server-side or client-side. Much like server-side sorting introduced with 19.1.2, you can now choose to have summary field processing done by the server or by the client.
The feature is on by default. If you want to revert to the old behavior, then you can disable it by adding a feature flag named DisableServerSideSummary to the ReleaseDebugOn text file (for instructions, see this blog post.)
What this feature is not: a miracle cure.
Just like PsoS that we discussed earlier in this article, and any other server-side activity, it may not be faster if the server is not up to the task. Even with the feature left enabled, whether the server will actually take on the extra request depends on server activity. It is very much a negotiation between FMS and FMP/Go.
Server will take on the task if:
- the summary field does not reference fields from ESS tables
- the summary field does not reference an unstored calculation that requires client computation (such as using plugin functions)
- the summary field is not a global field
- there is currently no sort in progress
- the found set to summarize is larger than 25
- the host CPU usage is not higher than 25%
Parallel Backups
Up to now, backups have been done sequentially through all the files, one file at a time.
The new behavior attempts to identify files that can be backed up in parallel to make backups complete faster. And it applies to both progressives and regular backups.
But when the behavior kicks in is murky.
FileMaker Server will group files into file sets based on how it notices data changes that happen between backup intervals. By inspecting the transactions that happen against the files, FileMaker Server determines which files belong together and groups those in a file set. Each file set gets backed up in parallel (in their own thread). Files within a file set still get backed up sequentially one by one. Given that data changes to a file (transactions) can be unpredictable, a file may be in a file set for one backup run but not for another run. At the end of the backup run, all files are backed up; the notion of a file set is largely irrelevant except to FileMaker Server as it decides which files to group together and backup in parallel.
The FileMaker Server event log will capture info on what files are grouped in file sets for each run.
This feature is off by default, and you can enable it through the Admin Console, the admin CLI, and the Admin API.
In the console, go to the new Backup Settings section to toggle the setting on.
From the command line, you can check the current setting with:
fmsadmin get serverprefs ParallelBackupEnabledAnd toggle it with:
fmsadmin set serverprefs ParallelBackupEnabled=true/falseIn the Admin API, you will find a new endpoint to use and check or change the setting:
https://<your server>/fmi/admin/api/v2/server/config/parallelbackup
What this feature will not do:
Make all backups faster every time. The behavior will be very dependent on the usage of the files. We highly recommend monitoring your deployment and evaluating whether you see an impact of this feature.
If backup performance is a concern to you, have a look at this blog post where we discuss speedy alternatives.
Scripting Engine Cache Size Increase
The scripting engine’s cache size has been increased from 64MB to 256MB. This applies to the whole process, not every single instance of the scripting engine running (each schedule or PSoS session). The increase in cache should help with bulk operations that touch a lot of data, such as import or bigger iterations.
What this feature is not: a magical setting that makes all PSoS and schedules faster. The impact very much will depend on the nature of the task that the server is asked to perform and will only kick in significantly if there is a lot of data that needs to be cached.
Cancel Backup Command
The new ability to cancel a backup that is in progress is not a pure performance tweak, but it can certainly free up performance by canceling a suspected runaway backup or come in handy at go-live time when you cannot wait for a particularly long backup to finish.
You can cancel a running backup from the Admin Console:
Also note that Progressive backups have been removed from that section and are now in the new Backups Settings panel (see screenshot a little earlier in this article).
From the Admin command line, you can use:
fmsadmin cancel backupAnd there is a new endpoint in the Admin API that you can call to halt a backup:
https://<your server>/fmi/admin/api/v2/server/cancelbackup
A big word of caution: it’s obvious, of course, but it must be said. Canceling your backup will leave you without this backup. That’s typically never a good thing.
As mentioned earlier: if backup performance is a concern to you, then have a look at this blog post where we discuss speedy alternatives.
Integration – OData API
We have been big fans of the OData API since it became available in the FM Cloud offering and the on-premise Linux version of FileMaker Server.
With 19.5, that OData API is now available in FileMaker Server for macOS and Windows. It brings many benefits that are not available to the regular Data API, such as transactions, the ability to update the schema, and no two-step authentication that requires a login to get a token. You will definitely see more postings about OData from us.
Deployment
Linux Ubunto 20
FileMaker Server now supports Ubuntu 20.04 LTS in addition to Ubuntu 18.04 LTS. Keep in mind that the Ubuntu 20 version has a different installer, and you will get errors trying to install the wrong version on the wrong operating system.
On Ubuntu 20, the web server is now nginx instead of Apache. If you typically make customizations to any of the web server config files, the syntax may be slightly different.
Windows and macOS
No significant changes on Windows; the supported versions are still Server 2016 and 2019. No support yet for Windows Server 2022.
The minimum version of macOS is still Catalina 10.5.
Save a Copy as XML Script Step on FileMaker Server
Earlier in the article, we had already highlighted the updates to the Save a Copy as XML script step.
The ability to do a save as copy as XML server-side as part of a schedule brings us another step closer to better and faster dev-to-prod workflows (think: Continuous Integration).
We lack the ability to create patch files out of the diff between two versions of the XML files and a version of the patch tool that reliably implements all schema changes. But we have good hopes that this is the path that we are on.
In the short term, think of this updated script step as a means to easily do source-code backups.
One of the oft-overlooked features is that the XML format tracks who made changes and when the schema element was last modified. When you take frequent code backups with Save a Copy as XML, you can easily build a modification log of your solution.
For an implementation of reading the XML file to list out the modifications, see this proof-of-concept tool from Proof+Geist: https://lab.geist.tools/
WebDirect Workers
If you have WebDirect worker servers, those will now communicate to the main FileMaker Server through port 443.
This change is similar to the previous change in 19.4 to also use port 443 for access to the Admin Console. The effort by Claris here is to reduce the number of ports that a FileMaker Server deployment uses.
This will, of course, affect your deployments where firewall ports need to be opened on the path between the workers and the main server.
Modified config files
When you make changes to the various FileMaker server config files (like those required to allow for WebDirect home Urls), then those modified config files are now preserved after an uninstall. You can then use these files if needed to verify what the new configuration needs to be in a scenario where you prefer not to do an upgrade-in-place of FileMaker Server but rather an uninstall-and-reinstall.
Data Migration Tool
Better standalone installation on Ubuntu
The new version of the Data Migration Tool (DMT) provides a smoother installation on an Ubuntu machine that does not already have FileMaker Server. Currently, you need to manually install a lengthy set of dependencies first before the DMT tool will function properly.
That problem now goes away, which is useful in scenarios where you want to use quick-setup, single-use virtual instances to speed up the data migration process. You can run the Data Migration Tool in parallel, as described in our blog post from May 2020, so you can certainly spin up a temporary Ubuntu AWS / Azure / Google Cloud instance with all the horsepower you need to get through the task quicker than you can by running it on your existing FileMaker Server.
Support for Ubuntu 20
There are now four versions of the Data Migration Tool: for Windows, macOS, Ubuntu 18, and now also for Ubuntu 20.04.
New locale option
As you know, the Data Migration Tool uses a clone of the file to which the data will be pushed. And per the normal FileMaker behavior when opening a clone file for the first time, that file will inherit the system locale settings (language, date formats, number formats, etc.) of the system where the Data Migration Tool is executed because the Data Migration Tool is the first FileMaker client so to speak to open the file.
Managing the desired locale for the file then becomes somewhat challenging as you would need to configure different machine settings ahead of running the DMT to make sure that the resulting file inherited the intended settings.
FileMaker 19.5 helps solve this challenge with a new option you can add to the DMT:
-target_locale
The locale can be specified in one of two ways:
- as just a named locale, from one of these options:
| System | English | Icelandic | Panjabi | Swedish |
| Arabic | Estonian | Italian | Persian | Tamil |
| Bengali | Finnish | Japanese | Polish | Telugu |
| Bulgarian | French | Kannada | Portuguese | Thai |
| Catalan | German | Korean | Romanian | Turkish |
| Chinese | Greek | Latvian | Russian | Ukrainian |
| Croatian | Gujarati | Lithuanian | Serbian | Vietnamese |
| Czech | Hebrew | Malayalam | Slovak | |
| Danish | Hindi | Marathi | Slovenian | |
| Dutch | Hungarian | Norwegian | Spanish |
- as the path to a JSON file. The JSON file needs to be the output from the FileMaker functions Get(FileLocaleElements) or Get(SystemLocaleElements).
Implementing New 19.5 Functionality Into Your Business Solution
If you need help leveraging new features, functionality, and settings in your FileMaker business application, our team can help. Contact us to talk with a FileMaker consultant.
Great Article Wim