It’s been a while since the last BaseElements update, and I’ve had a few emails asking me what’s going on. Rest assured we’re still working on it, and things are progressing.
And FileMaker 15 is out today, and happens to coincide with me finishing up a big set of changes to BE, so an opportune time to post an update. So to answer all the big questions :
Are you still working on it?
Absolutely. We still use BE every day ourselves, and even use BE to develop BE and couldn’t live with out it, much like a lot of you. We haven’t sat still but development has been slower while we’ve worked on client work, RefreshFM updates, RESTfm updates and additions and bug fixes to the Plugin.
Why is importing objects so slow in b12?
What’s mostly got us stuck is some changes to the Alerts that we introduced with the new 14 re-design. In order to better track the myriad of issues that can crop up, and be better able to handle errors, warnings, performance issues and unreferenced items, we’ve been consolidating them into a single alert. So we’ve added some extra items and included more detail for others, plus combined things into a single location.
And as every developer knows, when you add to a solution iteratively over a long period, all your changes add up. And what we ended up with was a solution for layout objects that was way too complex than it should be to get the job done. At some point in the last few betas our alerts auto enter calculation was doing a sum of a related field, which itself was un-stored, and that will never be fast.
You never set out to design a slow solution, and you never deliberately make a change that is going make things slow, but occasionally you add a great feature that has a cost you didn’t plan for.
The solution was to go back and re-think the whole way alerts are generated across all the elements, but especially the large ones ( objects, steps, calculations ). And to do that meant putting a lot of these calculations into the XSLT so that the import was as simple as possible, and therefore as fast as possible.
If you want fast imports, remove all your unnecessary auto enters, lookups, stored calculations and indexes.
We’ve spent a lot of time re-factoring our XSLT to shift a lot of the complexity there instead of the import. This is a complex, difficult process – XSLT is handled differently in the plugin, vs the command line, vs the import, vs the debugger tools. This makes it really time consuming to debug and test any change.
Our base line for speed testing was importing our sample files ( BaseElements itself ) into the current BE 4. That takes about 5 minutes. We knew that the latest beta of 14 was slower, quite a bit slower. We needed to at least get back to where things were with that version.
At various times during the changes we had imports that took one minute, only to realised hardly any data had come in, and 7 minutes only to make a simple namespace change that cut that in half.
Finally we’ve got all our data, all working as we want and import times are … drum roll … 3 minutes. About a 40% savings on BE 4 and much faster than BE14b12. Excellent, not only did we fix the b12 issue, but we also improved on the previous version.
What’s with the beta now that 15 is out?
Our plan with BE 14 was that we’d finish the beta late last year, and then release BE 15 on the release of FMP 15. Obviously that didn’t go to plan.
We’re fortunate that there are almost no changes to the DDR with 15 so our current release version of BE 4 will work with the FMPA 15 DDR just fine. So will the BE 14 beta.
So we’re going to re-start the BE versioning at 15 and go back to b1 and pretend that the whole BE 14 beta never happened 🙂 Well not quite, but it doesn’t make sense to have BE 14 around with FMP 15.
There is no upgrade cost from BE 14 to 15. So we will now work on finishing up the remaining features we were working on ( reports mostly ) and release BE 15 as what we were going to do with 14. Everyone who has bought BE back since mid last year will get the final release of BE 15 at no charge and all the current BE 14 licences will work just fine with BE 15.
What else have you done?
One of the biggest support issues we have is in bad XML that FMPA generates that is invalid and so can’t be imported. We are pre-processing a lot of that and removing some of these characters and so a lot of imports that would have failed in older versions will work in BE 15.
What else are you going to do?
We’ve still got some tweaks to how the UI works in BE 15 that we need to work on. There is the reports to do, and we’ve got some great ideas about those that we think are going to be awesome.