update

BaseElements 4.0.5

Submitted by Nicholas Orr on 10 April 2013 - 2:04pm

Sometimes BaseElements updates come thick and fast. Other times there's a wait between versions. This is one of the latter.

BaseElements 4.0.2 was released back in July of last year. We've since been working on some minor tweaks and fixes to BE that didn't at any point seem, by themselves, to warrant an new release. But we had also been working on a change to the free BaseElements plugin that allows us to remove nodes from XML documents. This was working with the 2.0.1 update to the plugin back in February.

What's the connection to BaseElements? Glad you asked. The biggest issue we have with the import process in BE is that on very large data sets ( many GB of XML DDR files ) we often hit limits in FileMaker. With the plugin, we don't have access to more than 2GB of memory. Above this we get errors ( which we can trap for ). So we switch our XSLT processing to the internal Xalan based import engine. This works great, except in some cases where not only does it not work, it just stops half way. No errors, nothing is reported, but people contact our support page to say that the import only half worked.

There is no way for us to alter the internal FMP import, as we don't control that part of the process. Until the plugin architecture moves to 64 bit in the client, there's no way to increase the amount of memory available. And until we move to a XQuery based import model we need to load all of the XML file into memory every single time we import, and there are multiple imports for a single Analysis.

So the new 4.0.5 release of BaseElements includes and option to reduce the size of the XML files by removing some nodes it doesn't need, and only importing the end results. We remove the HexData and PlatformData ( print config ) nodes, as well as ObjectStyle, CharacterStyleVector and ParagraphStyleVector nodes. None of this contains anything useful to BE, and the resulting XML files can be up to 90% smaller than the originals.

In all our testing so far, this has been able to fix issues with a DDR that just failed to import before in BaseElements. And in some cases it's also reduced the import times as well. The original files aren't touched, and the import result would be exactly the same either way.

This is a new checkbox on the setup layout inside BE :

ReduceFileSize.png

It's off by default, so if you're having this issue, or would like to see if it reduces import times on your large solution, give it a try. And if you find anything useful or have any issues to report, please contact us.

There's a whole slew of minor fixes in this version of BE, so we suggest everyone using v4 should update.

BaseElements Plugin Version 2 Released

Submitted by Nicholas Orr on 5 February 2013 - 2:12pm

Today we've made available version 2.0 of our free FileMaker plugin. You can grab a copy here :

Mac Version
Windows 32 bit Version
Windows 64 bit Version

This is our first big release since version 1 and so we've bumped the version number to 2.0. There is lots of new and changed functionality in this release.

Changes

Added a BE_Base64_URL_Encode function.

This is as well as the existing BE_Base64_Encode. The difference is in the character set and details of the encode. The short explanation is that if you're using the built in encode / decode, then it will all work fine. However, you're sending data to web services that use, for example, the php based base64 library then the character set they're using, and the encode method is slightly different. This often results in values that don't quite work at one end. The new function produces url safe output and also pads the value in the same way as the web version.

Added BE_CurrentTimeMilliseconds, BE_UTCMilliseconds, and BE_TimeZoneOffset.

BE_CurrentTimeMilliseconds is effectively a copy of GetAsNumber ( Get ( CurrentTimestamp ) ) but was being added anyway with the offset coding.

BE_UTCMilliseconds is just the UTC version of BE_CurrentTimeMilliseconds, but for the UTC timezone.

BE_TimeZoneOffset is the difference ( in minutes ) between UTC and local time. This one will be really handy when doing remote data entry and having to sync back to a server that is outside the current timezone.

BE_ExecuteShellCommand was deprecated. The function still exists, and will still work exactly as before, but no longer appears in the function list when editing a calculation. If typed, or copied it will continue to work though. It's been replace by ...

BE_ExecuteSystemCommand was added. This changes slightly the operation in terms of having a different second optional parameter. Now instead of a waitForResponse option, you can specify a timeout. This is much better in that you no longer get stuck if your process gets stuck. There is still the option to wait for ever by using a -1 parameter, or not at all with a value of 0.

Added BE_FileSize function.

Changed the way the BE_GetLastError function works, so that it doesn't reset on successive calls. So you can test the value of this result with multiple calls to the function and it doesn't change until you call some other BE plugin function.

Added BE_HTTP_DELETE and BE_HTTP_PUT, so we now have the full complement of GET, PUT, POST and DELETE, plus we added basic authentication to all of the HTTP functions.

Added default folders to the BE_SelectFile and BE_SelectFolder functions so now you can choose what folder it will start in.

Added a function BE_StripXMLNodes which although not useful to many people is going to be handy to us in the future for our changes to the BaseElements import process which hopefully gives us some useful optimisations we can do.

To Recap

Our BaseElements plugin continues to be free and open source. You don't need to licence it, register it or pay for it. You don't even need to tell us you use it, but if you like it, please let us know.

I've updated the Functions list with all of the new details and the Todo List page with the changes.

The Future

We already have some things we'd like to do, possibly in point releases of 2.x but possibly also in a 3.0 version. I'd like to add a progress dialog, just something basic with a start and an end. And then I'd like to look at both XQuery and SMTP support. SMTP is an optional one, so anyone who would like this functionality and would consider sponsoring us should get in touch.

BaseElements Plugin nearly goes to 2

Submitted by Nicholas Orr on 23 January 2013 - 8:41pm

We're going to call it done with additions to the BaseElements plugin for version 2.0. If you're interested you can get a copy of 2.0rc1 here :

Mac 64 and 32 bit
Windows 32 bit
Windows 64 bit

The only change since b4 of significance is that the BE_GetLastError doesn't reset when called multiple times, without calling another plugin function in between.

This the last change for 2.0. I think we're going to jump straight into developing the 3.0 release, we've got plans to work on SMTP and XQuery functions and I'd like to see both of those done soon.

Free FileMaker Plugin updated to 2.0b4

Submitted by Nicholas Orr on 5 December 2012 - 8:42am

We've posted another update to our free FileMaker plugin. This brings the version number to 2.0b4 and adds some new features and tweaks some existing ones. Download Links are :

Mac 64 and 32 bit
Windows 32 bit
Windows 64 bit

This release has the following changes :

  • Add BE_ExecuteSystemCommand and depreciate BE_ExecuteShellCommand - this means on Windows the command window no longer appears. The existing function won't go away so your code still works, but the new one also has a timeout parameter so you can set timeouts.
  • Use of whitespace in function signatures to match that used by FileMaker
  • Added BE_CurrentTimeMilliseconds, BE_UTCMilliseconds and BE_TimeZoneOffset
  • Added BE_HTTP_Set_Proxy function for use in the other HTTP functions

BE_CurrentTimeMilliseconds - is the same as the built-in Get ( CurrentTimestamp ) when wrapped with GetAsNumber. BE_UTCMilliseconds - as per BE_CurrentTimeMilliseconds but for UTC rather than local time. BE_TimeZoneOffset - the difference between UTC and local time (in minutes).

The TimeZoneOffset functionality is going to be the most useful, especially when working with sync solutions and needing to keep consistent time zones for data entry.

Free FileMaker Plugin updated to 2.0b2

Submitted by Nicholas Orr on 18 September 2012 - 10:45am

We have an updated version of our free FileMaker plugin available today. This is 2.0b2 and will likely be the last beta before a final 2.0 release. This version adds support for running on 64bit FMS in CWP for Mac and Windows as well as deprecating the BE_FileMakerFields and BE_FileMakerTables.

Those two SQL related functions will continue to work in any existing code, and they can be put manually into a new calculation, but they don't appear in the list of available functions. The reason for this is that they are now available natively inside FMP using the new ExecuteSQL function.

Our policy from here on is to keep the plugin limited to functions that FileMaker itself doesn't support, and removing this from the list, but still making the function work is the best way to achieve that without breaking existing code. At some point in the future we may remove the function all together, but for now having them work but be inactive is a good solution.

There are now three files to download ; the Mac version, a 32bit version for Windows and a 64 bit version for Windows. The Mac version is a universal build and has both 32 and 64 bit versions in the one file.

RESTfm updated

Submitted by Nicholas Orr on 5 September 2012 - 11:36am

We've released a minor point update to our FileMaker Web Services solution RESTfm. This version 1.1 adds :

  • Add script and pre-script support to layout and record CRUD operations.
  • Add OSX Snow Leopard specific instructions to report.php
  • Add report.php plain text to copy and paste report.php to support.
  • Add more diagnostic information to report.php

Full notes are on the RESTfm website.

Minor update for RefreshFM

Submitted by Nicholas Orr on 14 August 2012 - 2:27pm

We've posted some minor updates for RefreshFM today, fixing issues in the distribution version. If you're running a developer licence it won't be critical for you to update your copies to this release.

Also the 2.0 beta feedback is positive so far, so we'd expect to have a final release shortly.

Free FileMaker Plugin beta 2.0

Submitted by Nicholas Orr on 15 July 2012 - 4:21am

Just a quick note to let you know we've moved the 2.0 version of our free FileMaker plugin from alpha to beta. So we're finished adding new features, the last one to come in this release is a get file size function. So there won't any more big changes, only bug fixes to this release now.

Downloads are available for Mac and Windows and we'd love to know if you find any issues when using this release before we make it final.

Free BaseElements plugin version 2 already

Submitted by Nicholas Orr on 3 July 2012 - 11:46am

It seems like it was only yesterday I was mentioning the new up and coming version 2 of our free FileMaker plugin. And today I'm excited to let you know that we have an alpha version ready to play with.

This is 2.0a2 and includes new HTTP_DELETE and HTTP_PUT functions, as well as adding support for authentication for all the HTTP functions.

This is still considered alpha, while we look at some changes to the way shell commands work and also the Base64 encoding functions, but we'd love some feedback on the HTTP stuff in the mean time.

Also we've updated all the base libraries we're using ( Boost, CuRL etc ) so there should be some bug fixes in there too. And it's now a Xcode 4 project on the Mac - the source code should be up on Github shortly for those interested in that.

The big change is that this plugin is now FMP 12 and later only and won't load in 11 or earlier, and you can download a copy for Mac and Windows.

Please send us any feedback or issues for the new functions.

BaseElements Plugin Update

Submitted by Nicholas Orr on 19 August 2011 - 12:09pm

A quick note to let everyone know about a minor update to our free open source FileMaker plugin. This updates the version number to 1.2.1 and includes a fix for extra characters appearing at the end of the BE_ExecuteShellCommand function. This would have resulted in the commands not working or doing strange things.

In our case we are using this to launch AppleScripting of the FileMaker user interface in RefreshFM. So although it didn't show on our test or development machines we quickly started to see reports of this after launch. We'll have an updated RefreshFM version out very soon with the new plugin and some more features to boot.

As usual the downloads are available for Mac and Windows and the 1.2.1 source code is already available.