Goya Blog

Plugin Update - HTTP POST, Base64, preferences, zips and more

We've finalised the set of features to be included in the next update to our free FileMaker plugin. This one includes :

  • Changes to the SQL library to work with the new SQL API on FileMaker Pro 11 and later. This has more options and allows for a more robust feature set from now on.
  • Changed the url functions to do HTTPS as well as HTTP.
  • Two new functions to get and set preferences inside files in the user's folder.
  • HTTP Post on top of the HTTP Get function. Plus additional functionality with this to set and retrieve headers and other result codes.
  • An Open file function.
  • Ability to zip and unzip files from within FileMaker.
  • An optional starting folder for the select file and select folder functions.
  • An encoding function to specify the encoding type, you can select any of these : http://www.gnu.org/software/libiconv/
  • Improved error handling
  • Base64 encoding and decoding

You can download 1.3 beta 1 for Mac and Windows.

Plugin Updates - HTTP POST and Sponsorships

Background

Our free FileMaker plugin started it's life as something developed entirely for ourselves. We wanted to reduce the number of plugins we were using in our BaseElements FileMaker developer tool, and couldn't find anything that fit perfectly.

It seems, and it makes sense that most FileMaker plugins stick to a single set of functionality. So for there is the Troi File plugin for file operations, SMTPit for sending mail via SMTP and Troi Dialog for dialogs. ( There are others that do these things too, these are just the ones we recently used ). Each of these plugins is a complete package and will do everything you want in File, SMTP or Dialogs. If you have complex needs or specific requirements, then this is the way to go. We've used, and continue to use these plugins for various things when the needs arise.

But our requirements were that we needed a couple of file functions, one basic dialog function, some XSLT stuff that didn't seem to exist in any plugin yet and a couple of very esoteric things for speeding up processing in BaseElements. Some of the esoteric things could have been done in something like Scodigo's PHP plugin but we'd still have multiple plugins to install and manage.

In the end, writing our own was a no brainer. It allowed us to have a single, BaseElements branded plugin which did everything and was in our control in terms of updates and bug fixes. But the question we had was what to do with it. Should it be limited to our own solutions? Should we try to sell it? Give it away? Keep it in house?

Our decision was that we'd open source it. Give away the code and the plugin itself so that others can benefit from it and use it in the same way we do.

One of the things that this opens up is the idea of it growing beyond our initial plans, and including code for other people's desires as well as our own. We certainly have our own plans and ideas, all based around things we plan to do in the future in either BaseElements or RefreshFM.

Sponsorships

Recently we've had a couple of requests for functions we hadn't planned on adding at present, but we felt made sense to have in our "toolkit" plugin. So we've opened up the ability to sponsor BaseElements functions that we will then get coded and added in.

These were some HTTP POST functions ( to add to the GET function that is already there ) and a set of Base64 encoding to accompany them.

We're very excited that we've had interest from three different sources to sponsor these functions and that we've been able to add them into the plugin code.

How it works

If you have an idea for a function, or a set of functionality that you would like to see in the plugin, contact us via the form on this page, or the github comments page. If we think it's a good idea and could be done, we'll add it to the existing list and also add an individual function page. On there you can comment, and we'll flesh out the details of parameters and functionality.

Once all that is done and agreed on, we'll put up an estimated price and put out a request for sponsorships. Anyone can contribute, if you want to pay for the entire amount you can, if you want to round up individual donations and bundle them up you can. There is a paypal account listed to send contributions to. In the case of functionality not being developed, anything donated that isn't used will be returned to the donors.

We have an external contract developer we're using for the coding. All of the amounts go towards paying the developer and nothing is kept by Goya. And we're not restricting this to our own developer either, if you know of or have your own programmer who understands FileMaker plugin development, then we're happy to accept code via github as well.

The only restrictions would be that we stick to our original licence terms, so any code must be open sourced and that the functions work cross platform.

First Steps

Our first set of sponsored functions are on the sponsorship page, and the code is in the progress of being developed. We were so excited about this that we've already started some coding and we have some alpha releases available on the github page ( scroll down to the 1.3 alpha links ).

Some other bonus features in the 1.3a5 release that are things we're adding for ourselves are a File Open function, zip and unzip and the ability to set the default folder for the Select File or Select Folder dialogs.

The future

Hopefully this attracts others who have ideas for things not currently well supported by other plugins.

As well, we're constantly on the lookout for something to manage the donations. Keeping track of this manually is a pain, and although sites like kickstarter seem promising, they're limited to US residents only ( i.e., not us ) and don't really work in terms of having a project with 6 different ongoing functions that could all run separately. If people have other suggestions, we're open to ideas. Some sort of drupal module would be great, as that's what we're currently using on the site.

FileMaker Server and a Linux Webserver

You can run FileMaker Server on Mac OS X or Windows, but not Linux. But you can have a Linux based web server accessing either OS and it is quite simple to setup and run. The bonus is that because you're following a standard FileMaker Server install it should still be a supported configuration.

To start with, choose your FileMaker Server OS, and install FMS as a Single Server option. You can then use firewalls if required to block this machine from unwanted access. You still need a web server on this box, so in our case this is IIS, but it's firewalled off from public access and limited to access only from our Linux Web Server.

We chose Ubuntu 10 for our Linux box. Anyone with some Linux experience could replicate these steps on other platforms though.

The FileMaker API

First get a copy of the FileMaker API files, they're already provided in the FileMaker Server install at :

FileMaker Server\Web Publishing\FM_API_for_PHP_Standalone.zip

Take these zip files and move them to the Linux box, and unpack them somewhere that's NOT in the web root folder, for example in :

/home/myname/FileMaker

You then need to add this folder into the php.ini file, in the include_path option. So our new include_path was now :

include_path = '.:/home/myname/FileMaker:/usr/share/php:/usr/share/pear'

Also our default PHP install needs a couple of extra packages to work. PEAR Date.php and curl are required, so install those:

apt-get install php-date
apt-get install php5-curl

Finally edit the conf files inside the unpacked API to point to the actual FMS host. This file is located at :

FileMaker/conf/filemaker-api.php

in whatever folder you unzipped the API files previously. There is an option in there for $__FM_CONFIG['hostspec']. You need to change the hostspec setting to be the IP or domain name of the FMS box.

And finally restart Apache and you should be all good to go.

Troubleshooting

If it's not working, check your firewalls. The two web servers ( Apache on Linux and IIS on Windows ) need to be talking to each other via port 80. Also, check the web and PHP logs on both machines for other clues or issues.

Using IWP

This works perfectly well with Instant Web Publishing as well. You need to add another step whereby you use a Reverse Proxy to direct just those urls to the internal FileMaker Server address. So this would be something like :

ProxyPass /fmi/iwp/ http://your.fms.domain/fmi/iwp/
ProxyPassReverse /fmi/iwp/ http://your.fms.domain/fmi/iwp/

This can go in your .htaccess file if you're using those or in a VirtualHost section for multiple domains as well.

Storing user preferences on disk in FileMaker Pro

One of the things I've done manually in BaseElements is to have the some configuration options stored in the user's preferences folder in a text file. This helps to make it easier to have options unique for each user, but without a user table in the FileMaker file. This has worked well so far, but with this method you need to :

  • Manually locate the preferences folder on each platform as there is no Get ( PreferencesPath ) function.
  • Manage the reading, writing and creation of the file on disk.
  • Develop and maintain your own structure for the content of your preferences on disk.

None of these are insurmountable problems, but they can add to the complexity of a preference system. What would be much simpler would be a simple Get and Set method that handled all of the underlying details for you.

Version 1.3 of our BaseElements plugin now has just that functionality. There is a Get and SetPreference function call which allows you to set user level preferences and have them stored inside the users preference folder without any issue of managing files or storing data.

The documentation on the wiki describes the new functions, and the Todo list page contains download links for the new release.

This new functionality will be in the next release of RefreshFM, and will at some point come into a future release of BaseElements as well.

HTTPS as well

If you're following the development of the plugin, this version also adds HTTPS to the Get url function. At present this limits the plugin to FileMaker 11, so if you need it on older platforms hold off until a later release resolves this limitation.

Little RefreshFM Update

Hot on the heels of yesterdays Big RefreshFM update is another smaller RefreshFM update. This one bumps the version number to 1.1.6 and has two minor fixes :

Firstly the portal listing old file names is now editable, so you can modify file names directly there, or add in new files.

Secondly if you had a setup where you had multiple files, or filenames between old and new that weren't the same, it was possible that the copy old file step would copy files with the wrong name. This has been fixed.

As usual, downloads are available, and you can import from a previous version ( from 1.1.3 onwards ) into this release using the "Import" button when RefreshFM opens for the first time.

Big RefreshFM Update

Available for download now is a new release of our FileMaker update automation tool, RefreshFM, bringing it to version 1.1.5. Although it's only a minor point release in number, this version is a big update behind the scenes and has a lot of changes.

I'm really excited about the work that's been done on the FileMaker Server options in particular and how it's automatically opening and closing files, and even using Applescript on the Mac to authenticate and give the uploaded files the correct permissions before opening them so they can be accessed straight away.

Plus it adds an installer for the BaseElements plugin, so that can be updated without any manual intervention, some huge speedups in the Data section and lots of much smaller fixes to boot.

Also this is the first release since 1.1.0 to concentrate on the Distribution version, so there are lots of changes there too. Anyone with a distributed FileMaker based product would be well suited to test RefreshFM for their next update.

Testimonial

And if you're a FileMaker developer who hasn't tried RefreshFM, you might want to consider it. @solfilemaker did and he wrote about it on twitter :

Tried refreshFM 4 1st time Amazing tool it did a 3 hours job in 15 minutes.

RefreshFM has been saving us time like this with our client work for years and we're finally glad to see other FileMaker developers finding out just how useful it can be.

Accessing BaseElements Records from other Solutions

There is a feature in BaseElements where you can access any Analysis from any other solution by calling a script inside BaseElements. This is a neat feature for those people who use other FileMaker based project management tools and allows you to build a link into BaseElements for the current analysis of your FileMaker solution.

But there is also the ability to access the data tables inside the BaseElements_Data file, which means you could build your own UI using the BaseElements data and access it any way you want. This could be really neat for managing your solutions, but there is no way to link to the correct layout in the BaseElements UI file for any record that you're viewing the data for.

BaseElements 3.0.11 adds a new feature where there is a script you can call with layout and field details, and BaseElements will go to that layout and perform the find for you. This effectively gives you the ability to link to records in the UI file which you otherwise would not have. The actual data tables are all in the Data file, so a GTRR will let you go there, but there's nothing useful to see.

To get this working, there is a script in the main BaseElements file called "External: Go to Related Records" which you can call with parameters to set which layout to go to and which fields to search, along with which values to find in those fields. This script is modifiable, so you can see for yourself what it does, and make changes if required. The bonus is that it won't be changed in future releases, so you can call this script from other files and it will remain the same.

The parameters are a list of values you want to use. The first item is the layout name. All of the remaining values are in name-value-pair format according to our previously published examples. So for example, to go to the layout called "MyReport" and search for "abc" in the field called "widgetname", you would use the following as your parameter :

MyReport
widgetname="abc"

I've attached here a sample file containing scripts that do this for you, along with examples of Custom Functions we're using to set and retrieve these name value pairs.

AttachmentSize
External.fp764 KB
External.fp7.zip10.04 KB

BaseElements Updated to 3.0.11

A quick note about another point release to BaseElements. Version 3.0.11 is now available for download and contains the following changes :

  • Added an external script for accessing BaseElements data from other systems.
  • Modified the list layouts to have alternating backgrounds.
  • Added a 2px indent to fields on list layouts.
  • Added more detail about reference types to the field object portals.
  • Fixed a bunch of item count issues.
  • Updated the FileMaker Go Report to include the changes in 1.2
  • Updated the plugin to version 1.2.1

The manual page to explain the new external access setup will be updated shortly. As usual this is a free update for anyone who has a version 3 licence.

Updating Plugins with an Installer

A long time ago ( in internet years ) Digital Fusion published their method of using an installer to update FileMaker plugins. This was part of Craig Saunders presentation at the FileMaker Developer Conference 2010 in San Diego.

Although I've previously published methods for updating plugins via FileMaker Server, I've been meaning to look at the Digital Fusion method for a while, and build a sample file for doing this using our new free open source BaseElements plugin.

Well I finally had a concrete need for an plugin update mechanism in RefreshFM and because RefreshFM doesn't work on the server, it needed to be all local. As well, with the distribution version of RefreshFM is going to be in use by users who may know nothing of plugins it needed to be as simple as possible. So the idea of using this installer method really appealed to me.

Tweaks

I made a couple of changes to the process that Craig and Stephen outlined in their sample files.

Firstly, on windows, I elected to use InnoSetup instead of Install Creator. InnoSetup is free and has all of the features I needed, and I was already familiar with using it on other projects. But in theory you could use any installer builder software.

Secondly, on the Mac, the DF guys are exporting an XML plist file to do version comparisons with. I considered this overkill for what I wanted, and I was happy with just using the ability to check this detail inside FileMaker and then installing as required. This gives you a little more flexibility with where you install and also made the actual options in the installer simpler.

And finally, I also wanted the ability to re-open the database that you'd started with. I was sure from the demo or what I'd seen before that this was included with the demo materials, but I couldn't find it and the online demo files didn't show this ( or at least as far as I could see ). So I added some code to the post install scripts to open the RefreshFM file after the install is complete. It is mentioned in their slideshow from the DevCons session as an option for windows, so I assume they've at least considered it on the mac too.

Demo Files

The attached demo files contain a working copy of everything I've included in RefreshFM. There is the main PluginUpdates.fp7 file which contains a copy of the installers as well as the code to export and open them.

The .pmdoc file is the PackageMaker file. You need to download Apple's Developer tools to get access to this. XCode is available on the App Store, and is also available for people who have developer accounts with Apple.

The .iss file is the installer setup file on Windows, you need a copy of InnoSetup to open this, but it's free software.

Mac Installer Bug

There is a bug with the installer on the mac, where the settings I've used default to the User's Library folder. So when the installer comes up, there is only one choice ( Install for current user ) which appears to be selected ( it's got a blue highlight ) but the continue button is greyed out. You need to click on the "Install for current user" option to make it active ( which doesn't change the highlight ), and then the continue button becomes active and everything works.

I've left this as is in my sample files, so the issue is still present. The DF guys have got a workaround in their version of the installer, where they install to the temp folder, but into a link that points to the actual user's folder. So the installer doesn't present the dodgy dialog. However, that option requires Administrator privileges, which not all users will have.

I'm not sure that either solution is perfect, and I've spent a lot of time fiddling with settings or options in the installer to try to alleviate the issue, to no avail. If you have a perfect solution, I'd love to hear about it.

Caveats

And what solution wouldn't exist without caveats : Currently there is no way within FileMaker Pro that I'm aware of to systematically enable a plugin that is inactive. So if the user has the plugin installed, but inactive, this process will update it and leave it inactive, so it will essentially fail. I think this it is reasonable to assume that those using this process will not be making plugins inactive, but it needs to be noted that this is an issue.

Thanks

Thanks to Craig and Stephen for putting the work into this idea originally, and hopefully someone will find my minor additions and alterations useful.

AttachmentSize
BaseElementsPluginInstaller.zip4.8 MB

BaseElements Plugin Update

A quick note to let everyone know about a minor update to our free open source FileMaker plugin. This updates the version number to 1.2.1 and includes a fix for extra characters appearing at the end of the BE_ExecuteShellCommand function. This would have resulted in the commands not working or doing strange things.

In our case we are using this to launch AppleScripting of the FileMaker user interface in RefreshFM. So although it didn't show on our test or development machines we quickly started to see reports of this after launch. We'll have an updated RefreshFM version out very soon with the new plugin and some more features to boot.

As usual the downloads are available for Mac and Windows and the 1.2.1 source code is already available.

Syndicate content