Livecode Wiki
Register
Advertisement

Introduction[]

In an ideal world you would be able to solve any computing problem without leaving the confines of LiveCode. Unfortunately, though, there will always be things that very-high level programming is not suited for. For example, an implementation of an algorithm in LiveCode may be too slow, you may need a specific OS feature that isn’t wrapped by a command or function in LiveCode, or you may just want to access a pre-existing library in LiveCode.

To support such cases, LiveCode has what we call the externals interface. An external is simply a shared library (dll on Windows, bundle on Mac OS X) written in a lower-level language that can be loaded (at runtime) into the LiveCode environment. Once loaded, your scripting environment is augmented with the commands and functions exported by the external.

In this, the first of a series of articles, I will cover the basics of writing a (cross-platform) external for Windows and Mac OS X in C/C++.

Before we begin, before going any further you will need:


Note: The need for LiveCode 2.7.x is just because the External Creator assumes the new installation structure is 2.7 when setting up projects – the externals you build will operate in any LiveCode version and edition.

Setting up the environment[]

Before you can begin doing anything in a lower-level language, it is important that you set up your build environment. Modern IDEs such as Visual C++ and XCode make this straightforward when targetting a single platform – but it can be trickier if you want to use the same code base to compile on multiple platforms (or even multiple variants of the same platform such as exist with Mac OS X).

To help with this, a skeleton environment similar to the system we use internally is included. This skeleton has the basic structure allowing you to write externals using both Visual C++ 2005 and XCode 2.4.x.

After unpacking ExternalsEnvironmentV1.zip onto a suitable place on your hard-drive, it is worth spending a few moments familiarizing yourself with its contents:

Desktop externals 1.jpg


The contents you see here serve the following purposes:

  • External Creator V1.LiveCode – a small LiveCode utility to help you setup an external project
  • libexternal – this project folder contains the special glue-code that all externals need to be linked with in order to bind to LiveCode. (It is a project in its own right that builds as a static library).
  • templates – this folder contains template project files used by the External Creator to setup new external projects.
  • configurations – this folder contains two kinds of file vsprops and xcconfig. These contain settings for different variants of builds on different platforms.

In later sections we will refer to this folder as the environment – in addition to what it currently contains, it will also hold our external projects themselves.

Note: Although the environment folder will unpack to ExternalsEnvironmentV1, you are free to rename it to anything.

An aside on projects[]

Both Visual Studio and XCode rely on the idea of projects to organize development. A project defines the type of entity to be built along with all the build parameters and source-files needed to do the build.

In Visual Studio, projects are organized into solutions – a collection of projects that can reference each other to help organize dependancies. In our case, our solution will contain one project for each different external, all which depend on the project libexternal to ensure that the special glue-code that makes an external an external is available.

In XCode, on the other hand, there is no need for a solution. Any XCode project can reference another XCode project and depend on what it produces. However, in the end this amounts to the same thing as Visual Studio – although the XCode model is sligthly more flexible.

An aside on build configurations[]

One concept that will be new to you if you haven’t built projects in lower-level languages before will be that of build configuration. A build configuration is a collection of settings that define both the type of build and also the variant of the current platform to build for.

For example, lower-level languages typically require you to compile in a different way if you wish to debug your code (this is the Debug configuration in our case) or release your project (this is one of the Release configurations in our case).

Furthermore, the existence of single OSes targetting different architectures means that there are generally a multiplicity of Release builds needed. For example, for Mac OS X, we need ones for both PowerPC and Intel, as well as one for both (i.e. Universal).

However, you needn’t worry about all the details here as the skeleton environment comes with a collection of configurations that are completely compatible with LiveCode (i.e. they should just work!).

Our first external – working with text: a take on Hello,World![]

Setting up the project[]

The first thing to do is to create a skeleton project for our first external. Included with the skeleton environment is a LiveCode stack called External Creator. This little tool helps to start off creating an external by setting up the project (potentially for multiple platforms) including directory structure and build configurations. Furthermore, it creates an empty test stack to load the new external.

So, with the External Creator loaded into LiveCode do the following:

  • fill in the Name field with rnahello (we use the three-letter prefix rna to stand for LiveCode Newsletter Article)
  • check the appropriate platforms
  • choose C++ (no exceptions, no rtti) as the language
  • ensure the Installation paths are filled in correctly.

You should end up with something like:

Desktop externals 2.jpg

When you have configured all your settings, just click Generate.

Exploring our new project[]

Having generated the project, you should now find an rnahello folder within the environment. It is worth spending a few moments exploring this folder to see what has been created:

Desktop externals 3.jpg

Here we have the following:

  • rnahello.vsproj – the Visual Studio project file (present if you checked Windows)
  • rnahello.xcodeproj – the XCode project folder (present if you checks Mac OS X)
  • test_rnahello.LiveCode – an empty stack that will load the external (used for testing and debugging)
  • src/rnahello.cpp – the outline of our main C++ file that will contain the implementation of our commands and functions

So, now we have the structure in place we can actually add some functionality...

Getting started with the IDEs[]

Open up Visual C++ or XCode depending on the platform, and load either externals.sln (into Visual Studio), or rnahello.xcodeproj into XCode. In both cases you should be presented with (what will, at least, become) a familiar interface.

Both IDEs use a hierarchical approach to manage projects – in the case of XCode, the project itself will be the root node, whereas in Visual Studio the root node will be the solution:

Desktop externals 4.jpg
Desktop externals 5.jpg


You may notice in Visual Studio that one of the projects is highlighted in bold. This signifies which is the current Startup Project – the project that is built and run whenever you click the Run button. You can change the startup project by right-clicking on a project node and choosing Set as Startup Project – this will become particularly important for you if you add new externals to the environment.

My personal preference is to use XCode in all-in-one mode where you switch between Edit/Build/Debug by the little tab control in the top-left – this article reflects this preference. By default, however, XCode is not configured in this mode – to change this select the General pane in Preferences and choose All-in-one from the Layout option menu.

You may wish to spend some time looking around the IDEs and the project/solution structure we have created to familiarize yourself with them before going any further.

Adding functionality – our first function[]

At this stage, you should be looking at the rnahello project created with the External Creator. This project has all the appropriate build configurations configured as well as the necessary setup for launching the test stack on run so you can debug and/or see your external running – basically we have a skeleton external that will work, but not actually provide any functionality to the development environment.

In this section we will add an external function to our external. An external function is exactly the same as a function handler (defined by the function keyword) you will be familiar with in LiveCode, except that it’s implementation is in native code defined in an external. As with a function handler, an external function takes a number of parameters as input and returns a value as a result.

The External Creator will have created an outline main file rnahello.cpp which you may want to familiarize yourself with before continuing.

We will now add our first external function to our project – called rnahellouser. This will take one parameter (a name) and return a friendly message.

The external declarations block – describing the functionality the external provides[]

The external glue code provided in libexternal provides a number of C/C++ macros that help in the construction of the external declarations block that tells LiveCode what the external exports to it.

A block starts with:

EXTERNAL_BEGIN_DECLARATIONS(<external_name>)

This is followed by a sequence of command or functions declarations:

EXTERNAL_DECLARE_FUNCTION(<function_name>, <function_pointer>)

or

EXTERNAL_DECLARE_COMMAND(<command_name>, <command_pointer>)

Then the block finishes with:

EXTERNAL_END_DECLARATIONS

In our case we wish to declare the existence of an external function called rnahellouser. So, scroll to where the (empty) external declarations block is in the rnahello.cpp file and insert the line:

EXTERNAL_DECLARE_FUNCTION("rnahellouser", rnaHelloUser)

between the comments

// BEGIN USER DECLARATIONS

and

// END USER DECLARATIONS

This will cause a call to a function rnahellouser in LiveCode to be mapped to a call to the C++ function rnaHelloUser which we will implement next.

The function definition – implementing an external function[]

Now we have declared the existence of our external function, we now need to implement it. For any external handler (command or function), the C/C++ prototype is the same:

void handler(char *p_arguments[], int p_argument_count, char **r_result, Bool *r_pass, Bool *r_err)

Here:

  • p_arguments is an array of C-strings of length p_argument_count – each element of the array is an argument passed to the external handler.
  • r_result is a pointer to a C-string variable that should be set to a pointer to the result of the external handler. (The C-string returned in this way must be allocated using malloc/calloc/realloc and ownership of the memory will pass to LiveCode.)
  • r_pass is a pointer to a Bool variable that should be set to True if the handler invocation should be passed in the message path.
  • r_err is a pointer to a Bool variable that should be set to True if the invocation of this handler should raise a runtime error in LiveCode.

Add to the rnahello.cpp file between the

// BEGIN USER DEFINITIONS

and

// END USER DEFINITIONS

the following function:

// Function: rnaHelloUser(pName)
// Parameters: pName – string
// Result: a string containing a friendly message
void rnaHelloUser(char *p_arguments[], int p_argument_count, char **r_result, Bool *r_pass, Bool *r_err){ 
// First check we have been passed a single argument – if not it’s an error 
   if (p_argument_count != 1) {
      *r_result = strdup("wrong number of parameters");
      *r_err = True;
      *r_pass = False;
      return;
      }
   // Next compute the length of our required string = length of first argument (p_arguments[0]) + length of "Hello, !" + 1
   unsigned int t_buffer_length;
   char *t_buffer;
   t_buffer_length = strlen(p_arguments[0]) + 8 + 1;
   t_buffer = (char *)malloc(t_buffer_length);
   if (t_buffer == NULL){
       *r_result = strdup("out of memory");
       **r_err = True;
       *r_pass = False;
       return;
       }
   // We have allocated our buffer – so now construct our string
   sprintf(t_buffer, "Hello, %s!", p_arguments[0]);
   // t_buffer now contains a pointer to the result so just return...
   //
  *r_result = t_buffer;
  *r_err = False;
  *r_pass = False;
  }

In this function we have used a number of standard library functions which we will need to include in our environment. In C/C++ this is done via #include <revolution/external.h> directives, so add the following to the rnahello.cpp file before the #include directive.

#include <cstdlib>
#include <cstdio>
#include <cstring>

This imports memory management (cstdlib), i/o (cstdio) and string manipulation (cstring) functions into our file.

All being well, you should now be able to compile your external and launch it within LiveCode...

In Visual Studio:

  • Ensure rnahello is the Startup Project
  • Make sure Debug is chosen in the Solution Configurations drop-down list (on the toolbar)
  • Click the green Run button

Note: Due to the relaunch feature added to recent versions of LiveCode, you need to make sure the version of LiveCode you are debugging externals in is not already running when you launch a debug session from Visual Studio

Note: You will likely get a message saying Unable to find debug symbols for LiveCode.exe the first time you run your application – do not worry about this, just click to continue and choose not to show the dialog again.

In XCode:

  • Choose Build mode (either by choosing Build Results from the Build menu, or by clicking on the Build icon in the Page tab in the in the top-left).
  • Choose rnaHello from the Target drop-down
  • Choose Release from the Configuration drop-down
  • Choose Build
  • Now choose the Debug configuration and click Build and when it has finished click Debug. (The pLiveCodeious two steps shouldn’t be necessary, but there seems to be a glitch in XCode that pLiveCodeents it picking up the correct settings for launching the debug executable if you have only ever built a Debug variant of a project – from now on you won’t need to build a release variant before a debug one in this project).

Assuming the above steps were followed correctly, after a short while an empty stack called rnaHelloTest should pop-up inside the development environment.

Testing our function – hooking into our external[]

At this point you should be looking at the LiveCode IDE with a stack opened up. This stack should have already loaded the external into the LiveCode environment – so now we just need to hook into it.

So do the following:

  • First add a field called Name to the stack
  • Add a button called rnaHelloUser to the stack and place the following into its script:
on mouseUp
   answer rnaHelloUser(field "Name")
end mouseUp

Now enter any string into the field and click the button – you should get an answer dialog popping up with a friendly message.

Before continuing, make sure you save your test stack (we will be building on it in the next section) and then quit LiveCode.

Adding functionality – our first command[]

In the previous section, we looked at adding an external function. In this section we will look at adding an external command instead. Just with external functions, external commands are effectively the same as command handlers (defined with the on keyword) in LiveCode except that their implementation resides in a external.

In reality, there is very little difference between external commands and functions (just like there is little difference between command and function handlers in LiveCode). Indeed, the difference is mainly syntactic, and the fact that you have to use the result to get the return value of a previously executed command. Therefore, we will use this opportunity to introduce how to set and get local variables in an external handler.

The command we will implement will be called rnaHelloUserIndirect and will have syntax:

rnaHelloUserIndirect pUserVariableName, pOutputVariableName

Here pUserVariableName will be the name of a (handler) local variable from which to fetch the user’s name, and pOutputVariableName will be the name of a (handler) local variable in which to put the result.

The LiveCode externals API – getting and setting string variables[]

In addition to providing the necessary ‘glue’ to allow a shared library to be loaded as a LiveCode external, ‘libexternal’ also exports a number of functions that allow some access to the internals of the currently running LiveCode application.

In this section will make use of two of these functions GetVariable and SetVariable. These functions give you access to the contents of the local variables present in the context that called the external handler – as long as the value of those variables are text strings (i.e. not binary data).

The prototypes of these functions are:

char GetVariable(const char p_variable_name, int r_success)
void SetVariable(const char p_variable, const char p_value, int r_success)

The GetVariable call takes a name of a variable (as a C-string) and a pointer to a return variable. If the call succeeds, *r_success will be EXTERNAL_SUCCESS and a copy of the value of the variable as a C-string will be returned. If the call fails, *r_success will contain EXTERNAL_FAILURE and NULL will be returned.

Note: The ownership of the memory returned passes to you, i.e. you must free it when you are finished with it.

The SetVariable call takes a name of a variable, the value you wish to set the variable to and a return variable. If the call succeeds *r_success will contain EXTERNAL_SUCCESS and the specified variable will be set to the required value. If the call fails *r_success will contain EXTERNAL_FAILURE and no other change will have occurred.

Note: LiveCode copies the string you pass to it in p_value so there is no concern about ownership – i.e. if you’ve allocated memory to store it, you still have to free it.

Implementing our new command[]

As before, we first have to declare the existence our new command by adding an appropriate entry in the external declarations block. Therefore, add the following line in the ‘USER DECLARATIONS’ as before:

EXTERNAL_DECLARE_COMMAND("rnahellouserindirect",rnaHelloUserIndirect)

Next we have to provide an implementation of this command which we do by adding the following to the ‘USER HANDLER DEFINITIONS’ section of the file:

// Command: rnaHelloUserIndirect pUserVariableName, pOutputVariableName
// Parameters: 
// pUserVariableName – name of a variable in local context to get the name

from

// pOutputVariableName – name of a variable in local context to put the

result into

void rnaHelloUserIndirect(char *p_arguments[], int p_argument_count, char **r_result, Bool *r_pass, Bool *r_err) {
   // In this command we have to keep track of several (memory) resources. To help with this
   // we use a variable 't_error' that determines whether an error has occured. This is used
   // throughout the function to determine whether to continue processing, and at the end to 
   // decide whether to return an error or not.
   const char *t_error;
   t_error = NULL;
   // Check to see if we have been passed two parameters  
   if (t_error == NULL && p_argument_count != 2) {
          t_error = "wrong number of parameters";
          }
   // Fetch the value of the 'name' variable (argument 1)
   // Remember that the return value of 'GetVariable' becomes 'ours' so we have to
   // free it later if the call succeeds.   
   char *t_name;
   t_name = NULL;
   if (t_error == NULL) {
      int t_success;
      t_name = GetVariable(p_arguments[0], &t_success);
      if (t_success == EXTERNAL_FAILURE) {t_error = "unable to get value of name variable";}
      }
   // Allocate memory for the return message to be put into the output variable
   // (If this step succeeds, we need to free memory pointed to by t_message later)
   char *t_message;
   t_message = NULL;
   if (t_error == NULL){
       t_message = (char *)malloc(strlen(t_name) + 8 + 1);
       if (t_message == NULL){t_error = "out of memory";}
       }
   // Format our message appropriately 
   if (t_error == NULL){sprintf(t_message, "Hello, %s!", t_name);}
   // Set the value of the output variable
   // Remember that the ‘SetVariable’ API call copies the contents of t_message and so we still
   // have to remember to free it ourselves.
   if (t_error == NULL){
      int t_success;
      SetVariable(p_arguments[1], t_message, &t_success);
      if (t_success == EXTERNAL_FAILURE) { t_error = "unable to set output variable";}
      }
   // Free the message memory buffer if it was ever allocated
   if (t_message != NULL) {free(t_message);}
   // Free the name memory buffer if it was ever allocated
   if (t_name != NULL) {free(t_name);}
   // If we succeeded, return an empty result and no error, otherwise return a copy
   // of the error message and flag as an error.
   // (NB: We return a pointer to an empty string here because pre-2.5 versions of LiveCode
   // would crash if you returned a NULL pointer in r_result)
   if (t_error == NULL){
      *r_result = strdup("");
      *r_pass = False;
      *r_err = False;
      } else {
      *r_result = strdup(t_error);
      *r_pass = False;
      *r_err = True;
      }
   }

Again, you should now be able to Run (Visual Studio), or Build and Debug (XCode) your project and after a few moments, the test stack will appear again.

Testing our command[]

You should again find yourself staring at our test stack, loaded into the LiveCode IDE. Add a new button to the stack called rnaHelloUserIndirect and place the following in its script:

on mouseUp
  local tName, tOutput 
  put field "Name" into tName 
  rnaHelloUserIndirect "tName", "tOutput" 
  answer tOutput
end mouseUp

Notice here that we are passing strings to rnaHelloUserIndirect that contain the names of the variables that we wish to access in the external, rather than their values.

Clicking on rnaHelloUserIndirect should now give you exactly the same message as clicking rnaHelloUser – our new command works!

Integrating an external into the IDE[]

The External Creator helpfully builds us a stack that automatically loads our external when it is opened through either XCode or Visual C++. However, when you get to a point when you wish to start testing any externals you create in the context of actual LiveCode projects, you will need to be able to integrate them with the IDE.

To do this, you will first want to build Release builds of your externals. In Visual Studio, simply choose the Release configuration from the drop-down list on the toolbar, while in XCode you need to switch to the build pane and choose one of Release, Release x86-32 or Release PowerPC-32.

When you have done this, all you need to do then is copy them into appropriate places in your Documents folder for LiveCode to pick up next time it is launched. This can be done by creating the following hierarchy inside the standard Documents folder:

  • My LiveCode /
    • Externals/
      • Runtime/
        • Windows/
          • x86-32/
            • Externals/
        • Mac OS X/
          • Universal/
            • Externals/
          • PowerPC-32/
            • Externals/
          • x86-32/
            • Externals/

Then inside each Externals folder, put the appropriate build of the external along with a text file called Externals.txt. The externals.txt file is return-delimited list of pairs:

<external label>,<external library>

For example for rnahello on Windows the file should contains:

Hello External,rnahello.dll

Whereas on Mac OS X it should contain:

Hello External,rnahello.bundle

Note: You only need the Runtime hierarchy for building Standalones containing your externals. Furthermore, you need only create the architecture folders within each platform folder for the architectures that you need.

Note: In general, you should always build a Release (Universal) external on Mac OS X to place in the non-Runtime Externals folder – as this is the folder the IDE uses.

VariableEx calls – the Swiss-army knife of the externals API[]

Now you have to use:

In our rnaHello external we created earlier, we saw how to use the API calls GetVariable and SetVariable. These calls are fine if you wish to manipulate small amounts of text data, but because they operate using NUL-terminated strings, they are completely useless if you want to work with binary data. Furthermore, they give no way of accessing variables that might be arrays. Therfore, to deal with such cases, it is necessary to become familiar with a related pair of functions GetVariableEx and SetVariableEx.

This pair of functions have three advantages over their simpler cousins:

  • The value you pass in and out is a pair consisting of a data pointer and a length – i.e. there is no reliance on a NUL-terminator to determine the length of the data.
  • You can specify a key, enabling you to access values of individual elements of the array.
  • The data returned by GetVariableEx is not copied, resulting in a performance gain when fetching large blocks of data (and you don’t have to worry about freeing anything).

The syntax of these functions is a natural extension of GetVariable and SetVariable:

void GetVariableEx(const char p_name, const char p_key, ExternalString r_value, int r_success)
void SetVariableEx(const char p_name, const char p_key, const ExternalString p_value, int r_success)

Notice here that there is an extra p_key parameter and rather than the value being a char *, it is now an ExternalString *.

Manipulation of binary data[]

As mentioned above, one of the advantages of this pair of functions is that they allow you to work with arbitrary binary data. This comes about because a value is passed and returned as a pointer to an ExternalString structure. This structure is defined with two members:

struct ExternalString {
   const char *buffer;
   int length;
   }

Where buffer is a pointer to the data, and length is its length in bytes. For example to get the value of a variable called tBinaryData one would do something like:

ExternalString t_value;
GetVariableEx("tBinaryData", "", &t_value, &t_success);

And to set the value of a variable called tBinaryData to some pre-existing value, one would do something like:

ExternalString t_value;
t_value . buffer = (const char *)t_my_buffer;
t_value . length = t_my_buffer_length;
SetVariableEx("tBinaryData", "", &t_value, &t_success);

Here, you will notice we pass the empty string ("") as the second argument – the p_key parameter. This tells the engine to return the value of the given variable as if it were a normal (non-array) variable.

One thing to be wary of when using these forms is that the data you get back by using GetVariableEx will not (in general) be NUL terminated, even if it is ‘text’ in some sense. Importantly this means you should take great care when using the standard C string manipulation routines.

Delving into arrays[]

The other main use of VariableEx function pair is to access elements of a LiveCode array variable. To do this, simply pass the name of the key you wish to access in the p_key parameter. For example, if you want to get element foo of variable tBar one would do something like:

ExternalString t_value;
GetVariableEx("tBar", "foo", &t_value, &t_success);

Or if you want to get element n of variable tBaz, you would need to do something akin to:

char t_key[16];
sprintf(t_key, "%d", n);
GetVariableEx("tBaz", t_key, &t_value, &t_success);

Here you are seeing a side-effect of LiveCode arrays really being hash-tables with string-based keys – even numeric based arrays require you to explicitly convert the index to a string before fetching.

One question you might be caused to ask is – what happens if I try to get or set an element that doesn’t exist? The answer is exactly the same as what happens when you try to do that in LiveCode: in the case of setting, the element gets created; in the case of getting you get empty back (in this case empty is a buffer of length 0). In particular, r_success only reports EXTERNAL_FAILURE if the variable itself does not exist.

A word of caution[]

As the third benefit of the VariableEx calls, we mentioned the increased efficiency gained by the data not being copied when the external fetches it from LiveCode. This is all well and good, but does come with a small word of warning: any changes to the variable, or the referenced variable’s element will cause the pointer that is returned to become invalid.

For example, the following code is likely to cause a crash:

ExternalString t_original_value;
GetVariableEx("tBar", "", &t_original_value, &t_success);
SetVariable("tBar", "Hello World!", &t_success);
if (t_original_value . length == 5 && memcmp(t_original_value . buffer,"hello", 5) == 0) {printf("It starts with hello");}

This is because the engine will free the memory referenced in t_original_value during the course of the SetVariable call, and so when a pointer to it is used in the memcmp call there is no guarantee it will still be valid.

However, don’t get too concerned about this. If you keep to the general principal of fetching input from input variables, doing the necessary processing and then writing back the results to the output variables it is unlikely that you will end up with any dangling pointers.

Case-Study: an image-effect framework[]

So far this article has been somewhat dry and it is about time we got our hands dirty. Rather than produce a small and essentially useless external just to exhibit the introduced APIs, we will instead explore a slightly more meaty example of an external.

Background[]

What we will be producing in the remainder of this article is a basic image effect framework – an external with a simple API allowing you to apply effects to the image and alpha data of images. While simple, the framework has been designed to be easily extensible, requiring little more than implementing the algorithm that actually implements the effect you want.

The framework exports just two external handlers:

  • rnaEffectList – an external function returning a list of all known effects
  • rnaEffectApply – an external command taking details of an effect and an input image, and outputing a processed image

The external makes extensive use of both the array and binary manipulation features of the externals API. Indeed, from the point of view of LiveCode, effects and images are defined by arrays having certain sets of keys. The external then uses these arrays to dispatch to the appropriate effect and provide it with the correct data.

The framework is written using C++ and makes use of both abstract base classes for extensibility, and exceptions for implicit error-handling.

Getting Started[]

Before continuing we need to setup a new external. So, make sure you have an unpacked Externals Environment (the one from the last article will do) and load up the External Creator. Then, generate a new external with the following settings:

  • Name – rnaEffect
  • Platform – as appropriate
  • Language – C++

When this has been generated, unpack the NewsletterArticle2.zip archive and copy the source files from within the rnaeffect_skeleton/src folder into the rnaEffect/src folder inside your externals environment. The files you have just copied is the framework itself albeit without any implemented effects. (Don’t worry about overwriting the ‘rnaeffect.cpp’ file, you want the one from the archive – it actually does something!).

Now load the new project into XCode or Visual C++, depending on your current platform.

The next thing to do is to add the axillary files to the project. This can be done in one of several ways – the easiest is probably by using drag-drop. Select all the source files (both CPP and H) in the src folder and drag them onto the project.

In Visual C++, you should drag them onto the rnaEffect project node. In XCode, drag them into the Source node under the rnaEffect node and just choose Add when you get prompted with a dialog asking you which targets to add them to. Be careful you don’t get a duplicate rnaeffect.cpp node – if you do, just choose one, select Edit -> Delete, and then Delete References. These new files serve the following purposes:

  • rnaeffect.cpp – this contains the main implementation of the external, containing two external handler definitions
  • effect.cpp/effect.h – this pair of files define an abstract class Effect which you can inherit from in order to implement an effect
  • variable.cpp/variable.h – this pair of files define an abstraction of a Revolution variable making them easier to work with (see the next section)
  • utility.cpp/utility.h – this pair of files declare a number of utility functions and classes making development easier development.

It is probably now worth taking a brief look through the files in the project. Most source files have been heavily commented to help explain what they are for. In particular have a look through the implementation of the external handlers in rnaeffect.cpp.

Abstracting variable access[]

If you have taken a look through rnaeffect.cpp then you will notice that there is not a single mention of SetVariableEx or GetVariableEx. Indeed, having introduced our friends GetVariableEx and SetVariableEx, they have now been rendered obsolete by wrapping them in an abstraction – the Variable class. At this point take a moment to browse through the Variable.h header file.

The idea of this class is simple – you declare a variable of type Variable and pass a variable name to its constructor. This object is then (notionally) linked to the Livecode variable and you can use a rich collection of methods to manipulate its value.

For example, to fetch the value of a Revolution variable call tBar as an integer you would do:

Variable t_bar("tBar");
int t_value;
t_value = t_bar . GetInteger();

Or, to set the value of element foo of variable tBaz to the string “Hello World!” you would do:

Variable t_baz("tBaz");
t_baz . SetCStringElement("foo", "Hello World!");

As you can see, this significantly simplifies fetching and storing variable values particularly as any required type conversions are done for you by the wrapper class.

One point worth mentioning is that this version of the Variable class relies on exceptions to handle errors. i.e. If a variable doesn’t exist, or doesn’t contain a string that can be coerced to the requested type an exception will be thrown.

Being just a ‘wrapper’ class, Variable’s implementation is straightforward and those of you interested in the details may want to take a good look through the Variable.cpp class to see how it works. (It really is nothing more than a thin wrapper around the SetVariableEx and GetVariableEx calls).

Introducing effects[]

An effect is a class derived from the abstract class Effect. The Effect class has two pure virtual methods – Calculate and Apply – that must be implemented. In addition, a means by which to create an instance of the effect from a Revolution array must be provided.

The Calculate method is used by the framework to work out how big an output image will be given an input image of a given size. It has signature:

void Calculate(unsigned in p_in_width, unsigned int p_in_height,unsigned int& r_out_width, unsigned int& r_out_height)

The implementation must use p_in_width and p_in_height together with any parameters passed to the effect when it was created to compute the output size which should then be stored in r_out_width and r_out_height.

The Apply method is the workhorse of any effect, actually performing the effect’s operation on an input image and putting the output into an output image. It has signature:

void Apply(const Image& p_input, Image& p_output)

From the point of view of the framework, an Image is a quadruple:

struct Image {
   unsigned int width;
   unsigned int height;
   unsigned char color;
   unsigned char alpha;
   };

A structure containing the images width and height in pixels, together with pointers to memory buffers containing the color (pixel) and alpha (mask) data. These data buffers are in the same format as the imageData and alphaData properties of a Livecode image.

The framework takes care of fetching the input data and allocating the output buffers, so all the Apply method needs to do is take the input and write it to the output after it has applied its magic.

Our first effect[]

Our first effect will be the ‘no-op’ effect. It will take an input image and map it directly to the output image without changing it in any way. While not actually useful, it is the simplest possible effect we could implement and will illustrate how to add an effect perfectly.

The first thing we need to do is add the definition and implementation of the NopEffect class to our project.

In Visual C++, click the right mouse-button on the rnaeffect node of the project and choose Add -> New Item…. Choose ‘C++ file (.cpp)’, give it the name nop_effect.cpp, and make sure it is being created in the src folder (check the Location field). Repeat this again but choose ‘Header file (.h)’ and give it the name nop_effect.h.

In XCode, right click on the Source node in the project, choose New File… and select Empty File in Project. Give it the name nop_effect.cpp and make sure it is created in the src folder. Repeat this again but add a file called nop_effect.h. (In both cases, you do want it added to the rnaeffect target – which will be the default setting).

Open up the nop_effect.h file and paste in the following:

#ifndef __NOP_EFFECT__
#define __NOP_EFFECT__
#ifndef __EFFECT__
#include "effect.h"
#endif
class NopEffect : public Effect {
   public:
      NopEffect(const char *p_info_variable);
      void Calculate(unsigned int p_in_width, unsigned int p_in_height,unsigned int& p_out_width, unsigned int& p_out_height);
      void Apply(const Image& p_input, Image& p_output);
   }
#endif

This is nothing more than the declaration of our new class, derived from Effect.

Now in the nop_effect.cpp we need to define three methods – but first we need to ensure we include the relevant definitions. At the top of the file place the following includes:

#include <cstring>
#include "effect.h"
#include "nop_effect.h"

Follow this by the method implementations. First the constructor:

NopEffect::NopEffect(const char *p_info_variable){}

Since our effect does nothing to any given image, it has no parameters and so the constructor need not do anything.

Next, we need the Calculate method:

void NopEffect::Calculate(unsigned int p_in_width,unsigned int p_in_height,unsigned int& p_out_width, unsigned int& p_out_height){
  p_out_width = p_in_width;
  p_out_height = p_in_height;
  }

Again, our effect has no effect, so we just copy the input width and height into the output width and height.

Finally, the main part – the Apply method:

void NopEffect::Apply(const Image& p_input, Image& p_output){
   if (p_input . color != NULL) {
       memcpy(p_output . color, p_input . color, p_input . width  p_input . height  4);
      }
   if (p_input . alpha != NULL){
      memcpy(p_output . alpha, p_input . alpha, p_input . width * p_input . height);
      }

Again, our effect does no processing so we just copy the input data to the output data. Here it is worth noting two things. First, if either color or alpha data is not present in the input (indicated by the buffer being NULL), it cannot be present in the output. Second, the size of the buffers are one byte per pixel for alpha, and four bytes per pixel for color – just the same as the corresponding data properties of images in Livecode.

Having implemented the NopEffect class, all that remains is to hook it into the framework. To do this, open up the effect.cpp file and locate the // BEGIN USER EFFECT INCLUDES line. After this line put:

#include "nop_effect.h"

This will cause the declaration of our new effect to be imported. Then find the // BEGIN USER EFFECTS line and after it put the entry:

{ "nop", EffectCreate },

This line adds an entry to an array that tells the framework to create an instance of the NopEffect class whenever an effect with name nop is requested.

Having done all of this, click Run (Visual C++), or Build followed by Debug (XCode) and, all being well, an empty test stack should appear.

In Visual C++, make sure that rnaeffect is selected as your ‘Startup project’. Remember that in XCode, you need to do a release build before a debug build the first time a project is compiled – otherwise when you debug, the paths will be wrong and the external won’t get loaded.

Using the framework[]

You should now be looking at a blank stack ready to start testing our effects framework and new effect. First of all, let’s check that our effect is recognized.

Switch to the Message Box and ensure that the rnaEffectTest stack is the current target. Then execute put rnaListEffects(). You should see a list consisting of one item nop (if you don’t, retrace your steps and ensure that both nop_effect.cpp and nop_effect.h are in the project, and that the s_effects array in effect.cpp has a ‘nop’ entry).

As mentioned before, the framework works by using arrays to communicate. The rnaApplyEffect external command takes three parameters:

  • pEffectInfoName – the name of an array variable describing the required effect
  • pInputImageName – the name of an array variable describing the input image
  • pOutputImageName – the name of a variable to receive the output image description

The keys required in the effect array will depend on the effect, but at the very least there should be one key call name containing the name of the effect to invoke.

The array describing the input image requires the following keys:

  • width – the width of the image in pixels
  • height – the height of the image in pixels
  • color – the imageData of the image (optional)
  • alpha – the alphaData of the image (optional)

Similarly, the output image variable will contain an array with similar keys.

So to try out the nop effect, import a suitable image onto the test stack – any image will do – and call it Input. Then add an additional empty image object (by dragging from the tools palette) and call it Output. Finally, add a button called Nop with the following script:

on mouseUp
  local tEffect 
  put "nop" into tEffect["name"] 
  local tInput 
  put the width of image "Input" into tInput["width"] 
  put the height of image "Input" into tInput["height"] 
  put the alphaData of image "Input" into tInput["alpha"] 
  put the imageData of image "Input" into tInput["color"]
  local tOutput 
  rnaApplyEffect "tEffect", "tInput", "tOutput"
  lock screen 
  set the width of image "Output" to tOutput["width"] 
  set the height of image "Output" to tOutput["height"] 
  set the alphaData of image "Output" to tOutput["alpha"] 
  set the imageData of image "Output" to tOutput["color"] 
  unlock screen
end mouseUp

Clicking the button should result in the output image turning out to be identical to the input image! At this point save the test stack and quit Revolution.

An image adjustment effect[]

Now having implemented a completely useless effect, let us build something slightly more interesting – a simple implementation of contrast and brightness adjustment. This effect will be called adjust and take two parameters brightness and contrast.

Add a pair of files to the project adjust_effect.cpp and adjust_effect.h.

Next, copy the following into the adjust_effect.h

#ifndef __ADJUST_EFFECT__
#define __ADJUST_EFFECT__
#ifndef __EFFECT__
#include "effect.h"
#endif
class AdjustEffect: public Effect {
   public:
      AdjustEffect(const char *p_info_variable);
      void Calculate(unsigned int p_in_width, unsigned int p_in_height,unsigned int& p_out_width, unsigned int& p_out_height);
      void Apply(const Image& p_input, Image& p_output);
   private:
      int m_brightness;
      int m_contrast;
    };
#endif

This defines another derivation of Effect. This time, though it has some state – m_brightness and m_contrast. These two integral values will be extracted from the effect array when the an AdjustEffect object is created.

Next open up the adjust_effect.cpp file and start off by adding the following includes:

#include <cstring>
#include "variable.h"
#include "effect.h"
#include "adjust_effect.h"

After this we need our three methods. First the constructor:

AdjustEffect::AdjustEffect(const char *p_info_variable){
   Variable t_info(p_info_variable);
   m_contrast = t_info . GetIntegerElement("contrast");
   m_brightness = t_info . GetIntegerElement("brightness");
   }

Here, we first wrap the info variable name in a Variable object, then fetch two integer values from the keys contrast and brightness. These values are stored in the private state of the AdjustEffect object so the other two methods have access to them.

Next, we need the Calculate method:

void AdjustEffect::Calculate(unsigned int p_in_width, unsigned int p_in_height, unsigned int& p_out_width, unsigned int& p_out_height) {
   p_out_width = p_in_width;
   p_out_height = p_in_height;
   }

As before, this effect does not change the size of the image and so we just copy input size to output size.

Finally, we need the implementation. As previously mentioned, the image data is made available to the Apply method as (up to) two pointers to memory buffers. These buffers contain data in exactly the same format as Revolution image and alpha data properties – that is one byte-per-pixel for alpha data and four bytes-per-pixel for image data (the latter being in sequence pad, red, green, blue for each pixel). Therefore, applying an effect is a simple matter of looping over the input data in an appropriate way, and then writing the processed data to the provided output buffer.

In this case, we will be applying the following function to the red, green and blue components of each pixel:

new_value = min(0, max(255, ((value - 128) * contrast) / 128 + 128 + brightness))

This is nothing more than a simple-minded contrast and brightness adjustment – common in many image-processing applications.

Therefore, we can use something like the following to achieve our desired effect:

void AdjustEffect::Apply(const Image& p_input, Image& p_output){
   // Process the color data
   if (p_input . color != NULL) {
      const unsigned char *t_in_ptr;
      t_in_ptr = p_input . color;
      unsigned char *t_out_ptr;
      t_out_ptr = p_output . color;
      // Since this effect is not sensitive to row or column, we just iterate
      // through each pixel without regard for its x or y location.
      for(unsigned int p = 0; p < p_input . width * p_input . height; ++p) {
         // Skip the pad byte in both input and output
         t_in_ptr++;
         t_out_ptr++;
         // Since this effect is not sensitive to channel, we just iterate
         // through 3 channels for each pixel - red, green and blue.
         for(unsigned int c = 0; c < 3; ++c) {
            int t_in_value;
            t_in_value = *t_in_ptr++;
            int t_out_value;
            t_out_value = ((t_in_value - 128) * m_contrast) / 128 + 128 + m_brightness;
            if (t_out_value < 0) {
               t_out_value = 0;
               } else if (t_out_value > 255) {
               t_out_value = 255;
               }
           *t_out_ptr++ = t_out_value;
           }
        }
    }
   // We have no effect on the alpha data, so we just copy it straight
   // through.
   if (p_input . alpha != NULL) {
      memcpy(p_output . alpha, p_input . alpha, p_input . width * p_input . height);
      }
   }

With the adjust_effect.cpp file finished, all that remains is to add:

#include "adjust_effect.h"

to the appropriate place in effect.cpp. Followed by an entry in the effects table in the same file:

{"adjust", EffectCreate },

Now rebuild and run the project.

Using the adjust effect[]

Let us now hook into our new effect.

First of all, to the test stack add two sliders: one Brightness with range -127 to 127; and one Contrast with range 0 to 255. Then create a new button Adjust and give it the following script:

on mouseUp
  local tEffect 
  put "adjust" into tEffect["name"] 
  put the thumbPosition of scrollbar "Brightness" into tEffect["brightness"] 
  put the thumbPosition of scrollbar "Contrast" into tEffect["contrast"]
  local tInput 
  put the width of image "Input" into tInput["width"] 
  put the height of image "Input" into tInput["height"] 
  put the alphaData of image "Input" into tInput["alpha"] 
  put the imageData of image "Input" into tInput["color"] 
  local tOutput 
  rnaApplyEffect "tEffect", "tInput", "tOutput" 
  lock screen 
  set the width of image "Output" to tOutput["width"] 
  set the height of image "Output" to tOutput["height"] 
  set the alphaData of image "Output" to tOutput["alpha"] 
  set the imageData of image "Output" to tOutput["color"] 
  unlock screen
end mouseUp

Now try playing around with the sliders and clicking Adjust – you should see a familiar effect!

Building on the framework[]

The final code for the framework and effects decribed here can be found in the rnaeffect_final folder in the NewsletterArticle2.zip archive.


FAQ[]

Where can I get a copy of Visual C++ 2005?[]

Visual C++ 2005 is available in several editions as part of the Visual Studio 2005 software suite. Microsoft makes Visual C++ 2005 Express Edition available free of charge, and is available here: http://msdn.microsoft.com/vstudio/express/

Where can I get a copy of XCode 2.4.x?[]

The XCode tools are freely available from Apple’s developer site after registering for Apple’s Developer Connection (also free). The landing page is here: https://developer.apple.com/xcode/

Do I really have to wait for the IDE to load each time I want to test my external?[]

No. If you are on a slow machine, you may want to setup your test stack and build it as a standalone. You can then change the debugger parameters in either XCode or Visual Studio to run this executable rather than the IDE.

For Mac OS X: Navigate to the Executables branch of the project tree and Get Infoon the Test node. Change Executable Path to reference your standalone application. Go to the Arguments tab and remove all the arguments.

For Windows: Right-click on the appropriate project in the Solution tree-view and select Properties. Choose All Configurations from the drop-down list Expand Configuration Properties and choose Debugging. Select Command, choose Browse and select your test standalone executable. Clear the command arguments and working directory fields.

Can I reuse the source-code provided with this article in my own projects?[]

Yes. You are free to use all source-code and project files included with this article (or generated by External Creator V1) for the purposes of producing LiveCode compatible externals.

libexternal seems to be different from the old XCmdGlue implementation I have...[]

Yes – this is correct. libexternal is an updated version of the glue-code which is cleaner. It uses exactly the same LiveCode interface as the pLiveCodeious implementation and so will continue to produce externals that will be usable in any LiveCode version.

What’s the difference between C++ (no exceptions, no rtti) and C++?[]

Many people use C++ as ‘just a better C’ – and don’t take advantage of exception handling or runtime-type information. Used in this form, there is no need for the external glue-code to do anything special. However, if you wish to use C++ exceptions, it is important the glue code automatically handles any before control is returned to the engine and this option does this automatically for you.

Can I use the standard C++ library?[]

Yes – but make sure you choose C++ when creating the skeleton for your external. Most recent implementations of the standard C++ library require exception handling (Visual Studio for example) and are not guaranteed to work correctly if it is disabled. Creating your external with initial support for C++ will ensure that unhandled exceptions in your external code are caught before returning to LiveCode.

Can I write externals in other languages?[]

In theory any language that can build shared libraries and allows you to link with the provided glue code could be used. However, in practice, some languages will require extra glue-code in order to provide an environment in which their code can run. For example, to access Objective-C you will need to ensure you set up its memory/exception environment in C wrapper functions before calling any Objective-C code.

Can I build externals that run on Windows 98SE/ME?[]

Yes. Any external that does not depend on any OS APIs should run fine on the Windows 98/ME family of OSes. If you do you use any OS APIs you just need to make sure they are available on that platform by checking in the Platform SDK documentation (since you can’t explicity target a given Windows version, you won’t find out at compile-time if you’ve accidentally used an API that isn’t available).

Can I build externals with other versions of Visual C++?[]

Yes – but the External Creator only generates project and solution files compatible with the Visual Studio 2005 family.

Can I build externals with other Windows C/C++ development environments?[]

Yes. Externals are just DLLs which export a specific function and so any compiler should be persuadable to produce externals compatible with LiveCode. Of course, you will have to set up your own environment in this case.

Note: the only requirement that LiveCode puts on a DLL it trys to load as an external is that it exports one symbol whose resulting name is _getXtable. Some compilers don’t automatically prepend an ‘_’ but with appropriate options can be made to do so. (getXtable is defined in libexternal).

Can I build externals with other versions of XCode?[]

In theory, yes. In practice, no. With the advent of Universal Binary you really need to be producing externals that compile to both PowerPC and Intel – this requires a minimum of XCode 2.2 (if I recall correctly). Also, if you wish to support the same range of OS versions that LiveCode supports you need to build them in a specific way:

  • PowerPC with gcc-3.3 against the 10.2.x SDK
  • Intel with gcc-4.0.1 against the 10.4u SDK

We use XCode 2.4.1 internally and know these configurations work without problems and so we recommend that you do the same.

Of course, if you are targetting a specific OS version or architecture, then you can choose your version of XCode appropriately, but you may have to setup your environment from scratch.

Can I build externals with other Mac OS X C/C++ development environments?[]

This depends on the OS versions and architectures that you are targetting. Any environment that is based on the Apple GNU toolchains should be able to produce compatible shared libraries. However, if they are not based on these toolchains, your mileage will vary.

In all these cases, though, you will have to configure your build environment yourself from scratch.

Can I use the same environment folder to build on both Windows and Mac OS X?[]

Yes. As long as you checked both platform boxes in the ‘External Creator’ when you initially setup your external a project will have been created for both platforms. To save constantly moving the folder between the two platforms you could (for example) put the environment on a network share. However, I recommend that you look into setting up a version control system such as Subversion to manage your external projects – this isn’t at all hard to do and numerous tutorials and GUIs exist on the web to help.

I’ve setup a version control system, what files should I include?[]

As a general rule, you should only include ‘source’ files in the version control system – these are the files that cannot be derived from other files. So, in our case you will need:

  • The top-level folder
  • Each project folder
  • The configurations folder and its contents
  • The externals.sln file (if you are targetting Windows)
  • The src folders and their contents within each project folder
  • The vcproj file within each project folder
  • The vcproj.user file within each project folder
  • The xcodeproj folder and the project.pbxproj file within it (Mac OS X build only)
  • The plist file within each project folder (Mac OS X build only)
  • The LiveCode test stack within each project folder
  • Any other files that you have explicitly added

You should not add the _build or _cache folders, nor any ‘hidden’ files or .ncb files on Windows. Furthermore, all the files within the xcodeproj folder apart from project.pbxproj should be ignored.

The general principle is this – if a file or folder was not created by you, or the External Creator it shouldn’t go under version control!

Advertisement