Reference for interfacing MD codes with PLUMED

Plumed.h and Plumed.c contain the external plumed interface, which is used to integrate it with MD engines. This interface is very general, and is expected not to change across plumed versions. Plumed.c also implements a dummy version of the interface, so as to allow a code to be fully linked even if the plumed library is not available yet. These files could be directly included in the official host MD distribution. In this manner, it will be sufficient to link the plumed library at link time (on all systems) or directly at runtime (on systems where dynamic loading is enabled) to include plumed features.

Notice that in PLUMED 2.5 this interface has been rewritten in order to allow more debugging features and a better behavior in multithread environments. The interface is almost perfectly backward compatible, although it implements a few additional functions. See more details below.

A further improvement has been made in PLUMED 2.8, where the interface has been modified to allow dynamic type checking. See more details below.

Why is Plumed.c written in C and not C++? The reason is that the resulting Plumed.o needs to be linked with the host MD code immediately (whereas the rest of plumed could be linked a posteriori). Imagine the MD code is written in FORTRAN: when we link the Plumed.o file we would like not to need any C++ library linked. In this manner, we do not need to know which C++ compiler will be used to compile plumed. The C++ library is only linked to the "rest" of plumed, which actually uses it. Anyway, Plumed.c is written in such a manner to allow its compilation also in C++ (C++ is a bit stricter than C). This will allow e.g. MD codes written in C++ to just incorporate Plumed.c (maybe renamed into Plumed.cpp), without the need of configuring a plain C compiler.

Plumed interface can be used from C, C++ and FORTRAN. Everything concerning plumed is hidden inside a single object type, which is described in C by a structure (struct plumed), in C++ by a class (PLMD::Plumed) and in FORTRAN by a fixed-length string (CHARACTER(LEN=32)). Obviously C++ can use both struct and class interfaces, but the second should be preferred since it will automatically take care of objects constructions and destructions. The reference interface is the C one, whereas FORTRAN and C++ interfaces are implemented as wrappers around it. In the C++ interface, all the routines are implemented as methods of PLMD::Plumed. In the C and FORTRAN interfaces, all the routines are named plumed_*, to avoid potential name clashes. Notice that the entire plumed library is implemented in C++, and it is hidden inside the PLMD namespace.

Handlers to the plumed object can be converted among different representations, to allow inter-operability among languages. In C, there are tools to convert to/from FORTRAN, whereas in C++ there are tools to convert to/from FORTRAN and C.

These handlers only contain a pointer to the real structure, so that when a plumed object is brought from one language to another, it brings a reference to the same environment.

Moreover, to simplify life in all cases where a single Plumed object is required for the entire simulation (which covers many of the practical applications with conventional MD codes) it is possible to take advantage of a global interface, which is implicitly referring to a unique global instance. The global object should still be initialized and finalized properly. This global object is obviously not usable in a multithread context. In addition, it is difficult to use it in an exception-safe manner, so its usage in C++ is allowed but discouraged.

As of PLUMED 2.5, the interface contains a reference counter that allows for a better control of plumed initializations and deallocations. This is particularly useful for the C++ interface that now behaves similarly to a primitive shared pointer and can be thus copied. In other languages, to use the reference counter correctly it is sufficient to remember the following rule: for any plumed_create* call, there should be a corresponding plumed_finalize call. More examples can be found below.

The basic method to send a message to plumed is

  (C) plumed_cmd
  (C++) PLMD::Plumed::cmd
  (FORTRAN)  PLUMED_F_CMD

To initialize a plumed object, use:

  (C)        plumed_create
  (C++)      (constructor of PLMD::Plumed)
  (FORTRAN)  PLUMED_F_CREATE

As of PLUMED 2.5, you can also initialize a plumed object using the following functions, that load a specific kernel. The function plumed_create_dlopen2 allows to specify options for dlopen. The C++ version accepts an optional argument to this aim.

  (C)        plumed_create_dlopen or plumed_create_dlopen2
  (C++)      PLMD::Plumed::dlopen
  (FORTRAN)  PLUMED_F_CREATE_DLOPEN

As of PLUMED 2.8, you can also initialize a plumed object using the following function, that loads a kernel from an already loaded shared library. It accepts a handler returned by dlopen:

  (C)        plumed_create_dlsym
  (C++)      PLMD::Plumed::dlsym
  (FORTRAN not allowed)

To finalize a plumed object, use

  (C)        plumed_finalize
  (C++)      (destructor of PLMD::Plumed)
  (FORTRAN)  PLUMED_F_FINALIZE

To access to the global-object, use

  (C)        plumed_gcreate, plumed_gfinalize, plumed_gcmd
  (C++)      PLMD::Plumed::gcreate, PLMD::Plumed::gfinalize, PLMD::Plumed::gcmd
  (FORTRAN)  PLUMED_F_GCREATE, PLUMED_F_GFINALIZE, PLUMED_F_GCMD

To check if the global object has been initialized, use

  (C)        plumed_ginitialized
  (C++)      PLMD::Plumed::ginitialized
  (FORTRAN)  PLUMED_F_GINITIALIZED

Notice that when using runtime binding the plumed library might be not available. In this case, plumed_create (and plumed_gcreate) will still succeed, but a subsequent call to plumed_cmd (or plumed_gcmd) would exit. In order to avoid this unpleasant situation you have two options.

First, you can check if plumed library is available before actually creating an object using this function:

  (C)        plumed_installed
  (C++)      PLMD::Plumed::installed
  (FORTRAN)  PLUMED_F_INSTALLED

Alternatively, as of PLUMED 2.5, you can interrogate the just created plumed object using the following function:

  (C)        plumed_valid
  (C++)      PLMD::Plumed::valid
  (FORTRAN)  PLUMED_F_VALID

If you want to create on purpose an invalid Plumed object (useful in C++ to postpone the loading of the library) you can use Plumed p(Plumed::makeInvalid());.

To know if the global object is valid instead you should use the following function:

  (C)        plumed_gvalid
  (C++)      PLMD::Plumed::gvalid
  (FORTRAN)  PLUMED_F_GVALID

To convert handlers between different languages, use

  (C)        plumed_c2f                 (C to FORTRAN)
  (C)        plumed_f2c                 (FORTRAN to C)
  (C++)      Plumed(plumed) constructor (C to C++)
  (C++)      operator plumed() cast     (C++ to C)
  (C++)      Plumed(char*)  constructor (FORTRAN to C++)
  (C++)      toFortran(char*)           (C++ to FORTRAN)

As of PLUMED 2.5, when using C or C++ we allow a user to explicitly store a plumed object as a void pointer (indeed: that's the only thing contained in a plumed object). This might be useful in case you do not want to include the Plumed.h header in some of your headers. In order to convert to/from void pointers you can use the following functions

  (C)        plumed_v2c                 (void* to C)
  (C)        plumed_c2v                 (C to void*)
  (C++)      Plumed(void*) constructor  (void* to C++)
  (C++)      toVoid()                   (C++ to void*)

Using the functions above is much safer than accessing directly the pointer contained in the plumed struct since, when compiling with debug options, it will check if the void pointer actually points to a plumed object.

As of PLUMED 2.5, we added a reference count. It is in practice possible to create multiple plumed objects that refer to the same environment. This is done using the following functions

  (C)        plumed_create_reference     (from a C object)
  (C)        plumed_create_reference_f   (from a FORTRAN object)
  (C)        plumed_create_reference_v   (from a void pointer)
  (FORTRAN)  plumed_f_create_reference   (from a FORTRAN object)

In C++ references are managed automatically by constructors and destructor. In addition, you can manually manage them (with care!) using incref() and decref().

The interface of the FORTRAN functions is very similar to that of the C functions and is listed below:

  FORTRAN interface
    SUBROUTINE PLUMED_F_CREATE(p)
      CHARACTER(LEN=32), INTENT(OUT)   :: p
    SUBROUTINE PLUMED_F_CREATE_DLOPEN(p,path)
      CHARACTER(LEN=32), INTENT(OUT)   :: p
      CHARACTER(LEN=*),  INTENT(IN)    :: path
    SUBROUTINE PLUMED_F_CREATE_REFERENCE(p,r)
      CHARACTER(LEN=32), INTENT(OUT)   :: p
      CHARACTER(LEN=32), INTENT(IN)    :: r
    SUBROUTINE PLUMED_F_CREATE_INVALID(p)
      CHARACTER(LEN=32), INTENT(OUT)   :: p
    SUBROUTINE PLUMED_F_CMD(p,key,val)
      CHARACTER(LEN=32), INTENT(IN)    :: p
      CHARACTER(LEN=*),  INTENT(IN)    :: key
      UNSPECIFIED_TYPE,  INTENT(INOUT) :: val(*)
    SUBROUTINE PLUMED_F_FINALIZE(p)
      CHARACTER(LEN=32), INTENT(IN)    :: p
    SUBROUTINE PLUMED_F_INSTALLED(i)
      INTEGER,           INTENT(OUT)   :: i
    SUBROUTINE PLUMED_F_VALID(p,i)
      CHARACTER(LEN=32), INTENT(IN)    :: p
      INTEGER,           INTENT(OUT)   :: i
    SUBROUTINE PLUMED_F_USE_COUNT(p,i)
      CHARACTER(LEN=32), INTENT(IN)    :: p
      INTEGER,           INTENT(OUT)   :: i
    SUBROUTINE PLUMED_F_GLOBAL(p)
      CHARACTER(LEN=32), INTENT(OUT)   :: p
    SUBROUTINE PLUMED_F_GINITIALIZED(i)
      INTEGER,           INTENT(OUT)   :: i
    SUBROUTINE PLUMED_F_GCREATE()
    SUBROUTINE PLUMED_F_GCMD(key,val)
      CHARACTER(LEN=*), INTENT(IN)     :: key
      UNSPECIFIED_TYPE, INTENT(INOUT)  :: val(*)
    SUBROUTINE PLUMED_F_GFINALIZE()
    SUBROUTINE PLUMED_F_GVALID(i)
      INTEGER,           INTENT(OUT)   :: i

Almost all C functions have a corresponding FORTRAN function. As a simple mnemonic, if you know the name of the C function you can obtain the corresponding FORTRAN subroutine by adding F_ after the PLUMED_ prefix. In addition, all plumed objects are replaced by CHARACTER(LEN=32) objects holding the same information. These pointers basically contain a text representation of the stored pointer, that is suitable to be contained in a string. Finally, whenever a C function returns a value, the corresponding FORTRAN subroutine will have an additional INTENT(OUT) parameter passed as the its last argument.

When you compile the FORTRAN interface, wrapper functions are added with several possible name manglings, so you should not experience problems linking the plumed library with a FORTRAN file.

Error handling

In case an error is detected by PLUMED, either because of some user error, some internal bug, or some mistake in using the library, an exception will be thrown. The behavior is different depending if you use PLUMED from C/FORTRAN or from C++.

First of all, notice that access to PLUMED goes through three functions:

  • plumed_create: this, as of PLUMED 2.5, is guaranteed not to throw any exception. If there is a problem, it will just return a plumed object containing a NULL pointer
  • plumed_cmd: this function might throw exceptions.
  • plumed_finalize: this is a destructor and is guaranteed not to throw any exception.

The following discussion concerns all the exceptions thrown by plumed_cmd.

If you use C/FORTRAN, you will basically have no way to intercept the exception and the program will just terminate.

If you use C++ and you are calling the C++ interface (e.g. Plumed::cmd), as of PLUMED 2.5 we implemented a complete remapping of the exceptions thrown by PLUMED. This solves both the problems mentioned above. In particular:

  • Instead of throwing an exception, PLUMED will return (using a plumed_nothrow_handler) the details about the occurred error.
  • An equivalent exception will be thrown within the inline PLUMED interface compiled with your MD code.

As a consequence, you will be able to combine different compilers and avoid stack unwinding in the C layer.

If you use C++ but you are calling the C interface (e.g. plumed_cmd), then you might be able to catch the exceptions thrown by PLUMED. Notice that all the exceptions thrown by PLUMED inherit from std::exception, so you might want to catch it by reference. By default, as of PLUMED 2.8 plumed_cmd is redefined as a macro and directly calls the Plumed::cmd interface, and thus behaves in an equivalent manner. With previous versions of this header one could have encountered problems with stack unwinding performed during exception handling in the C layer.

Notice that, even if you use Plumed::cmd, if you are loading a kernel <=2.4 any exception generated by PLUMED will leak through the C layer. This might lead to undefined behavior. If you are lucky (with some compiler it works!) and the exception arrives to C, PLUMED will catch it and rethrow it as it would do if you were using a kernel >=2.5.

The remapping of exceptions takes care of all the standard C++ exceptions plus all the exceptions raised within PLUMED. Unexpected exceptions that are derived from std::exception will be rethrown as std::exception. Notice that this implies some loss of information, since the original exception might have been of a different type. However, it also implies that the virtual table of the original exception won't be needed anymore. This allows to completely decouple the MD code from the PLUMED library.

Typesafe interface

Starting with PLUMED 2.8, the cmd function of the C++ interface, and the similar function gcmd, can be called with several interfaces and can perform a typechecking on the passed argument. In particular, the following forms are now possible:

  cmd("string",value);        // by value
  cmd("string",&value);       // by pointer
  cmd("string",&value,nelem); // by pointer, specifying the number of elements of the passed array
  cmd("string",&value,shape); // by pointer, specifying the shape of the passed array
\endverbati
  The `nelem` and `shape` arguments are used by PLUMED to check that the user
  provided enough elements. If nelem is provided, the check is done on the flatten array, whereas if shape
  is passed a more thorough check is performed controlling each of the dimensions of the array.
  In addition to this, the type of the pointer (or of the value) is checked at runtime.

  All these checks are only implemented if the PLUMED library is recent (>=2.8). However, it will still be
  possible to load at runtime an older PLUMED library (<=2.7). For this reason, it is still compulsory
  to pass the correct types to the `cmd` function, also when the argument is passed by value.
  Type conversions are only performed between pointers and only in ways compatible with
  what is allowed in C++ (e.g., `const void*` cannot be converted to `void*`, but `void*` can
  be converted to `const void*`).

  Type checkes can be disabled in two ways:
  - By compiling `Plumed.h` with `-D__PLUMED_WRAPPER_CXX_TYPESAFE=0`
  - By setting `export PLUMED_TYPESAFE_IGNORE=yes` at runtime.

  Typechecks are also enabled in the C interface (plumed_cmd). This function is replaced with a macro by default.
  In particular:
  - If the C interface is used in C++ code, it calls the C++ interface. Can be disabled with `-D__PLUMED_WRAPPER_CXX_BIND_C=0`.
  - If the C interface is used in C code and compiled with a C11 compiler, it uses _Generic to pass type information.
    Can be disabled using `-D__PLUMED_WRAPPER_C_TYPESAFE=0`.

\section ReferencePlumedH-2-5 New in PLUMED 2.5

  The wrappers in PLUMED 2.5 have been completely rewritten with several improvements.
  The interface is almost perfectly backward compatible, although the behavior of C++ constructors
  has been modified slightly.
  In addition, a few new functions are introduced (explicitly marked in the documentation).
  As a consequence, if your code uses some of the new functions, you will not be able
  to link it directly with an older PLUMED library (though you will still be able to load
  an older PLUMED library at runtime). In addition, the reference counter changes slightly
  the behavior of the C++ methods used to interoperate with C and FORTRAN.

  An important novelty is in the way the runtime loader is implemented.
  In particular, the loader works also if the symbols of the main executable are not exported.
  The proper functions from the kernel are indeed searched explicitly now using `dlsym`.

  Some additional features can be enabled using suitable environment variables. In particular:
  - `PLUMED_LOAD_DEBUG` can be set to report more information about the loading process.
  - `PLUMED_LOAD_NAMESPACE` can be set to `LOCAL` to load the PLUMED kernel in a separate
    namespace. The default is global namespace, which is the same behavior of PLUMED <=2.4,
    and is consistent with what happens when linking PLUMED as a shared library.
  - `PLUMED_LOAD_NODEEPBIND` can be set to load the PLUMED kernel in not-deepbind mode. Deepbind
    mode implies that the symbols defined in the library are preferred to other symbols with the same name.
    Only works on systems supporting `RTLD_DEEPBIND` and is mostly for debugging purposes.

  Another difference is that the implementation of the wrappers is now completely contained in the `Plumed.h`
  file. You can see that the `Plumed.c` is much simpler now and just includes `Plumed.h`. With a similar
  procedure you could compile the wrappers directly into your code making it unnecessary to link
  the libplumedWrapper.a library. The corresponding macros are still subject to change and are not documented here.

  As written above, the plumed object now implements a reference counter.  Consider the following example
\verbatim
  plumed p=plumed_create();
  plumed_cmd(p,"init",NULL);
  plumed q=plumed_create_reference(p);
  plumed_finalize(p);
// at this stage, object q still exists
  plumed_cmd(q,"whatever",NULL);
  plumed_finalize(q);
// now plumed has been really finalized

In other words, every plumed_create, plumed_create_dlopen, plumed_create_reference, plumed_create_reference_f, and plumed_create_reference_v call must be matched by a plumed_finalize. Notice that in C++ whenever an object goes out of scope the reference counter will be decreased. In addition, consider that conversion from C/FORTRAN/void* to C++ implies calling a C++ constructor, that is increases the number of references by one. Converting from C++ to C/FORTRAN/void* instead does not call any constructor, that is the number of references is unchanged.

The change in the behavior of C++ constructors means that the following code will behave in a backward incompatible manner:

  plumed p=plumed_create();
  plumed_cmd(p,"init",NULL);
  Plumed q(p);
  plumed_finalize(p);
// at this stage, object q still exists with PLUMED 2.5
// on the other hand, with PLUMED 2.4 object q refers to an
// already finalized object
  q.cmd("whatever",NULL);

Another difference is that the value of the variable PLUMED_KERNEL is read every time a new plumed object is instantiated. So, you might even use it to load different plumed versions simultaneously, although the preferred way to do this is using the function plumed_create_dlopen. Notice that if you want to load multiple versions simultaneously you should load them in a local namespace. plumed_create_dlopen does it automatically, whereas loading through env var PLUMED_KERNEL only does it if you also set env var PLUMED_NAMESPACE=LOCAL.

Finally, a few functions have been added, namely: