POISE under automation

The command-line interface that POISE offers (see Frontend options) allows us to wrap POISE within a larger script. Here we present some examples of how this can be accomplished: after this, the extension to automation is largely straightforward. For example, if POISE is used inside an AU programme, then set the the AUNM parameter in TopSpin appropriately.

Basics

To incorporate POISE in an AU programme, you can use the syntax:

XCMD("sendgui xpy poise <routine_name> [options]")

inside the AU script.

(Optional: To suppress POISE’s final popup (telling the user that the optimum has been found), you can add the -q flag in the options. The popup won’t stop TopSpin from running whatever it was going to run, though, so it’s completely safe to show the message.)

Alternatively, you can wrap POISE within a Python script, which is arguably easier to write. The corresponding syntax for running an optimisation would be:

XCMD("xpy poise <routine_name> [options]")

As you can see, it is the same except that sendgui isn’t needed.

A simple(ish) example

Here’s an example of how the p1 optimisation (shown in Setting up a Routine) can be incorporated into an AU script (download from here). This AU script performs a very similar task to the existing pulsecal script: it finds the best value of p1 and plugs it back into the current experiment. However, as we wrote in the paper, it tends to provide a much more accurate result. In practice, we’ve already used it many times to calibrate p1 before running other experiments.

Note

This AU programme comes installed with POISE. However, you must create the p1cal routine before you can use this. Please see Setting up a Routine for a full walkthrough.

GETCURDATA
int old_expno = expno;
// Use EXPNO 99999 in the current folder for optimisation.
DATASET(name, 99999, procno, disk, user)
// Set some key parameters. Notice that these lines can be substantially cut
// if an appropriate parameter set is set up beforehand.
RPAR("PROTON", "all")
GETPROSOL
STOREPAR("PULPROG", "zg")
STOREPAR("NS", 1)
STOREPAR("DS", 0)
STOREPAR("D 1", 1.0)
STOREPAR("RG", 1)
STOREPAR("F1P", f1p)
STOREPAR("F2P", f2p)
STOREPARS("F1P", f1p)
STOREPARS("F2P", f2p)
// Run optimisation (uses BOBYQA by default).
XCMD("sendgui xpy poise p1cal -q")
// POISE stores the optimised value in p1 after it's done. We can retrieve it
// here. Don't try to get the *status* parameter, since that is not the
// optimised value (it is the value used for the last function evaluation!)
float p1opt;
FETCHPAR("P 1", &p1opt)
p1opt = p1opt/4;
// Move back to old dataset and set p1 to optimised value.
DATASET(name, old_expno, procno, disk, user)
VIEWDATA_SAMEWIN  // not strictly necessary, just re-focuses the original spectrum
STOREPAR("P 1", p1opt)
Proc_err(INFO_OPT, "Optimised value of p1: %.3f", p1opt);
// (Optional) Run acquisition.
// ZG
QUIT

Note that the six lines underneath “set some key params” can be collapsed to one line if an appropriate parameter set is set up beforehand.

Here’s the Python equivalent of the AU programme above (download from here):

# Read in F1P and F2P from current dataset.
f1p = GETPAR("F1P")
f2p = GETPAR("F2P")
# Use EXPNO 99999 in the current folder for optimisation.
old_dataset = CURDATA()
opt_dataset = CURDATA()
opt_dataset[1] = "99999"
# Create new dataset and move to it.
NEWDATASET(opt_dataset, None, "PROTON")
RE(opt_dataset)
# Set some key parameters. Notice that these lines can be cut if an appropriate
# parameter set is set up beforehand (and loaded using NEWDATASET()).
XCMD("getprosol")
PUTPAR("PULPROG", "zg")
PUTPAR("NS", "1")
PUTPAR("DS", "0")
PUTPAR("D 1", "1")
PUTPAR("RG", "1")
XCMD("s f1p {}".format(f1p))  # PUTPAR("status F1P") doesn't work.
XCMD("s f2p {}".format(f2p))  # Even though the documentation says it should.
# Run optimisation (uses BOBYQA by default).
XCMD("poise p1cal -q")
# POISE stores the optimised value in p1 after it's done. We can retrieve it
# here. Don't try to get the *status* parameter, since that is not the
# optimised value (it is the value used for the last function evaluation!)
p1opt = float(GETPAR("P 1"))/4
# Move back to old dataset and set p1 to optimised value.
RE(old_dataset)
PUTPAR("P 1", str(p1opt))
ERRMSG("Optimised value of p1: {:.3f}".format(p1opt))
# A TopSpin quirk: using MSG() will block subsequent commands until user hits
# "OK". You can use ERRMSG(), as is done here. There are other ways around
# this, see MSG_nonmodal() in the POISE frontend script.
# (Optional) Run acquisition.
# ZG()

Terminating scripts which call POISE

So far we have seen how POISE can be included inside an AU programme or a Python script in TopSpin (a “parent script”). One problem that we haven’t dealt with yet is how to kill the parent script when POISE errors out. In general, if POISE is called via XCMD(poise ...), then even if POISE fails, the parent script will continue running.

One way to deal with this is to use a trick involving the TI TopSpin parameter, which can be set to any arbitrary string. POISE, upon successful termination, will store the value of the cost function in the TI parameter. If it doesn’t successfully run (for example if a requested routine or cost function is not found, or some other error), then TI will be left untouched.

In order to detect when POISE fails from the top-level script, we therefore:

  1. Set TI to be equal to some sentinel value, i.e. any string whose exact value is just used as a marker. Note that this should not be a numeric value. You can set it to be blank if you like, but please read the note below.

Note

In an AU or Python programme, to set a parameter to a blank value, you have to set it to be a non-empty string that contains only whitespace. For example, in a Python script:

PUTPAR("TI", " ")   # this will work
PUTPAR("TI", "")    # this will NOT work!

TopSpin mangles empty strings: instead of putting an empty string in, it puts the string "0" in. On the other hand, if the string is not empty but contains whitespace, TopSpin automatically trims it to an empty string after it’s been put in. I don’t know why. The same applies to the STOREPAR macro in AU programmes.

  1. Run POISE.

  2. Check if TI is equal to that sentinel value. If it is, then quit unceremoniously with an error message of your choice.

Briefly, here is an example of this strategy in action. We show a Python script here, but the AU script is essentially the same, just with different function names.

PUTPAR("TI", "poise")  # here "poise" serves as the sentinel value.
XCMD("poise p1cal -q")

# Here POISE should be done. If it succeeded then TI will no longer be "poise".
if GETPAR("TI") == "poise":
    raise RuntimeError("POISE failed!")

# After this you can continue with whatever you wanted to do.