HTTP Status reporting¶
STEMsalabim simulations may take a long time, even when running them in parallel on many processors. In order to ease tracking of the status of running simulations, we built reporting via HTTP POST requests into the program.
In order to use that feature, the libCURL library has to be installed and STEMsalabim needs to be linked against it.
Configure HTTP reporting¶
To configure HTTP reporting, please add the http_reporting: {}
block to your simulations’s parameter file,
containing at least reporting = true;
and the url to report to, url = "http://my_server_address:port/path";
.
If you want to use HTTP basic authentication,
you may also specify the options auth_user = "your_user";
and auth_pass = "your_pass";
. Note, that HTTP basic
auth will be enabled as soon as auth_user
is not empty. You should therefore only fill in that field when you want
to use authentication.
Additional, custom payload for the HTTP requests may be specified in the sub-block parameters: {}
. Each key-value
pair in this block is translated into JSON and appended to each request. This allows you to use custom authentication
techniques, such as token-based authentication
etc.
An example configuration block with HTTP basic authentication may look like this:
http_reporting: {
reporting = true;
url = "http://my_api_endpoint:8000/stemsalabim-reporting";
auth_user = "my_user";
auth_pass = "my_pass";
parameters: {
simulation_category = "suitable for many Nature papers";
}
}
The status requests¶
In each request that STEMsalabim sends, some JSON payload is common. In addition to the JSON values specified in the parameter file (see Configure HTTP reporting), the following parameters are always reported:
time: // The currrent date and time
id: // the UUID of the simulation
num_threads: // the number of threads of each MPI processor
num_processors: // the number of MPI processors
num_defoci: // the total number of defoci to calculate
num_configurations: // the total number of frozen phonon configurations to calculate
event: // A code for what event is reported. see below.
The following four different event
codes, each for a different event, are reported:
START_SIMULATION
: A simulation is started¶
This request is sent at the beginning of a simulation. Additional key/value pairs sent are:
event: "START_SIMULATION"
version: // program version
git_commit: // git commit hash of the program version
title: // simulation title
START_DEFOCUS
: A defocus iteration is started¶
This request is sent at the beginning of a defocus iteration. Additional key/value pairs sent are:
event: "START_DEFOCUS"
defocus: // the defocus value in nm
defocus_index: // the index of the defocus, between 0 and num_defoci
START_FP_CONFIGURATION
: A frozen phonon iteration is started¶
This request is sent at the beginning of a frozen phonon configuration. Additional key/value pairs sent are:
event: "START_FP_CONFIGURATION"
defocus: // the defocus value in nm
defocus_index: // the index of the defocus, between 0 and num_defoci
configuration_index: // the index of the configuration, between 0 and num_configurations
PROGRESS
: Progress report¶
This request is sent during the calculation, typically after each integer percent of the simulation finished. Additional key/value pairs sent are:
event: "START_CONFIGURATION"
defocus: // the defocus value in nm
defocus_index: // the index of the defocus, between 0 and num_defoci
configuration_index: // the index of the configuration, between 0 and num_configurations
progress: // progress between 0 and 1 of this configuration iteration within this defocus iteration
FINISH
: Simulation finished¶
This request is sent when the simulation finished. Additional key/value pairs sent are:
event: "FINISH"
How to process the reports¶
Obviously, in order to register the requests, an HTTP(S) server needs to be running on the target machine. For example, a very simple server in python using the http://flask.pocoo.org/ package, that only echos the requests, can be implemented as:
#!/usr/bin/env python
from flask import Flask
from flask import request
import json
app = Flask(__name__)
@app.route('/', methods=['POST'])
def echo():
content = request.get_json()
print(json.dumps(content, indent=4))
return ""
if __name__ == "__main__":
app.run()
Run the script and then start a STEMsalabim simulation to see requests imcoming.