ROAM User Guide

Repository : https://gitlab.fit.fraunhofer.de/efpf-pilots/roam

Introduction

The ROAM (Risk, Opportunity, Analysis & Monitoring) Tool allows users to automatically use data streams (via MQTT) and transform them into an output; although the tool is capable of general transformations, we focus on risks, and assume that the output will be a risk (or a figure that visualizes a risk). The tool will also be capable of using REST API to receive data from external sources, or the EFPF Secure Data Store. When using the ROAM Tool, the user defines risk workflows. These workflows require a source; the MQTT topic on which to receive data. Then, the user has to choose the so-called risk recipes that transform this input into the desired output. The tool comes with built-in configurable recipes. These configurable recipes allow for the user to specify some parameters that tailor the recipes to the users specific situations. The user will also be able to submit their own (configurable) recipes; information on that can be found in the development guide. We will support workflows that accept multiple streams as input and yield an output based on these multiple inputs, and workflows will soon have caching capabilities that allow the user to make use of the previous output of the workflow. This is useful if some historic data is always present, or if one needs to consider multiple recent data points.

The ROAM Tool comes with a configuration user interface (UI) and a REST API. The configuration UI enables the user to create and edit recipes and workflows with a visual interface. The API has the same functionality, but requires the users to make an API call and supply a json object.

Configuration UI

This UI contains a panel with a list of all available recipes; the ones which are configurable, and the ones that have been configured. Furthermore, there’s also a panel with a list containing all defined workflows, which is empty by default. Each panel contains a list with searching functionality, sorting functionality, and filtering functionality. The recipes and workflows may be sorted by their name and id, while they can only be filtered by their tags. The user can define the tags of a workflow and recipe themselves. Moreover, the lists contains buttons that allow the user to create, edit, copy and remove recipes. For workflows, the same functionality exists, and on top of that the user may also toggle the workflow, meaning that the workflow actually listens to the incoming data (or not anymore). The user can also view the last output contents. Each recipe and workflow has a detail row that can be expanded. Along with all the details, this row also allows the user to test the recipes and workflows. This testing functionality is also available while creating/editing a recipe/workflow. This allows the user to make sure that a certain input invokes the expected output.

To show the configuration UI in practice, we will run you through the UI with screenshots, highlighting the different options and functionalities along the way:

Main frame:

Upon starting the tool, the user will always be met with the following screen:

The main frame of the ROAM tool

Keep in mind that the app for this tool is a single page app and that no progress will be saved and it will always behave the same if the page is restarted.

This main screen contains two expandable lists, one for recipes, and one for workflows.

Recipes:

Let us first take a look at recipes. Upon expanding the recipe list, the user is met with something like this:

A typical recipe list

This list contains two kinds of recipes; the first kind, we call base recipes, which either come with the ROAM tool, or have been submitted by the user (this is a future functionality). You can consider these as templates, which contain default parameter values and a generally elaborate description of what the recipe can do. The second kind of recipes are customized recipes, in which the user has already filled in the parameters, and tailored it to their own needs.

Base recipes cannot be edited or deleted, and they have no id. They can however have tags, but they can be copy-and-edited. Editing, copy-and-editing, and deletion can be done by clicking on the icon buttons to the left.

In this list, the user can filter by tag, or the user can search for (part of) a name or an id. The user is also able to sort the list by recipe name and id. This can be done by clicking on the arrow next to “Name” or “Id”. The arrow next to Id appears when hovering over it. Note: when searching, the user needs to press enter to finalize their search.

Each recipe also has an expandable detail row, which lists the recipe description and parameters. In this row, the user can also test the recipe:

A typical recipe detail row

When designing recipes, there are thence three options.

  1. You make one from scratch, by selecting a desired base recipe.
  2. You edit an existing customized recipe.
  3. You copy-and-edit an existing or base recipe. In the case of a base recipe, this is the same as creation, but the base recipe is already set.

The first can be accessed by the “+"-button in the top left of the list, the other two can be accessed by the aforementioned buttons.

Recipe Creation & Editing

The recipe creation window looks like this:

A typical recipe creation window

It consists of three parts. The first is the recipe creation method: the user may define it manually, or the user may upload a file (multiple recipes will be supported in the future). The second is completely visible here and the recipe definition is performed in this part. This one only shows up when defining the recipe manually. The third is the recipe testing part, which we will get to after introducing workflows.

When this form is sufficiently filled in, a “Submit” button appears in the bottom right below the test box, allowing the user to finalize their recipe. It may happen that the user is met with an error window when submitting a recipe or workflow. This then means that some field is not properly filled in, and an appropriate error message will point you in the right direction.

Furthermore, the two buttons in the top right (the save and trash bin icons) allow the user to exit the window (pressing outside the window, or pressing Esc will also do). However, only when pressing the save icon button, the progress is saved. This is only valid for the current configuration session, though. When restarting the tool, every kind of progress is lost, except for submitted recipes/workflows.

Depending on the recipe type, the additional parameter fields differ. For example, when choosing the simple MeanRecipe, that averages values:

Recipe Creation for MeanRecipe

A new button shows up that allows the user to see the recipe details. The user can then see what each parameter exactly does. Additionally, four new fields show up: “axis type”, “field”, “output” and “remove”.

Whenever a field is empty, and only shows its label (like for “field” here), its default is an empty string (""). It is generally a good idea to change the defaults, but especially when they are empty.

To illustrate all different kinds of supported fields, consider the situation where we select “AddDataRecipe”. For context, this recipe allows the user to insert their own data into the workflow in a json format:

Different field types in AddDataRecipe

Each field has appropriately been named to make it more obvious. This should show how to properly input all supported data types: arrays (or lists, if you will), strings, numbers (floats/integers), booleans, and it also supports None types. Strings and numbers are straightforward. When inputting a boolean, the user has to choose True or False from a dropdown box. In an array, all values are separated by a “,”. All spacebars before and after the comma are removed when parsing it. Lastly, the user can also input a None (null) type, which obviously does not allow for additional input.

For the array form, the user may click on the icon to the right, to toggle between string and number arrays. Green means text/string, and red means number. When inputting text in number mode, the following happens:

Wrong data in an array

The form is outlined in red, and it shows “The input is invalid.” This can also occur when typing a string in the number form. Whenever this happens, the last valid input is saved. Let’s say the user typed “1, 2, 3” and changed it to “1, 2, 3a”, for sake of example, then it will be internally saved as an array reading [1, 2, 3].

When opting to upload the recipe, the form looks as follows:

The form when uploading a file

The “Upload files” text functions as a button that allows the user to upload a file (again, multiple files will be supported in the future).

Workflows

When expanding the workflow window, the user is met with something similar to the following:

A typical workflow list

The workflow list functions much the same as the recipe list, with only a couple of minor differences. For starters, the user will encounter this list empty, upon first starting the tool, as we do not provide any default workflows. Of course, workflows are very much tailored to a specific user, so providing them does not make sense.

For demo purposes, we already made a demo workflow. As you can see, the creation, editing, copy-and-editing, deletion, sorting, filtering, and searching functionalities remain entirely the same. However, there are two new buttons; the user may “start” (and stop) their workflow, by clicking on the green play icon (resp. red stop icon that will take its place when it is already running). There is also a “Last workflow data” button, with the save icon. This is the last output this workflow has provided. It mostly serves for reference purposes, and the user may download it as well:

An empty last workflow data window

Of course, this will be populated once the workflow has been run, and any MQTT data was received at least once.

Additionally, workflows also have a slightly different detail row, containing a description, subscription details, and recipe list:

A typical workflow details

The subscription details can be collapsed to show information about how many times the workflow has received data, when it was started, when it was changed, etc. In the future, it will also show when the last data came through.

The recipes can also be expanded to show each recipe in the workflow. Each recipe can again be expanded to show its details, showing information similar to the recipe detail row mentioned before.

Typical subscription and recipe details

Workflow Creation & Editing

Again, overall this window is similar to the workflow creation/edit window.

A typical workflow creation window

The user may again input a name, description, and tags, but this time the user will also be able to input input and output topics. These serve for the MQTT functionality, and the tool will listen to the input topic, and send a message to the output topic once the workflow has run. In the future, multiple input topics will be supported for one workflow.

When dealing with workflows, there are two new boxes; the recipe order box, and the recipe configuration box. Additionally, the user may again upload a file, and test the workflow, which will be discussed in the next section.

In the recipe order box, the user may select the recipe under consideration (which are then highlighted, like the one in this picture), and add a recipe, that will default to “Choose a Recipe”, by clicking on the small “+"-button below the arrows. The user may highlight a new recipe in the list by clicking on its name, or by clicking on “Choose a Recipe” when none has been selected yet. When a new recipe is added, the focus will automatically be shifted to the new recipe.

The recipe configuration box allows user to remove the focused recipe, and to choose between base recipes and customized recipes, by toggling the radio buttons and selecting the desired recipe from the dropdown box.

The recipe configuration box with a selected recipe

When selecting a recipe, two buttons show up: an info button, showing the recipe details, and an edit button, allowing the user to edit recipes while creating a workflow. These recipe edits will not be saved permanently, although the user has the option to permanently save the recipe as well.

The recipe configuration box with toggled buttons

When clicking the edit button, the user may change the individual parameters, just like explained before, when creating or editing a recipe. When clicking the save button, the user may also change the name, description and tags. However, the save functionality does not yet work properly, but it will be fully implemented in the future.

When a recipe is left at “Choose a Recipe”, it will be removed from the workflow just before submission.

Testing

Let us finally move on to testing. We will first discuss workflow testing, as recipe testing is a special case of workflow testing.

Workflow Testing

The workflow testing box

When testing a workflow, the user can choose between three different ways of providing input data:

  1. Uploading a file. This is the recommended way, as the user probably already has a sample in json format somewhere anyway, but it is also much easier to work with here.
  2. Plain text. This will be pased as json. The testing input should be an array with nested json objects, like [{“test”: “test”}]. It may contain multiple json objects.
  3. Empty. In the app, this is called “Data already present or not needed”. Usually this third option is not the case, but it can be, if the first recipe generates data, like “AddDataRecipe”, which we used as an example for recipe creation. In reality, this option uses [{}] as input.

Keep in mind that, when choosing for manual inputs, the field names should be surrounded by “”, just like in the above exmaple after option 2. ‘’s do not work! The same holds for strings

The user also has the option to select whether the testing output should be compared to some value, to make sure that the workflow is working as expected. For example, if the only recipe being tested is the AddDataRecipe, adding the value ‘test’ to field ‘test’, with an empty input, the output should look like [{“test”: “test”}]. After the test has been submitted and completed, a new “Results” button shows up, next to the Submit button.

This button spawns the following window:

The workflow test results window

Here, the user can see the output by a series of expansion panels. The entire output array, or only individual json objects in the output array, can be downloaded. At the bottom the user can see whether the output is the same as expected. If it does not match, the user will get a message notifying which field(s) do not satisfy.

Recipe Testing

Recipe testing works mostly the same; the user may choose an input and eventually an output for comparison. The user can also define a workflow to test the recipe in, though, if desired. Otherwise, it will be tested in a workflow containing only the tested recipe.

The recipe testing box

Upon toggling the workflow specification radio button, a “Specify Workflow” button appears. This button spawns the following window:

The workflow definition window for recipe testing

The to be tested recipe is greyed out, as it cannot be altered here, and it is denoted by its newly given name. For the remainder, this window is exactly the same as regular workflow creation.

API

The API specification can be found here.

Previous
Next