Test Case Format¶

Alyvix test cases are stored as JSON files with an .alyvix extension, and represent the unique mechanism for interchange between Alyvix tools. Each file is self-contained, without any external dependencies.

Alyvix Core and its data structures are open source. The entire JSON file format of an Alyvix test case file is described in detail below.

Test Case Files¶

Each time you exit Designer or Selector, or save an object in Editor, the JSON object representing the test case is saved in the current directory under the file name:

<filename>.alyvix

where filename is the value of the -f parameter passed when starting Editor or Designer.

When you call Alyvix Robot via the command line with that file as a parameter, it will create a new data file based on the original file name, but with the resulting measures included. Appended to the file name is the timestamp at the moment of execution:

<filename>_<full-timestamp>.alyvix

For example:

logintest_20191220_145228_UTC+0100.alyvix

Note

The timestamped file is not produced when a test case is executed within Editor via , only when executed directly from Alyvix Robot, since storing the results is only useful in an automated monitoring scenario where the results may need to be used later.

Test Case JSON Structure¶

A single .alyvix file stores the entire test case, and a timestamped file additionally contains all measures resulting from its single execution run.

The following example JSON structure illustrates the high-level structure of the test case object:

{
   "maps": { },
   "objects": {
      "<test-case-object-name>": {
         "call": { },
         "components": {
            "<resolution>": {
               "groups": [
                  { "main": { },
                    "subs": [ ] },
                  { "main": { },
                    "subs": [ ] },
                  { "main": { },
                    "subs": [ ] }
               ],
               "screen": "<base64>"
            }
         },
         "date_modified": "<timestamp>",
         "detection": {
            "break": true,
            "timeout_s": 10,
            "type": "appear" },
         "measure": {
            "group": "<transaction-group-name>",
            "output": true,
            "series": { },
            "thresholds": { } }
      }
   },
   "script": {
      "case": [
         "<test-case-object-name>",
         { "disable": "<test-case-object-name>" },
         { "flow": "<test-case-object-name>",
           "if-true": "<test-case-object-name>" },
         { "flow": "<test-case-object-name>",
           "if-false": "<test-case-object-name>" },
         { "flow": "<test-case-object-name>",
           "for": "<test-case-map-name>" }
      ],
      "sections": {
         "<section-name>": [
            "<test-case-object-name>"
         ],
         "exit": [ ],
         "fail": [ ] }
   }
}

The individual sections making up the JSON structure are explained below.

Top Level¶

The top level corresponds to the principal elements of Alyvix Editor:

{ "maps": {
     "map-name":  { } },
  "objects": {
     "<test-case-object-name>":  { } },
  "script": {
     "case":  [ ],
     "sections":  { } }
}

maps – An ordered set of values that a script can loop over, for instance to insert a sequence of values in multiple text fields.
objects – A list of the individual test case objects created with Alyvix Designer. Each test case object is identified uniquely in the list by its object name.
script – The scripts created for a test case, both the main script and any section (subroutine) scripts, in the order they are executed.

Maps¶

Each named map under the maps field contains a set of named keys (rows) and a fixed number of cell values, to represent an m x n matrix (columns cannot be named):

{ "<map-name-1>": {
     "<key-name-1>": [ "key1-val1",
                       "key1-val2" ],
     "<key-name-2>": [ "key2-val1",
                       "key2-val2" ] }
}

When arguments are passed to Alyvix Robot in the command line interface, a new map structure will be added under the name cli, and containing the arguments as keys. Because of this, the map name cli is reserved.

Objects¶

The value of each property contained within the objects substructure (shown here as “<test-case-object-name>”) corresponds to a test case object created in Designer/Editor. Those values have the following meanings:

{ "<test-case-object-name>": {
     "call": { },
     "components": { },
     "date_modified": "<timestamp>",
     "detection": { },
     "measure": { }
  }
}

call – The recorded options to start or kill an external application when a test case object is first called
components – The representation of the component tree
date_modified – The time the test case object was last modified (also displayed in Selector)
detection – The test case object’s detection conditions such as timeout, break, and appear/disappear
measure – Details of the last execution via Alyvix Robot (see below) if this is a timestamped .alyvix file, along with the settings for transaction group, output, and thresholds.

Objects > {object} > Components¶

The components section represents the component tree of a given test case object. The example below shows how you can have a different component tree at each distinct screen resolution.

{ "components": {
    "<resolution>": {
        "groups": [
            { "main": {
                "detection": { },
                "interactions": { },
                "visuals": { } },
              "subs": [ ] },
            { "main": { },
              "subs": [ ] },
            { "main": { },
              "subs": [ ] } ],
        "screen": "<base64>" } }
}

<resolution> – The resolution of a screen capture as shown in Alyvix Selector. This value is not selected by the user, but depends on the hardware and the settings in use when the screen is captured.

groups – A JSON array of exactly 3 items, each of which corresponds to one of the groups in the component tree (red, green and blue).
1. main – The main component in a group.
  1. detection – Information about the match method to use when trying to detect the region of interest
  2. interactions – Details on any mouse or keyboard actions to initiate in the event of a positive detection
  3. visuals – Information about the region of interest and the selection box laid over the screen capture
2. subs – A JSON array of exactly 4 items, each of which corresponds to one of the subcomponents in a group. The structure of each element is identical to the structure of the main component above.
screen – The Base64 representation of the screen capture. There is exactly one screen capture for each resolution in a test case object.

Objects > {object} > Measure¶

The measure structure holds some of the user-selected options visible in Selector for a given test case object, along with the series array, which contains recorded data resulting from each run of the test case object by Alyvix Robot in the test case’s script within a single session.

The measure structure is empty in a plain .alyvix file, but contains measurements in a timestamped .alyvix file.

When a test case is executed, its .alyvix file is taken as the starting point, and thus the series structure for all of its test case objects is empty. Each time a test case object is called by the test case script, the results and auxiliary data are added to series as a new array element.

This example shows the structure and some default values:

{ "measure": {
    "group": "<transaction-group-name>",
    "output": "<true/false>",
    "series": [
        {
            "accuracy_ms": "<ms-amount>",
            "annotation": "<base64>",
            "exit": "<true/false/fail/not_executed>",
            "performance_ms": "<ms-amount>",
            "records": {
                "check": "<true/false>",
                "extract": "<extracted-text>",
                "image": "<base64>",
                "text": "<scraped-text>"
            },
            "resolution": {
                "height": "<pixel-amount>",
                "width": "<pixel-amount>"
            },
            "scaling_factor": "<zoom-amount>",
            "screenshot": "<base64>",
            "timestamp": "<epoch>"
        }
    ],
    "thresholds": {
        "critical_s": 5,
        "warning_s": 3 }
    }
}

group – The transaction group assigned to the test case object
output – Whether the test case object’s “Measure” checkbox is marked
series – An array containing the results and auxiliary data for each run of the parent test case object, containing the following information:
1. accuracy_ms – Given the performance measurement from the same run (see below), this is the interval (+/-) over which that measurement is guaranteed to have occurred
2. annotation – The Base64 screen captured when the test case object was being run, and overlaid graphic annotations to indicate those components that matched as well as potentially the first component that did not match
3. exit – The result type of the most recent run. If the test case object did not match, then the type false or fail is determined by whether the break flag was set
4. performance_ms – The number of milliseconds that elapsed from when the run of a test case object began until a match was confirmed, subject to the accuracy interval described above; otherwise -1
5. records – Temporary variables that are computed during the matching process, such as a text string extracted by a text component
6. resolution – The screen resolution at which the test case object was executed
7. scaling_factor – The Windows zoom level at the time when the test case object was executed
8. screenshot – The Base64 representation of the screen during the test case screen capture, but is made during the execution phase, rather than during the building phase
9. timestamp – The time (Unix epoch) when this series element was run; otherwise -1

thresholds – The number of elapsed seconds from when the test case object has been started until a warning and/or critical alert should be triggered; if no values are specified, no alerts will be triggered

Script¶

The script parameter records the main script and any subroutines used to compose more complex interaction behaviors.

{ "case": [
     "<test-case-object-name>",
     { "disable": "<test-case-object-name>" },
     { "flow": "<test-case-object-name>",
       "if-true": "<test-case-object-name>" },
     { "flow": "<test-case-object-name>",
       "if-false": "<test-case-object-name>" },
     { "flow": "<test-case-object-name>",
       "for": "<test-case-map-name>" }
  ],
  "sections": {
     "<section-name>": [
        "<test-case-object-name>",
        "<test-case-object-name>"
     ],
     "exit": [],
     "fail": [] }
}

case – The main script that will be executed by Alyvix Robot. The script is an ordered list of scripting elements in which a simple, enabled element is represented just by the name of a test case object. Other elements have the following meanings:
1. disable – Skip this test case object and continue with the next
2. flow – Indicates the test case object or section that is the value of this key should be executed by Alyvix Robot in the context of a conditional (if-true or if-false) or a loop (map)
3. if-true – Evaluates whether the detection part of a given test case object would match, but without executing its actions
4. if-false – As above, but if the test case object would not match
5. for – Indicates there should be a loop over the values in the specified Map
sections – Corresponds to the named subroutine scripts created in Alyvix Editor. Each section is a parameter where its key is the name of the section and its value is the same as the case value described above.
1. exit – The (teardown) script to execute when a script has completed, regardless of whether it succeeded or failed
2. fail – The (teardown) script to execute when a script has failed during the execution of its test case objects