|Picture checks / Key Points Modification tool|
For most of TestArchitect's picture-handling actions, you actually have a choice of methods for comparing a tested image against a baseline. Keypoint object detection is supported by all but the check picture action. Unlike pixel-by-pixel comparison, which requires 100% equivalence between the pixels of a baseline and image under test in order to consider them a positive match, keypoint comparisons tolerate differences between compared images. Moreover, you can adjust the tolerance factor in keypoint comparisons.
Keypoint object detection is also the technique of choice when you are searching for the presence of a pattern, or object, within an image under test. This is especially true when the object in the test image can vary from the baseline to some degree, perhaps due to changes of scale, position, color or background. Moreover, the presence of multiple copies, or variations, of the baseline object may also be detected within the image under test.
As with pixel-by-pixel detection, keypoint detection begins with a baseline, or reference
image. However, the baseline image in keypoint detection is used to generate a
keypoint profile – a set of points determined by means of an algorithm
applied to the image:
It is the keypoint profile of an image, rather than its bitmap, that is used as the reference for detecting the presence of the baseline image in the picture under test presented by an AUT. Instead of a baseline picture, you can think of the keypoint profile as defining a baseline object.
For example, let's say that we have the following baseline picture, taken from a game,
already stored in a picture check:
and that we are interested in testing for the presence of similar images of the fedora-wearing pig within other bitmaps at certain points in the automated test. We can certainly use this existing baseline picture, but because it's best practice to limit the baseline picture to the object of interest, we capture a new baseline, limiting the area to just that surrounding our object of interest:
The next step is to open the baseline picture of the pig in TestArchitect's Key Points Modification
Tool. (This is done by pressing the Browse
next to the Key Point box for the baseline picture.) This causes TestArchitect to immediately generate keypoints for the
The keypoints are initially purple, indicating that they are inactive, or unused. The upcoming configuration session is used to determine which of those keypoints should be made active.
As mentioned, this is often a highly iterative process, the intention of which is to configure the baseline object, and the associated similarity threshold, so as to best ensure that keypoint-based picture handling actions that use this baseline will find what you want them to find, and not have any "false positives".
You begin by loading a training image into the
Key Points Modification Tool. This is generally a picture previously
captured from the AUT and saved as a .png file. Naturally, it should be
either identical or similar to pictures that the AUT is expected to produce when tested with
Note that the fedora-wearing pig appears in the above image four times, at differing scales. During tests, we want our picture-handling functions to detect all of them. We do not want anything else to be mistaken for a fedora-wearing pig, including the hatless pig.
when in Match mode (discussed in a bit), you can activate or deactivate
baseline keypoints via the training image, again either individually or within selection
rectangles. (That is, you can turn keypoints on or off in the training image to have their
corresponding keypoints in the baseline activated or deactivated.) The training image in
unaffected. Instead, the best-match correspondence lines connecting the two images
determine which baseline keypoints are affected.
Matching: Matching is the process of determining best keypoint matches between the current baseline object and the training image, and displaying those matches. It is intended as an aid in letting you determine which points to include in your keypoint profile (that is, active keypoints).
With one or more baseline keypoints active, click the Match button. TestArchitect responds by calculating all the keypoints in the training image. Then, for each active keypoint in the baseline image, it determines the closest match in the training image, based on the contents of each keypoint descriptor, and renders a gray best-match correspondence line between each associated pair of keypoints.
With the lines displayed, you can decide which baseline keypoints to keep active, and
which to deactivate. The figure below, for example, suggests two active keypoints that
might be good candidates for removal.
When in Match mode, the Match button changes to Clear. Click Clear to remove the correspondence lines and exit Match mode.
Object detection: With your tentative settings for active keypoints established,
click Find Object. This activates keypoint-based object detection,
using your current keypoint profile, along with the similarity threshold set in the
Min Accuracy (%) box, to find copies, or variants, of the
baseline object. The process generally takes a few seconds, during which a progress bar
appears at the bottom of the dialog box:
Upon completion of object detection, the training image is overlaid with circles
indicating where TestArchitect found matching
object(s). Inside each circle is a label indicating its degree of similarity with the
baseline object, and another label specifying its x,y location.
Once you have tweaked everything to your satisfaction (established an active keypoint set,
set a similarity threshold in Min Accuracy (%), and tested your
settings with one or more training images), click OK in the
Key Points Modification Tool. You are taken back to the associated
baseline tab, where you can see that your settings for Min Accuracy
and the active keypoint set (Key Point box) have been recorded: