Getting Started with the ADLINK Edge Intel Arc Welding Solution

Reading Time: 13 minutes

Solution Architecture

The ADLINK Edge Intel Arc Welding Solution is comprised of several applications which communicate via the ADLINK Data River. Together these applications ingest video from cameras, perform inferencing on the captured images and output the inference results for other applications to consume.

Optionally, additional ADLINK Edge applications can be added to the solution to enable bi-directional communication with OT devices such as PLCs.

The solution monitors a video stream and performs automatic visual detection of weld defects and can then act on their occurrence if required. An example action would be to stop the weld automatically to reduce the costs associated with wasted material.

The use case of the ADLINK Edge Intel Arc Welding Solution can be further broken down into the following:

The core applications are:

Additionally, the VMLink desktop application forms part of the solution and can be used to:

Setting up the Edge Device

Edge Device Requirements

Intel OpenVino supports only x86 based machines so the target edge device must be of this architecture in order to deploy the profile. Detailed information on hardware requirements can be found on the Intel OpenVino website.

Ubuntu 18.04 LTS is recommended as the base operating system of your edge device.

ADLINK Edge Profile Builder can be installed on either the edge device or the PC used for development. To install ADLINK Edge Profile Builder follow the relevant guide below for your OS.

VMLink is a useful tool for viewing video streams along with associated inference results. It can also be used to collect images for training AI models. VMLink is currently available for Ubuntu 18.04 on x86 and ARM64.

For more information on installing VMLink please see How to Install VMLink on Ubuntu 18.04 LTS.

Installing IDS Software Suite on an Edge Device

The ADLINK Edge Intel Arc Welding Solution has been tested using IDS uEye cameras, as such, instructions in this guide will assume that you are using an IDS uEye camera. If you require assistance with camera selection, configuration, and mounting options, please contact ADLINK for further information.

In order to utilise an IDS uEye camera within the solution, you must download and install the IDS Software Suite for your specific camera model from the IDS website, this can be found by clicking here.

Before the ADLINK Edge applications making up the solution can be deployed, the applications must first be configured using ADLINK Edge Profile Builder.

To assist with this ADLINK provides the following template profile for download within Profile Builder:

Downloading the Profiles

From within a project in ADLINK Edge Profile Builder click on Add profile and select Download a profile. Click the Next button.

Ensure the ADLINK-Templates repository is selected and select the intel-arc-welding-x86 profile. Click Next.

The profile will now be downloaded.

When the download is complete the profile will be available to edit in Profile Builder.

Deploying the Profile

Configuring the Profile

The profile includes the following applications:

Other ADLINK Edge and Docker applications can be added to the profile as required. For example, the ADLINK Edge Modbus Connect application can be added to support connections to Modbus devices. Please note these additional applications may require separate licenses to be purchased.

Frame Streamer Application

The Frame Streamer (aea-frame-streamer) application is included as part of the profile in order to stream a demo welding video to the DataRiver. It is pre-configured to stream the /videos/weld.mp4 video file to weldStream1. If you choose to utilise a uEye camera, remove this app from the profile.

To remove Frame Streamer from the profile:

  1. Click the three dots in the top right corner of the frame-streamer app tile
  2. Click “Delete Application” then “Delete forever”

Deep Learning Streamer Inference Engine Application

The Deep Learning Streamer (aea-deep-learning-streamer) application ingests video frames from a number of different sources and performs inferencing on those frames. It also provides the ability to record videos and images to the file system when configurable trigger events occur.

Configuring the input video stream

The profile comes pre-configured to ingest video frames from the DataRiver on weldStream1. If you wish to utilise a uEye camera the following configuration settings must be changed:

The configurationPath setting above allows you to specify a uEye camera configuration ini file. This file can be exported from IDS Camera Manager. This setting is optional, but if you choose to set a configuration path, the corresponding file should be added to the application’s profile by adding the file to /camera-configs/config.ini using the Files tab.

Configuring event capture

The Captures section configures the conditions which trigger video or image recording. The default configuration included with the profile will capture video files and inference metadata whenever a normal_weld or porosity_weld is detected, recording the entirety of each weld session automatically. It will store these files in /adlinkedge/capture/weldStream1 within the container. This path is mounted to the local file system of the profile at ./capture/weldStream1.

If you wish to change the default configuration the following settings are of interest:

Training Manager Application

The Training Manager (aea-training-manager) application monitors a directory for images, videos, and inference result files captured by Deep Learning Streamer. It then transfers them to either AWS S3 or an FTP server. By default, only the directory is configured, to the same location that Deep Learning Streamer is configured to output its capture files: /adlinkedge/capture/weldStream1. Any file that is detected in that directory will be uploaded automatically to your configured destination.

To configure the application to upload the files to the desired destination, you must add a new AWSS3 or FTPServer destination section to the config.

Once a destination is configured, the app will delete files from the local file system automatically after upload, to disable this, set DeleteFilesAfterUpload to false.

Configuring an AWS S3 bucket

When you configure AWSS3, you must also provide your AWS credentials as environment variables. The AWS user that you configure must have permission to write to the bucket. More information on AWS credentials can be found in this article and information on S3 policies and permissions can be found in this article. Once you have the required credentials, go to the Docker tab of the Training Manager application and add your credentials as environment variables named AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY.

Configuring an FTP server

If you configure SFTP with a public and private key, be sure to add the keys to the Training Manager container in the Files tab.

Node-RED application

The Node-RED application does not need to be configured. By default, it comes complete with an example flow file that receives inference results from the default weldStream1 stream and displays the current weld status on a local web dashboard, accessible at http://<Device IP>:1880/ui. If the flow is subsequently changed by users, the flow can be exported and added to the application’s profile by adding the file to /adlinkedge/config/flows.json using the Files tab.

Exporting the Profile

Once the profile has been configured it can be exported from Profile Builder as a docker-compose bundle.

  1. On the intel-arc-welding-x86 profile page click Deploy
  2. On the window that appears, choose Download a docker-compose and click Next

Downloading the AI model and demo video file

ADLINK provides a zip archive containing the following files to support the deployment of the profile:

Click here to download the supplementary zip archive.

Running the Profile

After exporting your profile as a docker-compose bundle and downloading the supplementary zip archive in the above two steps, you must transfer the files to your edge device. Assuming a Ubuntu 18.04 LTS base operating system, this can be done using ssh. The following commands demonstrate this, replace user with your user name on the device, and edgedevice with the IP address or hostname of the device:

scp intel-arc-welding-x86.zip user@edgedevice:
scp intel-arc-welding-x86-resources.zip user@edgedevice:

After transferring the files, connect to the device Edge device via SSH and extract the files into the required directory structure:

ssh user@edgedevice

sudo apt install unzip
unzip intel-arc-welding-x86.zip -d intel-arc-welding-x86
unzip intel-arc-welding-x86-resources.zip -d intel-arc-welding-x86

You should end up with the below directory structure:

user@edgedevice:~$ tree intel-arc-welding-x86
intel-arc-welding-x86
├── deep-learning-streamer
│   └── adlinkedge
│       └── config
│           └── DeepLearningStreamer.xml
├── docker-compose.yaml
├── frame-streamer
│   └── adlinkedge
│       └── config
│           └── FrameStreamer.xml
├── models
│   ├── weld_porosity_detection.bin
│   ├── weld_porosity_detection.json
│   ├── weld_porosity_detection.labels
│   └── weld_porosity_detection.xml
├── node-red
│   └── adlinkedge
│       └── config
│           └── flows.json
├── training-manager
│   └── adlinkedge
│       └── config
│           └── TrainingManager.xml
└── videos
    └── weld.mp4

14 directories, 10 files

To run the profile, first, install docker-compose, and then, from within the directory you extracted the files to, use docker-compose to bring up the profile:

sudo apt install docker-compose
cd intel-arc-welding-x86
sudo docker-compose up -d

To list running containers you can use the following command:

sudo docker ps

To check the logs for a running container you can use the following command:

sudo docker logs deep-learning-streamer

Once you are finished running the profile, you can bring it down with the following command:

sudo docker-compose down

Viewing the Inference Results

In addition to displaying video frames from frame streamer applications, VMLink automatically overlays any inference results associated with the stream on top of the images.

Video streams can be viewed within VMLink using the Streamer application.

Viewing the saved captures and inference data

With the default configuration, when capture is triggered in Deep Learning Streamer, the video and inference metadata files will be saved in the ./capture/weldStream1 directory within the extracted profile directory.

Viewing the Inference Results in Node-RED

An example Node-RED Dashboard is included with the inference profile and can be viewed by navigating to the following URL in a web browser:

http://<Device IP>:1880/ui

The dashboard shows the current weld status of weldStream1.

Node-RED Inference results post-processing

To reduce the chance of false positive results, the Node-RED flow performs some post-processing of inference results. The dashboard UI provides a few settings to configure this:

When the number of porosity classifications in a row required to trigger a porosity detection has been reached, the Node-RED flow will display Porosity Weld Detected on the dashboard. The DataRiver out node will then write to the PorosityEvent tag group shown below. The PorosityDetected tag will be set to true.

{
  "name": "PorosityEvent",
  "context": "com.adlinktech.weld",
  "qosProfile": "event",
  "version": "1.0.0",
  "description": "A tag group to indicate porosity events for an arc welder.",
  "tags": [
    {
      "name": "PorosityDetected",
      "description": "True if porosity is detected, false if no porosity is detected.",
      "kind": "BOOLEAN",
      "unit": ""
    }
  ]
}

A number of seconds after the last porosity detection, the Node-Red flow will display back to “No issues” on the dashboard. The DataRiver out node will then write to the PorosityEvent tag group with the PorosityDetected tag set to false.

Acting on Inference Results

When the Node-RED flow writes to the PorosityEvent tag group on the DataRiver, it provides an event that you may choose to act upon. One action that you may want to perform is to stop the arc welder when a weld defect is detected.

When a sample is received on the PorosityEvent tag group where the PorosityDetected tag is set to true; you can send a signal to the arc welder to stop via Modbus or OPC-UA.

The protocol choice will be dependent on the specifications of your arc welder. Depending on your protocol choice, the following sections describe how to configure either the Modbus or OPC-UA apps.

Modbus

Adding the Modbus app to your profile

First, you must add the Modbus app to the profile in Edge Profile Builder.

  1. From the intel-arc-welding-x86 profile page, click Add application.
  2. Search for aea-modbus and click on the application tile.
  3. On the aea-modbus application page, choose version 3.0.0 and click Add application.
  4. On the Adding aea-modbus panel that appears, ensure the intel-arc-welding-x86 profile is selected, enter the name modbus, and click Add.

Configuring the Modbus app

The following XML shows an example configuration of the Modbus app which will write the value of the PorosityDetected tag to a Modbus coil. For Further information on individual configuration settings, refer to the Modbus app documentation by clicking on the Documentation tab within Profile Builder.

<?xml version="1.0" encoding="UTF-8"?>
<Modbus xmlns="http://www.adlinktech.com/vortex-edge/0.9/Modbus">
  <MODBUSConfiguration>
    <Connection>
      <TCP>
        <IP>127.0.0.1</IP>
        <Port>502</Port>
      </TCP>
      <SlaveID>1</SlaveID>
    </Connection>
    <Publication>
      <FunctionCode>FC05 Write Single Coil</FunctionCode>
      <StartAddress>0</StartAddress>
      <Slices>
        <Name>modbus.PorosityDetected</Name>
        <Kind>UINT8</Kind>
        <StartIndex>0</StartIndex>
        <Expression>tagGroup.PorosityEvent.PorosityDetected</Expression>
      </Slices>
    </Publication>
  </MODBUSConfiguration>
  <DataRiverConfiguration>
    <Thing>
      <ContextID>com.adlinktech.modbus</ContextID>
      <Class>
        <Context>com.adlinktech.modbus</Context>
        <Version>v1.0</Version>
        <Input>
          <TagGroupPath>file:///PorosityEventTagGroup.json</TagGroupPath>
          <PublicationAlias>PorosityEvent</PublicationAlias>
        </Input>
      </Class>
    </Thing>
  </DataRiverConfiguration>
</Modbus>

Click View as XML on the Modbus app configuration page and paste in the above XML. Download the PorosityEventTagGroup.json file and add it to the root of the Modbus app using the Files tab.

Repeat the Exporting the Profile and Running the Profile steps to run your modified profile. You should now see the value of your Modbus coil change from 0 to 1 when weld defects are detected.

OPC-UA

Adding the OPC-UA Client app to your profile

First, you must add the OPC-UA Client app to the profile in Profile Builder.

  1. From the intel-arc-welding-x86 profile page, click Add application.
  2. Search for aea-opc-ua-client and click on the application tile.
  3. On the aea-opc-ua-client application page, choose version 1.1.0 and click Add application.
  4. On the Adding aea-opc-ua-client panel that appears, ensure the intel-arc-welding-x86 profile is selected, enter the name opc-ua-client, and click Add.

Configuring the OPC-UA Client app

The following XML shows an example configuration of the OPC-UA Client app which will write the value of the PorosityDetected tag to an OPC-UA node. For Further information on individual configuration settings, refer to the OPC-UA Client app documentation by clicking on the Documentation tab within Profile Builder.

<?xml version="1.0" encoding="UTF-8"?>
<OPCUAClient xmlns="http://www.adlinktech.com/vortex-edge/0.9/OPCUAClient">
  <OPCUAConfiguration>
    <ServerConnection>
      <OPCUAServerEndpoint>opc.tcp://localhost:53530/OPCUA/Server</OPCUAServerEndpoint>
    </ServerConnection>
    <Publication>
      <Alias>PorosityDetected</Alias>
      <NodeNamespace>0</NodeNamespace>
      <NodeIDString>PorosityDetected</NodeIDString>
    </Publication>
  </OPCUAConfiguration>
  <DataRiverConfiguration>
    <Thing>
      <Class>
        <Context>com.adlinktech.opcua</Context>
        <Input>
          <InputExpression>
            <Name>PorosityEvent</Name>
            <Context>com.adlinktech.weld</Context>
          </InputExpression>
          <Mapping>
            <TagName>PorosityDetected</TagName>
            <PublicationAlias>PorosityDetected</PublicationAlias>
          </Mapping>
        </Input>
      </Class>
    </Thing>
  </DataRiverConfiguration>
</OPCUAClient>

Click View as XML on the OPC-UA Client app configuration page and paste in the above XML.

Repeat the Exporting the Profile and Running the Profile steps to run your modified profile. You should now see the value of your OPC-UA node change from false to true when weld defects are detected.