How Userful helps organizations integrate thermal cameras | Part 2

In part 1 of this blog, we described how thermal cameras and other new technologies will become second nature in the post-COVID world. For this part we'll showcase an example setup.

We used a video management server (VMS) with a software platform that integrates with cameras from hundreds of different vendors—including cameras supporting in-camera analytics and/or thermal imaging. Users can configure events inside the VMS platform or inside a smart-camera's rule engine. These events can be object-distance triggers (i.e. detection of objects with a distance less than 1.5 meter between them), object counting triggers (i.e. detection of a certain number of visitors inside a location) or temperature triggers (i.e. detection of objects with temperature above 38 degrees centigrade).

Any of these events can trigger an HTTP call to the Userful API, allowing the Userful Server to automatically change the content on a video wall, an individual display or a group of displays. How simple this is to deploy? It only takes three steps:

1. Configure the events on the camera built-in analytics engine or on the software based VMS platform.
2. Configure a HTTP listener; to monitor incoming HTTP calls coming from external sensors.
3. Configure a source switcher; a set of software instructions for the Userful API about what content to display on the different screens based on the external trigger. This can be text-warnings, an instruction video, live camera feeds, a powerpoint slide or any other type of content the customer decides to display.

The end result (using Python) contains 26 lines of code in the main application: 

if __name__ == '__main__':

    # Bind the socket to the port        
    sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    server_address = ('192.168.1.100', 8081)
    print ('starting up on %s port %s' % server_address)
    sock.bind(server_address)
    
    # Listen for incoming connections
    sock.listen(1)    
    while True:
        # Wait for a connection coming from external sensor
        print ('waiting for a connection')
        connection, client_address = sock.accept()
        try:
            print ('connection from', client_address)
            # Receive the data in small chunks and read it
            while True:
                data = connection.recv(512)
                if data: 
                    # extract the data coming from the Camera HTTP call. 
                    camera = extract(data)
                    # login to Userful Server to retrieve authentication cookie                   
                    session_header = login()
                    # determine what is currently playing on the displays
                    current_source = is_playing(session_header, 'Shop-Floor')
                    # switch content on the displays to a prefconfigure source (in 
                    # this example an HTML Warning message
                    switch_source(session_header,'shop-Floor','HTML Warning')
                    # wait for 5 seconds 
                    time.sleep(5)
                    # switch back to the content that was originally playing
                    switch_source(session_header, 'Shop-Floor', current_source)
                    #delete_source(session_header,'PIP_Dynamic')    
                    time.sleep(5) 
                    break
                else:
                    print (sys.stderr, 'no more data from', client_address)
                    break           
        finally:
            # Clean up the connection
            connection.close()

So let's run through this project step for step.

Step 1: Configuring events on the camera or VMS

In this example we use the integrated analytics engine of our third party VMS platform that receives and records the camera feeds from multiple cameras. Administrators can configure different events inside the camera rule engine… This can be a movement trigger or an analytics trigger whereby the VMS software analyses the incoming video feeds and detect events for each camera, for example:

1. A person entering a specific area of the recorded window.
2. A person crossing a virtual line from A -> B (visitor counter)
3. A person crossing a virtual line from B -> A (visitor counter)
4. A person with elevated body temperature (> 38 degrees)

Each event has a call to action; for example an HTTP REQUEST to our web listener at 192.168.1.100:8081 with message contents like the camera name and event qualification. An example event rule is shown in the picture below.

Step 2: the HTTP Listener

An HTTP web server is a simple process that is running on your machine and does exactly two things:

1. Listens for incoming http requests on a specific TCP socket address (IP address and a port number)
2. Handles this request and acts when a message is received on the socket address.

In this example; our listener is listening to port 8081 on it's external Ethernet interface on IP address 192.168.1.100. If incoming connections are detected, the data received in the HTTP message is collected and processed under a <call to action> routine. This is the routine of instructing the Userful server what to do. Enter your text here ...

# Bind the socket to the port        
    import socket
    sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    server_address = ('192.168.1.100', 8081)
    print ('starting up on %s port %s' % server_address)
    sock.bind(server_address) 
    # Listen for incoming connections
    sock.listen(1)   
 
    while True:
        # Wait for a connection coming from an external sensor
        print ('waiting for a connection')
        connection, client_address = sock.accept()
        try:
            print ('connection from', client_address)
            # Receive the data in small chunks and read it
            while True:
                data = connection.recv(512)
                if data: 

                else:
                    print (sys.stderr, 'no more data from', client_address)
                    break           
        finally:
            # Clean up the connection
            connection.close()
  

Before we switch our source on the screens, we check what source is currently playing in a zone (i.e. group of screens). That way we can return to the original content once an event has passed. The code below will detect the name of the currently playing source (i.e. signage player name, or other). When calling this function, we pass along our authentication cookie and zone name.

  def is_playing (session_header, zone):
    get_url = api_url_base + '/api/zones/byname/' + zone
    response = requests.get(url=get_url, headers=session_header)
    if response.status_code == 200:
        data = response.content
        dict = json.loads(data)
        sourceID = dict['playingSourceId'] 
        sourceName = get_source_info(session_header,sourceID)
        print(sourceName, 'is playing')
    else:
        print (response.status_code)
    return sourceName

def get_source_info (session_header, sourceID):
    get_url = api_url_base + '/api/sources/' + sourceID
    response = requests.get(url=get_url, headers=session_header)
    if response.status_code == 200:
        data = response.content
        dict = json.loads(data)
    else:
        print (response.status_code)
    return dict['sourceName'] 

  

Now that we have determined the name of the source currently playing, we can change the source for that zone:

  def switch_source(session_header, zone_name, source_name):
    post_url = api_url_base + '/api/zones/byname/' + zone_name + '/switch?destinationSourceName=' + source_name
    response = requests.put(url=post_url, headers=session_header)
    value = False
    if response.status_code == 200:
        data = response.content
        value = json.loads(data)["isPlaying"]
        print('Source Switched')
    else:
        print (response.status_code)
    return value 

  

And that is it…. The <CALL TO ACTION> in our HTTP Listener becomes: (1) retrieve the AUTH cookie for the Userful Server, (2) Detect current source playing in a particular zone, (3) Switch the content on the displays to a 'HTML Warning' message stored on the local server (4) Pause for some time, (5) Switch back to the original source.

  # login to Userful Server to retrieve authentication cookie                   
                    session_header = login()
                    # determine what is currently playing on the displays
                    source = is_playing(session_header, 'Zone-3')
                    # switch content on the displays to a prefconfigure source (in this example an HTML Warning message
                    switch_source(session_header,'Zone-3','HTML Warning')
                    # wait for 5 seconds 
                    time.sleep(5)
                    # switch back to the content that was originally playing
                    switch_source(session_header, 'Zone-3', source)
                    #delete_source(session_header,'PIP_Dynamic')    
                    time.sleep(5) 
                    break 

  

We can expand this script by actually looking at the data that was received in the HTTP call coming from the camera or the VMS platform. This data can contain information about the camera and/or trigger source and this information can be used to display different types of content. Think of live camera feeds or - if the camera is counting people - displaying how many customers are in the shop and how many additional customers can enter the store.