RTC (Real Time Communication)
Walkthrough

Note

All the HTTP samples are done with curl program, but any HTTP 1.0/1.1 compatible client should work as described.

This is a full example of how to achieve ingesting an image and gathering the results using Robbie Visio® HTTP API. The walkthrough if you have a curl program installed should not take more than 5 minutes!

1. Authenticate

1.1 Get Credentials

Open a browser and enter the URL, depending on your organization (change yourorganization by the real organization name you registered with):

https://yourorganization.robbie.ai/dashboard/settings/credentials/


Copy the client_id and client_secret properties:

Credentials Settings

1.2 Request oAuth Token

Tip

See full resource spec here oAuth Token

Now let's get the necessary access_token for further API calls:

JSON Body:

1
2
3
4
5
6
7
{
    "client_id": "265d38cfd8f9ee465cc3bd6bd07108d15d659750",
    "client_secret": "56f2a92b0ca04e601e078ca5198bf83ba36cb
    cdb7a88a391767b0df92e32be40",
    "grant_type": "client_credentials",
    "scope": "vision"
}

Curl Command:

1
2
3
4
5
6
7
8
9
$ curl -i https://visio.robbieapis.com/api/v1/oauth/token/ \
    -X POST -H 'Content-Type: application/json' \
    -d '{
        "client_id": "265d38cfd8f9ee465cc3bd6bd07108d15d659750",
        "client_secret": "56f2a92b0ca04e601e078ca5198bf83ba36cbc
        db7a88a391767b0df92e32be40",
        "grant_type": "client_credentials",
        "scope": "vision"
     }'

Command Response:

Finally we got the access_token from the response:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
HTTP/1.1 200 OK
Date: Sat, 03 Jun 2017 20:37:36 GMT
Content-Type: application/json
Content-Length: 552
Connection: keep-alive
Pragma: no-cache
Vary: Accept-Encoding
Cache-Control: no-store
Server: Robbie
Access-Control-Allow-Origin: *
Access-Control-Allow-Methods: GET, POST, OPTIONS

{
    "access_token": "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.
    eyJ0ZW5hbnRfdWlkIjoidmlzaW9uLWIyZGEwMzI3N2M1MjQ4MDY4YTF
    hYzUyNTBlMjM0ODU2IiwiZXhwIjoxODA3NTYyMjU2LCJpc3MiOiJ2aX
    Npb24iLCJzY29wZXMiOlsiY29tbW9ucyJdLCJuYW1lc3BhY2UiOiJwc
    m9maWxlcy52aXNpb24udGVzdCIsIm9yZ2FuaXphdGlvbl91aWQiOiI1
    YTY2NGJlYjkyMjMzNzIwMzUzNTgzNzIzMjEjNzE2NTQ0IiwiYXVkIjo
    idmlzaW9uIiwiYWNjb3VudF91aWQiOiI4NGY2MDlmYTkyMjMzNzIwMz
    UzNTgzNzIzMTkjMDE3NDMzIiwiZW1haWwiOiJkZW1vQHJvYmJpZS5ha
    SJ9.YXGAZ6nHwB_4-7litC8yMTywor7xmLR1bI2vA1FQ8b8", 
    "scope": "vision",
    "token_type": "Bearer", 
    "expires_in": 3600
}



2. Create RTC (Real Time Communication) Content

Now we need to create an RTC Content, which can be understood as a real time session. All images and frames bound to the same RTC Content are considered part of the same session. This is mainly useful for conferences or real time tracking, where images/frames analyzed are related because some reason, i.e the same user during a web conference, the emotional response of a player while he or she in a certain app screen, etc.

Perform the following command call:

Tip

See full resource spec here RTC Resource

Body:

1
2
3
{
    "source": "HTTP"
}

Curl command:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
$ curl -i https://visio.robbieapis.com/api/v1/rtc/create/ \
    -X POST -H "Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.
    eyJ0ZW5hbnRfdWlkIjoidmlzaW9uLWIyZGEwMzI3N2M1MjQ4MDY4YTFhYzUyNTBlMjM0OD
    U2IiwiZXhwIjoxODA3NTYyMjU2LCJpc3MiOiJ2aXNpb24iLCJzY29wZXMiOlsiY29tbW9u
    cyJdLCJuYW1lc3BhY2UiOiJwcm9maWxlcy52aXNpb24udGVzdCIsIm9yZ2FuaXphdGlvbl
    91aWQiOiI1YTY2NGJlYjkyMjMzNzIwMzUzNTgzNzIzMjEjNzE2NTQ0IiwiYXVkIjoidmlz
    aW9uIiwiYWNjb3VudF91aWQiOiI4NGY2MDlmYTkyMjMzNzIwMzUzNTgzNzIzMTkjMDE3ND
    MzIiwiZW1haWwiOiJkZW1vQHJvYmJpZS5haSJ9.YXGAZ6nHwB_4-7litC8yMTywor7xmLR
    1bI2vA1FQ8b8" -X POST \
    -d '{ 
            "source": "HTTP" 
        }' \
    -H "Content-Type: application/json" 

Command Response:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
HTTP/1.1 201 Created
Date: Sat, 03 Jun 2017 21:35:38 GMT
Content-Type: application/json
Content-Length: 192
Connection: keep-alive
Vary: Accept-Encoding
Server: Robbie

{
    "id":"36df5d399223372035358250069#530959"
}



3. Ingest Image

Note

Robbie Visio® HTTP API only accepts JSON1 format, and this means images cannot be attached or sent as binary. Due to JSON spec images or binary files are not supported, so in the examples we will use @base64_encoded_image parameter that must be following examples with the string representing the image encoded in Base642 to work.

Know more about Base642 format and JSON spec1

In this example, we want to track frames for the session created (from the previous step, content=36df5d399223372035358250069#530959) with the following labels:

1
2
3
4
5
6
7
8
9
User labels:

- id: joedoe
- age: 45

General Labels:

- city: Barcelona
- center: BCN7895

Perform this following call for every image or frame you want to process:

Tip

See full resource spec here RTC Resource

JSON Body:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
{
   "labels": {
        "$id": "ANONYMOUS",
        "$age": "45",
        "city": "Barcelona",
        "center": "BCN7895"
    },
   "timestamp": "2014-08-11T05:26:03.869245+00:00", 
   "content": "36df5d399223372035358250069#530959",
   "image": "@base64_encoded_image"
}

Curl Command:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
$ curl -i https://visio.robbieapis.com/api/v1/rtc/create/ \
    -X POST -H "Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.
    eyJ0ZW5hbnRfdWlkIjoidmlzaW9uLWIyZGEwMzI3N2M1MjQ4MDY4YTFhYzUyNTBlMjM0OD
    U2IiwiZXhwIjoxODA3NTYyMjU2LCJpc3MiOiJ2aXNpb24iLCJzY29wZXMiOlsiY29tbW9u
    cyJdLCJuYW1lc3BhY2UiOiJwcm9maWxlcy52aXNpb24udGVzdCIsIm9yZ2FuaXphdGlvbl
    91aWQiOiI1YTY2NGJlYjkyMjMzNzIwMzUzNTgzNzIzMjEjNzE2NTQ0IiwiYXVkIjoidmlz
    aW9uIiwiYWNjb3VudF91aWQiOiI4NGY2MDlmYTkyMjMzNzIwMzUzNTgzNzIzMTkjMDE3ND
    MzIiwiZW1haWwiOiJkZW1vQHJvYmJpZS5haSJ9.YXGAZ6nHwB_4-7litC8yMTywor7xmLR
    1bI2vA1FQ8b8" -X POST \
    -d '{
           "labels": {
                "$id": "joedoe1982",
                "$age": "45",
                "city": "Barcelona",
                "center": "BCN7895"
           },
           "timestamp": "2014-08-11T05:26:03.869245+00:00",
           "content": "36df5d399223372035361315390#889779" ,
           "image": @base64_encoded_image
      }' \
    -H "Content-Type: application/json" 

Command Response:

Finally we got the emotional output for the frame sent from the response:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
HTTP/1.1 100 Continue
HTTP/1.1 201 Created
Date: Sat, 03 Jun 2017 22:01:41 GMT
Content-Type: application/json
Content-Length: 294
Connection: keep-alive
Vary: Accept-Encoding
Server: Robbie


{
    "BASIC": {
        "DISGUST": 0.0,
        "HAPPINESS": 0.47,
        "SADNESS": 0.02, 
        "FEAR": 0.108, 
        "ANGER": 0.023, 
        "NEUTRAL": 0.116, 
        "SURPRISE": 0.262
    },
    "MAIN-NON-BASIC": "NON_BOREDOM",
    "FRAME": { 
         "id": "0#9223372035447041044869"
    },
    "TIME":1407734763.8692450523,
    "NON-BASIC": {
        "BOREDOM": 0.055,
        "NON_BOREDOM": 0.945
    },
    "MAIN-BASIC":"HAPPINESS"
}

4. Congratulations!

You have successfully:

  • authenticated
  • created a real time session
  • sent an image with some labels
  • and obtain the emotional output to take action

The events should start showing up into the dashboards!

Warning

NON-BASIC emotions and other advanced features might be ony available for premium tiers. Please see Pricing or contact sales@robbie.ai.

Segmentation-1-example