A more powerful IoT data management in a Ethereum-like Blockchain

Intro

In the previous post concerning the management of some data coming from an ioT device, we had created a smart-contract that could contain all the data exactly as they were transmitted by the device. The data was stored on the blockchain exactly in the sending format. A fundamental problem now arises. How much data are we willing to store in the Blockchain in the case of very frequent arriving data? Is it necessary to create a smart contract for every type of data coming from the device? We can think of storing only the hash of the data and storing off-chain all the data coming from the device. Let's see how.

Storing the data Hash 

Let's look at the picture below and then describe it in detail:

1) the IoT Device sends its data in the default format for that specific device

2) the IoT Hub (in our case an Asp.net Core Web application) receives data from the device, creates the Hash and then sends (as parameters of a smart contract method)  the hash and an array of strings containing all the data received to the blockchain 

3) The BlockChain creates a smart contract that contains the newly transmitted hash

4) This causes an event that retransmits the input data (concatenated by means of a special character, in our case an asterisk). In this way we are sure that the blockchain has received exactely those data that must obviously coincide with the data received from the device.

5) A record is created on an off-chain Data Base that will contain the newly created Blockchain address and the transmitted data linked by the special character ('*')

What are the advantages of this approach? Surely only the hashes of the data are stored on the blockchain. It simplifies the type of smart contract used which will have only the field containing the data hash, therefore the smart contract can be used for all types of devices, without having to create a type of smart contract for each different data format.

The smart contracts

The first smart contract that we see is the smart contract that contains the data hash

We must note the first pragma directive. This indicates that we are using a new (experimental) version of the ABI encoder. Naturally, the Ethereum team's advice is to not use this directive in contracts in production, because always in an experimental phase. Furthermore we know that under certain conditions (certainly rare and complex) this directive can introduce vulnerabilities. However, this directive must be used if, for example, we want to pass an array of strings to a method or a constructor as in the case of the smart contract illustrated above.

In the figure above we have instead the first part of the smart contract that manages the storage and retrieval of data in IoTHash contracts. Note how the DataArrived event is defined, which is invoked when the new IoTHash smart contract instance is stored and returns the concatenated data and address of the newly created smart contract.

Managing strings in Solidity

The second part of the smart contract code concerns the methods for managing the creation of the new smart contract.

In input the method receives both the data hash and the string array containing the data coming from the device (the array is not sized, so I can pass an array with any number of elements). In Solidity the string management is really very particular (in the sense that it must be done practically everything by hand) and in this case we have resorted to a particular function of the 'abi' object called encodePack to which we can pass two strings and these are concatenated managing the sequence of the characters of the string as a succession of bytes.

All data is stored off-chain and it will be possible to verify its correctness with the following procedural scheme:

1) The application reads the record with the data to be verified from the off-chain database

2) The hash of the data received at the time from the device and the corresponding Blockchain address is calculated

3) The Blockchain is queried to obtain the hash stored at the address. Recall that this hash was calculated based on the data received at the time the device sent it.

4) if the Hash it's the same then the off-chain data are authentic and unchanged as transmitted by the device.

Conclusion

We have seen how it is possible using arrays of strings, managing any type of data coming from a device, thus being able to always use the same smart contract on all occasions. The only caveat is the use of the experimental pragma directive for using arrays as parameters.

 

Using REST API in Azure Workbench Blockchain

Code

You can find all the code of the Web App WebClientWorkbench with calls to the Azure Workbench Blockchain Rest APIs here (Git Repository)
You can find the HelloBlockchain sample code here

Intro

We started using Microsoft Workbench Blockchain to implement an OpenBedge management application through the Blockchain. Following the official Microsoft tutorial we finally got the Blockchain version recently made available by the Redmond house. Prerequisites to follow the example of the post is the presence of a Workbench Blockchain distribution on Azure and the creation of the HelloBlockchain test application that you can find in the examples available on Git. In this post instead of using the administration app provided by default during the creation of the distribution,a Web App (.NET Core) will be used which, through calls to the Rest API services  will interact with the HelloBlockchain application. The scheme is as follows:

 

Azure Active Directory App registration

Our Web App will use Azure Active Directory for authentication, thus exploiting the Oauth2 integrated authorization mechanism. This must be implemented by registering an app in Azure Active Directory with the features we will see shortly. Once the user has accessed our Web App using the credentials of a User in Azure Active Directory, it means that he has obtained an Authentication Token (Auth Code). In order to invoke the Blockchain APIs, however, the Web App must use its own Authentication Token (Auth Code) to obtain an Access Token to the Blockchain API resource. Once you have the Access Token you can finally use it (by inserting it in the request header) to invoke the Blockchain's Rest API. The scheme is:

Let's start by creating a new Asp.NET core MVC project in Visual Studio and write down the default URL (eg http: // localhost: 51369). Now we're going to register an app in our Azure portal. From the Active Directory menu go to the "Registration App" blade and start recording our app with these features:

It will be essential to provide our newly registered app with two key features. The first is a key called ClientSecret that will be useful to prove your identity, i.e. your credentials, when requesting an Access Token. The second will be the authorization to use the Blockchain API. In this way, when an Access Token is requested, it will be created embedding in it the authorization to use the Blockchain APIs. For the key go to the Properties of the app just registered and enter the blade Keys and produce the key, taking care to immediately store it as soon as it is created otherwise we will not be able to recover it.

At this point we enter the "Required Permission" blade and advance to the selection of the API to be authorized. In the API search text file, enter the name of the API created during the Azure Workbench Blockchain deployment. We kept the default name "Blockchain API" as we can see from the list of registered apps:

It will be shown the possibility of assigning the app to use the blockchain as a Global Administrator or just to access the APIs. We choose this second option:

Finally to complete the operation we must not forget to click on the "Grant Permission" button, otherwise we will have only set the permissions, but we will not have activated them

We note the Application ID and Client Secret. We'll also need to take note of the ID of the Blockchain API app and the base URL by going to see the app's properties. Other information to note is the tenant domain name that we use, the Tenant ID (which is the unique ID of our Azure Active Directory). With this informations we return to our .NET Core Web Application and set the appsetting.json file with these features:

Web App Asp.NET Core

And we come to our web application. Let's try now to test if the flow we have described above really works.

We have available a controller (Accountcontroller) that at the moment of clicking on the "Sign In" it redirects us to the default Microsoft Endpoint to be able to log in to our Azure Active Directory. This allows us to use AAD users who are automatically mapped to Ethereum's address transparently by Workbench.

We insert a Controller called BlockchainController to which we assign an action called Index that will contain a first call to the Blockchain API requesting the list of the Application Objects present in our distribution. Inside the method the code looks like this:

 

// Because we signed-in already in the WebApp, the userObjectId is know
                string userObjectID = (User.FindFirst("http://schemas.microsoft.com/identity/claims/objectidentifier"))?.Value;

                // Using ADAL.Net, get a bearer token to access the WorkbenchListService
                AuthenticationContext authContext = new AuthenticationContext(AzureAdOptions.Settings.Authority, new NaiveSessionCache(userObjectID, HttpContext.Session));
                ClientCredential credential = new ClientCredential(AzureAdOptions.Settings.ClientId, AzureAdOptions.Settings.ClientSecret);
                result = await authContext.AcquireTokenSilentAsync(AzureAdOptions.Settings.WorkbenchResourceId, credential, new UserIdentifier(userObjectID, UserIdentifierType.UniqueId));

                HttpClient client = new HttpClient();
                HttpRequestMessage request = new HttpRequestMessage(HttpMethod.Get, AzureAdOptions.Settings.WorkbenchBaseAddress + "/api/v1/applications");
                request.Headers.Authorization = new AuthenticationHeaderValue("Bearer", result.AccessToken);
                HttpResponseMessage response = await client.SendAsync(request);

                if (response.IsSuccessStatusCode)
                {
                    JsonSerializerSettings settings = new JsonSerializerSettings();
                    String json_string = await response.Content.ReadAsStringAsync();
                    ApplicationReturnType applicationsResponse = JsonConvert.DeserializeObject<ApplicationReturnType>(json_string);

                    List<IQC_WebClient_Workbench.Models.Application> applications = applicationsResponse.Applications
                    return View(applications);
                }

 

Having already logged into my Web App via the Sign In link at the top right, I have already obtained an Authentication Code and therefore the userObjectID is already known, which I will use later to create a unique identifier. I create an AuthenticationContex that uses a Cache to store the already requested Access Tokens and possibly not ask them for each request. In our case we have built a class that inherits from TokenCache that uses the Session to store Access Tokens. I create credentials using the app ID registered in Azure Active Directory and its associated Secret Client. At this point you just have to invoke "AcquireTokenSilentAsync (String, ClientCredential, UserIdentifier)" to get my Access Token.

Once obtained, use it as a header value (header Key = "Bearer") for the request to the Blockchain API. The request uses the url "/api/v1/applications" to which the base URL stored in a key must be placed in the appsettings.json. If everything is OK, the Blockchain API will return the list of Applications in the Workbench distribution

 

How Workbench REST API works

Let's start with the application ID that is important to us. In our case we see that HelloBlockchain has the iD 1. Let's use it to get the application Workflows. Workflows are the flows that describe exactly the status changes that the various contracts of the application make as a result of calls of functions of the related Smart Contract. Our application is extremely simple and contains only one workflow with just one HelloBlockchain type contract. Workbench has a very interesting approach because it considers the application as a finite state machine where each contract precisely follows a workflow passing from one state to another. In our case the application workflow is this:

As we see, once created, the contract goes into the Request state. In this state we can invoke a function of the smart contract (sendResponse) that will bring the contract into the state of Respond. At this point the contract can return to the status of Request again invoking SendRequest and will remain here waiting for a subsequent response. Who can invoke the methods of smart contract? Users obviously belonging to the roles of competence (Requestor and Responder). Users are those of AAD and the Workbench Admin can assign the correct roles from the Dashboard that is created by default at the time of distribution.

In correspondence to all this in the solidity smart contract we can see:

States become an enum in solidity. In the SendRequest function, it is checked that the address Ethereum of the invoker is the same as the Requestor stored at the time the contract is created as shown in the constructor's code:

(Note that the Workbench compiler does not yet support the keyword 'constructor' for which you need to write a function with the same name as the Smart Contract which is rightly reported as a warning from Visual Studio Code). The Requestor and the Responder are the addresses of Ethereum in correspondence of those who make requests and those who answer. The passage of states is described in the HelloBlockchain.json file that must be fed to Workbench when the application is created. But let's focus on the API again, leaving the in-depth description of the Workbench elements in another post.

After recovering the usual Access Token (which in the meantime may have been renewed silently) here is the invocation to receive the workflow of our application:

 HttpClient client = new HttpClient();
                HttpRequestMessage request = new HttpRequestMessage(HttpMethod.Get, AzureAdOptions.Settings.WorkbenchBaseAddress + "/api/v1/applications/1/workflows");
                request.Headers.Authorization = new AuthenticationHeaderValue("Bearer", result.AccessToken);
                HttpResponseMessage response = await client.SendAsync(request);

                if (response.IsSuccessStatusCode)
                {

                    JsonSerializerSettings settings = new JsonSerializerSettings();
                    String json_string = await response.Content.ReadAsStringAsync();
                    WorkflowReturnType workflowResponse = JsonConvert.DeserializeObject<WorkflowReturnType>(json_string);

                    List<IQC_WebClient_Workbench.Models.Workflow> workflows = workflowResponse.Workflows;


                    return View(workflows);
                }

Here is the result of the call:

Similarly for when contracts related to a workflow are required (in our case there is only one contract type - HelloBlockchain):

 HttpClient client = new HttpClient();
                HttpRequestMessage request = new HttpRequestMessage(HttpMethod.Get, AzureAdOptions.Settings.WorkbenchBaseAddress + "/api/v1/contracts?workflowId=1");
                request.Headers.Authorization = new AuthenticationHeaderValue("Bearer", result.AccessToken);
                HttpResponseMessage response = await client.SendAsync(request);

                if (response.IsSuccessStatusCode)
                {
                    JsonSerializerSettings settings = new JsonSerializerSettings();
                    String json_string = await response.Content.ReadAsStringAsync();
                    WorkflowInstancesReturnType workflowistancesResponse = JsonConvert.DeserializeObject<WorkflowInstancesReturnType>(json_string);

                    List<IQC_WebClient_Workbench.Models.Contract> contracts = workflowistancesResponse.Contracts;
                    return View(contracts);
                }

Naturally for the sake of simplicity we have hardcoded some parameters within the URLs. The result of the contracts is this:

Very interesting now is the creation of a new contract that must be made (according to official documentation) through a POST call to the relative URL. After creating a simple form for the collection of the text of the Request Message (remember that the constructor in solidity wants the string of the request message) and taking care also that the user with which we are logged belongs to the role of Requestor, we will be able to make a call to create a new contract. Here is the list of my users and their respective roles taken from the dashboard:

Here is the request form:

At this point the final code. In the documentation it is clear that a Json must be prepared in the WorkflowActionInput format with relative properties containing among other things also the RequestMessage. Once serialized and inserted in the payload of the request the answer that is obtained is the ID of the new contract entered (there is an error in the official documentation at this time).

Pay more attention because in the documentation it is reported that some parameters included in the query string are optional, but in reality after receiving some error messages, investigating with Fiddler it turns out that the parameter workflowId is actually mandatory

After fixing the correct URL and received the new contract ID we make an additional request to view the new contract in a new page:

 HttpClient client = new HttpClient();
                client.BaseAddress = new Uri(AzureAdOptions.Settings.WorkbenchBaseAddress);
                HttpRequestMessage request = new HttpRequestMessage(HttpMethod.Post, "/api/v1/contracts?workflowId=1&contractCodeId=1&connectionId=1");
                request.Headers.Authorization = new AuthenticationHeaderValue("Bearer", result.AccessToken);
                request.Content = new StringContent(jsonObject.ToString(), System.Text.Encoding.UTF8, "application/json");
 
                HttpResponseMessage response = await client.SendAsync(request);

                if (response.IsSuccessStatusCode)
                {

                    JsonSerializerSettings settings = new JsonSerializerSettings();
                    String json_string = await response.Content.ReadAsStringAsync();
                    int newContractID = JsonConvert.DeserializeObject<int>(json_string);

                    HttpRequestMessage newRequest = new HttpRequestMessage(HttpMethod.Get, AzureAdOptions.Settings.WorkbenchBaseAddress + "/api/v1/contracts/" + newContractID);
                    newRequest.Headers.Authorization = new AuthenticationHeaderValue("Bearer", result.AccessToken);
                    HttpResponseMessage newResponse = await client.SendAsync(newRequest);

                    if (newResponse.IsSuccessStatusCode)
                    {
                        String new_json_string = await newResponse.Content.ReadAsStringAsync();
                        Contract newContract = JsonConvert.DeserializeObject<Contract>(new_json_string);

                        return RedirectToAction("ContractDetail", newContract);
                    }

Here is the result:

Summary

In this post we saw how to use the Azure Workbench Blockchain API. The Workbenchclient that can be found in the example codes on Git was not intentionally used because we wanted to highlight in detail the direct calls to the APIs. Surely this technique allows you to build custom applications once you have created smart contract in solidity. We have noticed how, given the extreme simplicity of contract deployment and the management of users and their roles (which makes transparent a series of operations on Ethereum to be carried out to manipulate account addresses), it is initially difficult to disengage from structure of the applications that must follow the system of workflow and finite-state development. For many applications, however, this approach is certainly the best one.

See at the top of this post the link for the shared code.

Happy chain!