How to use AWS with your C++ application

Nowadays AWS is a known cloud computing service provider. AWS is used in a myriad of applications, to reduce hardware- and development costs. The main benefit of shifting complex functionality (e.g. load balancing, scaling, queuing) from your environment to AWS, is that you’ll be able to focus 100% on your product. You don’t have to thinker about a stable infrastructure, and you will not need to hire highly skilled system administrators which cost you a fortune and rune your business at the end. AWS is the service we’ve been waiting for.

AWS services

However using AWS in your software application can be tough, especially when programming in C++. Most of the programming languages (including C++) are backed up with a SDK. Those SDKs allow you to communicate with the different AWS services by having a transparent and easy-to-understand layer to program against. Well-written documentation is available for most of the programming languages; unfortunately not for C++.

So there’s a SDK for C++, why this blog post then?

The lack of well-written documentation, forced me to write this article. In this post I’ll show you a strategy how to easily implement AWS and use the S3 service in your C++ application. Without the need of advanced libraries and just plain C++ functionality.

What’s available on the market?

The first step before creating the final product/application, is doing an investigation of what’s available on the market:

  • Don’t reinvent the wheel. Probably someone else already solved your problem.
  • Can I find useful information or guidelines?
  • What are the well-known pitfalls?
     

As mentioned before there is an official C++ SDK available but the documentation describing the SDK is too cryptic; at least for me. I even didn't found any clear tutorial, describing the different steps that need to be taken, to interact with AWS properly. Another disadvantage is that it depends on newer C++ 11 features.

Besides the official SDK, other third-party libraries are available, but those are in most of the cases written in pure C, and again without eloquent documentation.

What's next

After doing my research, my goal was to use the above mentioned libraries, to upload files to my AWS S3 bucket. I've spend hours finding out how the different libraries worked. Unfortunately I stopped, because I really didn't know what I was doing; it was merely trial and error.

The biggest problem for me was the compilation. The official SDK depends on C++ 11, and my current project uses dependencies which depends on older versions.

The positive thing is that the project is compiled with CMake, and that I can easily intregrate it in my current project; despite the requirement of the C++ 11 features. Another problem is that the SDK is huge, and that you will need to strip the CMakelists file manually, if you only want to use a specific service, like S3.

REST API

The libraries I was using, where black-boxes for me, I didn't know what to do, and even I worse if it did work I didn't know why. I was determined to find out the truth behind that dark magic.

I bounced on the AWS REST API; this is basically, where those previous libraries are built on. After doing some research, and yes there's a lot information available for this one, I've found the different steps to upload a file to my S3 bucket. It is as simple as calculating an Authorization header, and performing an HTTP request to the correct endpoint. I was relieved..

Authorization header

Almost every application requires some form of authentication. The strategy of this authentication model depends on the application itself.

Traditional webapplications will guide you to a login page and ask you for your username and password. After submitting your credentials it is validated by the webserver, and when succesful a session is stored on the server, and a cookie is set in your browser.

Modern applications which rely on a REST API use a different technique: tokens. The process is similar to the traditional example: users are guided to a login page and need to enter their credentials. The main difference is that when signed in, no session is stored on the server (which verified the authentication). Instead an authorization key is generated on the server and returned to the browser. This key is then send with every subsequent request to authenticate.

This last scenario is also the case how the AWS REST API works. An authorization header is calculated for each request, the idea is that details of the request are encrypted with your AWS secret key, and afterwards decrypted by AWS using your public key. By doing this AWS can verify if you're the correct owner of the AWS service, as you're expected to be the only owner of the credentials.

An example of how the Authorization header looks like:


Authorization: AWS AKIAIOSFODNN7EXAMPLE:frJIUN8DYpKDtOLCwo//yllqDzg=****

To calculate the authorization header you'll need two values:

  • AWS Secret Key
  • A string to sign
     

The formula:


=SHA1(UTFENCODE(stringToSign), secretKey)

The secretKey is the AWS Secret key you can find on your AWS profile page. The string to sign is a string which contains multiple fields, which specify the type of the request. An image is worth a thousand words.

String to sign structure

E.g. if we want to put an object to our S3 bucket, the string to sign will look like:


PUT\n
\n
\n
Thu 12 dec 2015, 23:12:40\n
/bucketName/file.jpg

C++ code

So how does this look in C++? The source looks straight forward, the only thing we need is to include some extra libraries in our project and that's it.


string S3::authorize(const string request)
{
    BYTE digest[20];
    CHMAC_SHA1 hmac_sha1;
    hmac_sha1.HMAC_SHA1((BYTE*)request.c_str(), request.size(),(BYTE*)m_privateKey.c_str(),m_privateKey.size(),digest);
    string signature(base64_encode((unsigned char*)digest,20));
    return "AWS " + m_publicKey + ":" + signature;
}

HTTP request

Now we've managed to create the Authorization header, the next thing will be to create the actual request. We'll use the Curl library, which is installed on every OS natively, and is commonly used in every kind of programming language and framework.

To put an object to our S3 bucket, we'll end with following Curl request:


bool S3::put(const string & url, const vector & headers, const vector & body)
{
    // Initialize curl 
    curl_global_init(CURL_GLOBAL_ALL);
    CURL* curlHandle = curl_easy_init();
    CURLcode result;

    // Initialize headers
    curl_slist* httpHeaders = NULL;
    for(int i = 0; i < headers.size(); i++)
    {
        httpHeaders = curl_slist_append(httpHeaders, headers[i].c_str());
    }

    // Intialize body
    string file = body[0];
    FILE * fd = fopen(file.c_str(), "rb"); /* open file to upload */  
    struct stat file_info;
    fstat(fileno(fd), &file_info);

    // Execute
    std::string output = "";
    if(curlHandle) 
    {
        curl_easy_setopt(curlHandle, CURLOPT_FOLLOWLOCATION,1);//Set automaticallly redirection
        curl_easy_setopt(curlHandle, CURLOPT_MAXREDIRS,1);//Set max redirection times
        curl_easy_setopt(curlHandle, CURLOPT_HTTP_VERSION, CURL_HTTP_VERSION_1_1);//Set http version 1.1fer
        curl_easy_setopt(curlHandle, CURLOPT_HEADER, true);//Set header true
        curl_easy_setopt(curlHandle, CURLOPT_HTTPHEADER, httpHeaders);//Set headers
        curl_easy_setopt(curlHandle, CURLOPT_URL, url.c_str());//Set URL
        curl_easy_setopt(curlHandle, CURLOPT_TRANSFER_ENCODING, 1L); 
        curl_easy_setopt(curlHandle, CURLOPT_CUSTOMREQUEST, "PUT");
        curl_easy_setopt(curlHandle, CURLOPT_UPLOAD, 1L); 
        curl_easy_setopt(curlHandle, CURLOPT_READDATA, fd);
        curl_easy_setopt(curlHandle, CURLOPT_READFUNCTION, reader);
        curl_easy_setopt(curlHandle, CURLOPT_INFILESIZE_LARGE, (curl_off_t)file_info.st_size);
        curl_easy_setopt(curlHandle, CURLOPT_WRITEFUNCTION, write);
        curl_easy_setopt(curlHandle, CURLOPT_WRITEDATA, &output);

        result = curl_easy_perform(curlHandle);//Perform
        curl_global_cleanup();

        return true;
        return (result == CURLE_OK);
    }

    return false;
}

A complete example

You've learned how to create the Authorization header, and that you can use the Curl library to perform an HTTP request. However if you would like to see a complete example, checkout this github repository.

Hey, I'm Cédric a software engineer, who's motivated to broaden his horizon and to discover and learn new methodologies. With a huge interest in web technologies, artificial intelligence and image processing, I try to understand our mountainous world a little bit better.