In my previous post I looked at a basic use of plain HTTP in a NiFi ingest pipeline. In practice however, an encrypted communication channel is an imperative. In an ideal world, switching to HTTPS is easy, but in reality we frequently face SSL errors of various kinds. This post shows how to go about establishing trust and identity verification checks.
NiFi allows to configure TLS / SSL by the means of a StandardSSLContextService.
First of all, let’s consider a server whose certificate is not trusted by the client’s browser.
Problem #1: Certificate is not Trusted
Certificate is not trusted.
To simulate the problem, I use a WireMock proxy to JSONPlaceholder – a fake API server. Conveniently, WireMock relies on a self-signed certificate which is not trusted by definition.
Here is how to start WireMock proxy with SSL support.
java -jar wiremock-standalone-2.2.2.jar \ --port 9999 --https-port 8443 \ --proxy-all="https://jsonplaceholder.typicode.com" \ --record-mappings --verbose
Suppose I make use of an InvokeHTTP processor to download details about a specific blog post in my hypothetical API. The communication is mediated by the proxy.
HTTP(S) GET details about a blog post
Since the API server’s certificate is not trusted, my request is bound to fail.
The API call fails, since the server’s certificate is not trusted.
To solve the problem, I have to make the client trust the server’s root certificate, which I first need to download. Since WireMock relies on a self-signed certificate, there is no hierarchy. In practice, I would most likely see a hierarchy of certificates, where the one for me to download would sit on top (Root Certificate).
WireMock’s self-signed certificate.
To download the certificate in Chrome, I simply drag the icon to a folder anywhere on my local filesystem.
Next, I create a new trust store and import the downloaded certificate into it (using an existing trust store would equally be fine). If you are new to this topic, a trust store is simply a repository of certificates trusted by a client, i.e. my NiFi job talking to the API server.
keytool -import -v -trustcacerts \ -file wiremock.cer -alias wiremockca \ -keystore cacerts.jks
Just for clarification, wiremock.cer is the certificate to be added as trusted. An alias can be any string, as long as it is unique in the trust store. I give my trust store a fairly standard name of cacerts.jks, where the jks suffix suggests a mere Java KeyStore format. Note the trustcacerts option which allows me to import the certificate as if it was issued by a trusted Certification Authority (CA).
Equipped with a trust store acknowledging the server certificate as trusted, I am ready to move on and link it to my NiFi job. That’s when an SSL context service comes into play.
SSL context service linked to a trust store.
Once I have added the SSL service to my HTTP client (see the screenshot below), the API server will be considered trustworthy and I will start to see the expected API responses.
HTTP client (InvokeHTTP) linked to a custom SSL service.
Problem #2: Client Authentication during SSL Handshake
I would argue this is somewhat less common, but as a matter of fact, an SSL-enabled server can be set to validate client’s identity. Since this is not an SSL tutorial, I won’t focus on the very details of how the handshake is established. Instead, let me oversimplify by saying that for the authentication to work, a client’s certificate has to be registered with the SSL server.
To simulate the problem, I restart WireMock using https-require-client-cert option. Note, I also provide references to a key- and trust store. The key store will eventually contain the certificate of my HTTP client.
java -jar wiremock-standalone-2.2.2.jar \ --port 9999 --https-port 8443 \ --proxy-all="https://jsonplaceholder.typicode.com" \ --record-mappings --https-require-client-cert \ --https-truststore cacerts.jks \ --truststore-password changeit \ --https-keystore keystore.jks \ --keystore-password changeit \ --verbose
After the restart HTTPS “breaks” again, as the client is unable to authenticate as part of the SSL handshake.
Client authentication is required.
To mitigate the issue, I create a key store containing client’s self-signed certificate. In reality, I would obtain the certificate from a renowned CA.
keytool -genkey -alias clientca \ -keyalg RSA \ -keypass changeit \ -storepass changeit \ -keystore keystore.jks
Next, I export the client’s certificate and put it in the trust store on the server side. Again, I only need to do this because of the certificate being self-signed.
keytool -genkey -alias clientca \ -keyalg RSA \ -keypass changeit \ -storepass changeit \ -keystore keystore.jks keytool -import -v -trustcacerts \ -file client.cer -alias clientca \ -keystore cacerts.jks
Additionally, I should also import the client’s certificate into the server’s key store (authentication). Since both my client and the server run on the same machine, I get away with using the same key store on either end. In reality though, they would be separate.
As the last step, I tweak my SSL service and add a reference to the (client’s) key store, so that the client can provide authentication during the handshake.
SSL service with key store containing client certificate.
From this point on, my HTTP client is able to provide the requested authentication details and, since its certificate is registered with the API server, the handshake is successful.
I will leave it with that,thanks for reading. Today, I have gone through an example of how to establish trust towards an SSL server and authenticate a client. Stay tuned for my next post about NiFi, where I will take a closer look at a pragmatic use of NiFi’s expression language.
