Blog
Cloud

Head first Serverless with AWS Lambda in Kotlin

-

In this post I’ll take you with me on my journey into Serverless with AWS. Head first, because I won’t dive into details like what is Serverless, why should you or should you not want Serverless, what those Amazon Web Services (AWS) are, nor into Kotlin. I will just work out a concrete case.

At our office we regularly hold knowledge sessions. Mostly around lunch time a colleague will tell and show something about a topic. In the morning of an upcoming knowledge session someone sends out an announcement via email to all our colleagues. Since most of us also are on Slack, we could announce it there. And this is something we could easily automate. In this post I am going to build a Slack bot that announces knowledge sessions in the morning. What do we need?

  • A schedule of planned sessions
  • Piece of code that sends a message to a Slack channel

For the sake of brevity I will focus on the absolute minimum of these requirements.

Amazon Web Services

The following AWS services will be used:

  • S3 to store the scheduled sessions.
  • Lambda to retrieve data from store and send a message to Slack when needed.
  • CloudWatch for logging/monitoring and triggering a scheduled event every morning.

 

IAM (Identity and Access Management)

One service that’s missing in the list above list is IAM. This service is an inherent part of AWS. Before starting with the other services first we’ll create a role for our App. This role should have permissions to:

  • AmazonS3FullAccess
  • AWSLambdaExecute
  • CloudWatchLogsFullAccess

This single role will be used for all services I’ll be creating next.

S3 (Simple Storage Service)

First step is to create a S3 bucket. S3 is an object storage that can be used to store and retrieve files. Open S3 from the AWS console and create a bucket with the name ‘upcoming-sessions’ and use the default settings (click next, next, next). In this bucket upload a file ‘next-session.json’ with the following contents:

 

{
"date": "2019-01-20",
"presenter": "Rachid",
"topic": "Creating a Slack announcer using AWS",
"type": "Chalk & Talk"
}

Lambda

Next step is to create a lambda which will read this file and send a message to Slack when needed. In the AWS management console go to: Lambda -> functions -> Create function. As name choose: “session-announcer”.  Since I want to write the lambda in Kotlin for runtime I choose Java 8. For Role select “choose an existing role” and select the Role we just created. Please note that everything can be done via scripts using the AWS CLI as well. Creating a new lambda function would look like:

 

aws lambda create-function \
--function-name session-announcer \
--runtime java8 \
--role arn:aws:iam::123456789:role/sessionAnnouncerRole \
--handler eu.luminis.chefke.event.ScheduledEventAnnouncer::handleRequest \
--zip-file announcer.jar

Since we don’t have any code or jar file yet I won’t use this now.

 

Coding in Kotlin

Now it’s time for the fun part; coding the logic to retrieve the sessions from S3 and send a message to Slack when needed. The handleRequest function is the entrypoint of our code. This function should be invoked to run our code.

 

import com.amazonaws.services.lambda.runtime.Context
import com.amazonaws.services.lambda.runtime.RequestHandler
import com.amazonaws.services.lambda.runtime.events.APIGatewayProxyRequestEvent
import com.amazonaws.services.lambda.runtime.events.APIGatewayProxyResponseEvent
import com.amazonaws.services.s3.AmazonS3ClientBuilder
import com.beust.klaxon.Klaxon
import java.io.InputStream
import java.time.LocalDate
import java.time.format.DateTimeFormatter
 
data class SlackResponseMessage(val text: String)
 
data class LuminisEvent(val topic: String, val type: String, val presenter: String, val date: String)
 
const val filename = "next-session.json"
 
class SessionAnnouncer : RequestHandler<APIGatewayProxyRequestEvent, APIGatewayProxyResponseEvent> {
 
    val s3client = AmazonS3ClientBuilder.defaultClient()
    val restClient = RestClient()
 
    override fun handleRequest(event: APIGatewayProxyRequestEvent, context: Context): APIGatewayProxyResponseEvent {
        val today = LocalDate.now()
        getEvent(today)?.let { e ->
            val message = "Today we have a ${e.type} session about '${e.topic}' presented by ${e.presenter}."
            println("Sending the event to Slack: $message")
            val slackJsonMessage = Klaxon().toJsonString(SlackResponseMessage(message))
            // POST the announcement as a JSON payload to the Slack webhook URL.
            restClient.post(Config.slackWebhook, slackJsonMessage)
            return APIGatewayProxyResponseEvent().apply {
                statusCode = 200
                headers = mapOf("Content-type" to "text/plain")
                body = "Message sent to Slack: $message"
            }
        }
 
        val notFoundMessage = "No event found for ${today} to post to Slack"
        println(notFoundMessage)
        return APIGatewayProxyResponseEvent().apply {
            statusCode = 404
            headers = mapOf("Content-type" to "text/plain")
            body = notFoundMessage
        }
    }
 
    /**
     * Retrieves JSON file from S3, parse it and return the Object when it's today.
     */
    private fun getEvent(day: LocalDate): LuminisEvent? {
        getEventFromBucket()?.let {
            val event = Klaxon().parse(it)
            event?.let { nextEvent ->
                val eventDay = LocalDate.parse(nextEvent.date, DateTimeFormatter.ISO_DATE)
                if(eventDay.equals(day)) {
                    return nextEvent
                }
            }
        }
        return null
    }
 
    fun getEventFromBucket(): InputStream? {
        val s3Bucket = Config.s3Bucket
        if(s3client.doesObjectExist(s3Bucket, filename)) {
            return s3client.getObject(s3Bucket, filename).objectContent
        }
        println("'$filename' does not exists in the bucket: $s3Bucket")
        return null
    }
}

There are several ways to write this code and publish it to AWS. For any serious coding I personally prefer to use my favorite IDE on my local machine. To keep build/test/package related logic in a central place I use Gradle. Luckily there’s a Gradle plugin for uploading the code to an AWS lambda. My build.gradle file is as follows:

 

import com.amazonaws.services.lambda.model.InvocationType
import jp.classmethod.aws.gradle.lambda.AWSLambdaInvokeTask
import jp.classmethod.aws.gradle.lambda.AWSLambdaMigrateFunctionTask
 
buildscript {
    ext.kotlin_version = '1.2.31'
 
    repositories {
        mavenCentral()
        maven {
            url "https://plugins.gradle.org/m2/"
        }
    }
    dependencies {
        classpath 'com.github.jengelman.gradle.plugins:shadow:2.0.2'
        classpath "org.jetbrains.kotlin:kotlin-gradle-plugin:$kotlin_version"
        classpath "jp.classmethod.aws:gradle-aws-plugin:0.30"
    }
}
 
version '1.0-SNAPSHOT'
 
apply plugin: 'com.github.johnrengelman.shadow' // To create a fatjar which can be uploaded to AWS
apply plugin: 'jp.classmethod.aws'              // Gradle tasks for AWS stuff
apply plugin: 'jp.classmethod.aws.lambda'       // Gradle tasks for deploying and running lambda's
apply plugin: 'kotlin'
 
repositories {
    jcenter()
    mavenCentral()
}
 
dependencies {
    implementation "org.jetbrains.kotlin:kotlin-stdlib-jdk8:$kotlin_version"
    // AWS API
    implementation 'com.amazonaws:aws-lambda-java-core:1.2.0'
    implementation 'com.amazonaws:aws-lambda-java-events:2.1.0'
    implementation 'com.amazonaws:aws-java-sdk-s3:1.11.308'
    // JSON parser for Kotlin
    implementation 'com.beust:klaxon:3.0.1'
}
 
compileKotlin {
    kotlinOptions.jvmTarget = "1.8"
}
 
lambda {
    region = "eu-west-1"
}
 
// Task to deploy the code to AWS
task deployFunction(type: AWSLambdaMigrateFunctionTask, dependsOn: [shadowJar, test]) {
    functionName = "session-announcer"
    runtime = com.amazonaws.services.lambda.model.Runtime.Java8
    role = "arn:aws:iam::${aws.accountId}:role/scheduleCntRole"
    zipFile = shadowJar.archivePath
    handler = "eu.luminis.blog.SessionAnnouncer::handleRequest"
    memorySize = 512
    timeout = 20
}
 
// Task to directly invoke the lambda in AWS
task invokeFunction(type: AWSLambdaInvokeTask) {
    functionName = "session-announcer"
    invocationType = InvocationType.RequestResponse
    payload = ""
    doLast {
        println "Lambda function result: " + new String(invokeResult.payload.array())
    }
}

On top the plugins are applied and the dependencies are declared, at the bottom there are two custom tasks; deployFunction and invokeFunction. The first should be executed to upload the code to AWS, and the latter can be used for directly invoking the code running at AWS. Note that in deployFunction we’ve specified the handler function of our lambda. Now let’s upload the code by executing the task deployFunction. In IntelliJ this can be done by expanding the tasks in the Gradle view and double click deployFunction. Or just open a Terminal in the root of the project and run ./gradlew session-announcer:deployFunction. NB: When using this Gradle plugin or AWS CLI for the first time you may need to authenticate first. When the function is deployed successfully we can try to invoke it with: ./gradlew session-announcer:invokeFunction Oh noes, I got an error: {"errorMessage":"Missing env var 'S3_BUCKET'!","errorType":"java.lang.IllegalStateException"} Of course, the code uses Config.s3Bucket which reads the bucket name from an environment variable because we don’t want to have any configuration in code.

 

object Config {
    val s3Bucket by lazy { getRequiredEnv("S3_BUCKET") }
    val slackWebhook by lazy { getRequiredEnv("SLACK_WEBHOOK_URL") }
 
    private fun getRequiredEnv(name: String): String {
        println("Retrieving environment variable: $name")
        return System.getenv(name) ?: throw IllegalStateException("Missing env var '$name'!")
    }
}

We should add two required environment variables. Open the AWS console and go to lambda -> session-announcer and scroll down to the section “Environment variables”. Here we can add the key/value pairs. So add the key S3_BUCKET and for value the name of the bucket we created, e.g. ‘upcoming-sessions’. Before we can add the second environment variable, first a bit more about Slack.

Slack integration

In order to send messages to a Slack Channel you need to create a so called ‘Slack App‘ and install it in the Slack workspace. After you’ve added the app to a channel, a webhook URL is generated and can be retrieved from the Slack console via: https://api.slack.com.  This URL is the second environment variable we need to configure for the AWS lambda. So go back to the AWS console and add to the section “Environment variables” a new key: SLACK_WEBHOOK_URL with the slack webhook URL as value. Now try to invoke the lambda again with: ./gradlew session-announcer:invokeFunction Hopefully you will either see something like: {"statusCode":200,"headers":{"Content-type":"text/plain"},"body":"Message sent to Slack: Today we have a Chalk & Talk session about 'Creating a Slack announcer using AWS' presented by Rachid."} Or: {"statusCode":404,"headers":{"Content-type":"text/plain"},"body":"No event found for 2019-01-20 to post to Slack"} To get more insights, the logs can be viewed in CloudWatch. In the AWS console go to CloudWatch -> Logs. There should be a log with the name /aws/lambda/session-announcer containing all the logs of our lambda.

Scheduling

Now we have a piece of code running as a lambda in AWS that can announce the sessions of the current day. Next step is to trigger this lambda every morning. This can be done with ‘Events’ in CloudWatch. In the AWS console go to CloudWatch -> Events -> Rules and click “create rule”. Then select Schedule (instead of Event pattern). To let it run every morning at 7:30 AM you can enter the following CRON expression: 30 7 * * ? *. At the right-hand side click ‘Add target’ and for function select the name of our lambda: session-announcer. Click configure details, choose a name and Save the rule. The next morning at 7:30 AM local time of the AWS region the lambda will be triggered. For testing purposes you could trigger it every minute with this CRON expression: */1 * * * ? *. Quite quickly we have automated a simple task. When you want to play around with this yourself, all the code from this post can be found in this Git repository.

Becoming an AWS developer

In the beginning playing around with AWS can be quite overwhelming. When you want to get some serious knowledge I recommend the Udemy course AWS Certified Solutions Architect – Associate  (most of the time priced around $10,-). A lot of information is scattered on the internet via blog posts and in the Amazon documentation. But what I like about this course is that it is kept up-to-date and gives you a single cohorent story which is easy to follow.

Wind up

In this post I focused on the bare minimum, but actually I also built:

  • REST interface to add sessions to the store; using API gateway and another lambda
  • REST interface to get session from store; called by a custom Slack command and backed by another lambda

Now we have a small working app, some improvements and new features I have in mind:

  • Store the session in a data store like Dynamo DB
  • Make it possible to schedule multiple sessions
  • Automatically provisioning of the services with CloudFormation