The Desktop is Dead! Long live the Desktop!

So it seems like desktop applications are dead, or the were… I don’t know. I’d rather a full native app instead of a web-based app. EVERYWHERE! And that includes desktop apps. Electron.js based apps are fast to develop, easy to maintain, and cross platform, but they s*ck$. They are hungry for resources, CPU and memory, as they are based on the always-hungry (Chrome) browser.

So, a few solutions come to scene in the last few years. The most famous I believe it’s Flutter, it covers mobile and desktop, but also web. But it’s not the only one! JetBrains released Compose for Desktop, which targets macOS, Linux and Windows. It’s a port of Android’s Compose for Desktop, what makes this a great alternative for multiplatform apps. If we add Kotlin Multiplatform to this equation for creating Business Logic that can also run on iOS, Kotlin seems like a great starting point for fully native desktop and mobile apps.

Web-based multiplatfom solutions are hungry for resources. Native apps are more resource-efficient, load faster, work better in offline mode.

Similar to the previous article, I will explore the development process of a Linux app using Kotlin Compose, and also will run that app on macOS. Take into consideration that Kotlin Compose is yet an experimental feature.

System Requirements

The only requirement for developing apps using Compose for Desktop, is Kotlin 1.4.20 or newer. A Java Runtime Environment is of course required for this JVM language, and even Kotlin can run on JRE 7 (for example) it’s recommended to use an as new as possible Java Virtual Machine, for example JVM 11 or 13.

At the time of this post there is a bug in JDK 14 that doesn’t allow to generate a Debian package in Linux. If you are using Linux, please switch to either JDK 11 (LTS) or JDK 15.

IntelliJ IDE, made by JetBrains, supports creating Kotlin Compose apps out-of-the-box. Two flavours are available:

  • Desktop
  • Multiplatform

Desktop targets macOS, Linux and Windows using Compose, while Multiplatform adds Android support.

In this tutorial, a desktop-only app will be created.

Getting Started

When the IDE finished starting up the project, all dependencies are downloaded and the index is updates, a single Main.kt file will be present in your source folder. The entry point of the app is a main function that returns a Window.

A simple Compose app may look like

@ExperimentalAnimationApi
fun main() = Window(title = "Hello, Compose!") {
    MaterialTheme {
        Column(
            horizontalAlignment = Alignment.CenterHorizontally,
            verticalArrangement = Arrangement.Center,
            modifier = Modifier.padding(32.dp).fillMaxWidth().fillMaxHeight()
        ) {
            Card {
                var expanded by remember { mutableStateOf(false) }
                Column(
                    horizontalAlignment = Alignment.CenterHorizontally,
                    verticalArrangement = Arrangement.Center,
                    modifier = Modifier.clickable { expanded = !expanded }) {
                    Image(
                        bitmap = imageResource("drawable/kotlin.png"),
                        alignment = Alignment.Center,
                        modifier = Modifier.padding(32.dp).preferredWidth(128.dp).preferredHeight(128.dp)
                    )
                    AnimatedVisibility(expanded) {
                        Column {
                            Text(
                                text = "Hello, Compose!",
                                style = MaterialTheme.typography.h2
                            )
                        }
                    }
                }
            }
        }
    }
}
Native Kotlin Compose app running on macOS

Distribution

Packaging an application can be a little tricky, you can run the app everywhere using a gradle o maven task, but you can only create a package, a distributable installation file, in the same architecture of the target package. In other words, there is no “cross packaging”. You cannot create a Debian compatible -deb- file from macOS. So, in order to cover all the available flavours, you will need access to Windows, macOS and Debian.

Which, if you have a macOS machine, it’s pretty easy to archive using VirtualBox or any other Virtual Machine.

The package task generates a distributable file for the current platform. There are other related tasks, but I couldn’t successfully run any of the on a platform different that the current on.

$ ./gradlew package

> Task :packageDmg
WARNING: Using incubator modules: jdk.incubator.jpackage
The distribution is written to build/compose/binaries/main/dmg/DesktopApp-1.0.0.dmg

BUILD SUCCESSFUL in 11s
5 actionable tasks: 3 executed, 2 up-to-date
  • msi (Microsoft Installer) for Windows
  • deb (Debian package) for Debian/Ubuntu compatible
  • dmg (Disk Image File) for macOS
Inside generated DMG file

Artefacts

The very simple app that you see in this post needs 80MB of disk space, and generates a DMG file of nearly 42MB. Where only 1MB belongs to this application (assets and compiled code). The rest is just Kotlin libraries (coroutines, stdlib, material design, etc).

/Applications/DesktopApp.app/Contents/app
kimi@Kimis-Mac-mini /A/D/C/app> ls -la
total 37024
drwxr-xr-x  38 kimi  admin     1216 Mar 23 14:34 ./
drwxr-xr-x   8 kimi  admin      256 Mar 23 14:34 ../
-rw-r--r--   1 kimi  admin   104427 Mar 23 14:34 DesktopApp-1.0.0.jar
-rw-r--r--   1 kimi  admin     1522 Mar 23 14:34 DesktopApp.cfg
drwxr-xr-x   3 kimi  admin       96 Mar 23 14:34 META-INF/
-rw-r--r--   1 kimi  admin     1522 Mar 23 14:34 MainKt$main$1$1$1$1$1$1.class
-rw-r--r--   1 kimi  admin     9533 Mar 23 14:34 MainKt$main$1$1$1$1$2$1.class
-rw-r--r--   1 kimi  admin    15698 Mar 23 14:34 MainKt$main$1$1$1$1.class
-rw-r--r--   1 kimi  admin     9692 Mar 23 14:34 MainKt$main$1$1.class
-rw-r--r--   1 kimi  admin     2157 Mar 23 14:34 MainKt$main$1.class
-rw-r--r--   1 kimi  admin     1292 Mar 23 14:34 MainKt.class
-rw-r--r--   1 kimi  admin   208635 Mar 23 14:34 animation-core-desktop-0.2.0-build132.jar
-rw-r--r--   1 kimi  admin   201364 Mar 23 14:34 animation-desktop-0.2.0-build132.jar
-rw-r--r--   1 kimi  admin    17536 Mar 23 14:34 annotations-13.0.jar
-rw-r--r--   1 kimi  admin    15414 Mar 23 14:34 desktop-jvm-0.2.0-build132.jar
drwxr-xr-x   4 kimi  admin      128 Mar 23 14:34 drawable/
-rw-r--r--   1 kimi  admin   822462 Mar 23 14:34 foundation-desktop-0.2.0-build132.jar
-rw-r--r--   1 kimi  admin   357370 Mar 23 14:34 foundation-layout-desktop-0.2.0-build132.jar
-rw-r--r--   1 kimi  admin  1488745 Mar 23 14:34 kotlin-stdlib-1.4.20.jar
-rw-r--r--   1 kimi  admin   191485 Mar 23 14:34 kotlin-stdlib-common-1.4.20.jar
-rw-r--r--   1 kimi  admin    22355 Mar 23 14:34 kotlin-stdlib-jdk7-1.4.20.jar
-rw-r--r--   1 kimi  admin    16233 Mar 23 14:34 kotlin-stdlib-jdk8-1.4.20.jar
-rw-r--r--   1 kimi  admin   196243 Mar 23 14:34 kotlinx-collections-immutable-jvm-0.3.jar
-rw-r--r--   1 kimi  admin  1672388 Mar 23 14:34 kotlinx-coroutines-core-jvm-1.4.1.jar
-rw-r--r--   1 kimi  admin    10884 Mar 23 14:34 kotlinx-coroutines-swing-1.4.1.jar
-rw-r--r--   1 kimi  admin  1261868 Mar 23 14:34 material-desktop-0.2.0-build132.jar
-rw-r--r--   1 kimi  admin   674505 Mar 23 14:34 material-icons-core-desktop-0.2.0-build132.jar
-rw-r--r--   1 kimi  admin   584609 Mar 23 14:34 runtime-desktop-0.2.0-build132.jar
-rw-r--r--   1 kimi  admin    19435 Mar 23 14:34 runtime-dispatch-desktop-0.2.0-build132.jar
-rw-r--r--   1 kimi  admin    56960 Mar 23 14:34 runtime-saved-instance-state-desktop-0.2.0-build132.jar
-rw-r--r--   1 kimi  admin   248772 Mar 23 14:34 skiko-jvm-0.1.18.jar
-rw-r--r--   1 kimi  admin  8553451 Mar 23 14:34 skiko-jvm-runtime-macos-x64-0.1.18.jar
-rw-r--r--   1 kimi  admin  1378337 Mar 23 14:34 ui-desktop-0.2.0-build132.jar
-rw-r--r--   1 kimi  admin    38650 Mar 23 14:34 ui-geometry-desktop-0.2.0-build132.jar
-rw-r--r--   1 kimi  admin   304483 Mar 23 14:34 ui-graphics-desktop-0.2.0-build132.jar
-rw-r--r--   1 kimi  admin   311880 Mar 23 14:34 ui-text-desktop-0.2.0-build132.jar
-rw-r--r--   1 kimi  admin    81156 Mar 23 14:34 ui-unit-desktop-0.2.0-build132.jar
-rw-r--r--   1 kimi  admin    11744 Mar 23 14:34 ui-util-desktop-0.2.0-build132.jar

It may seems to much, but, remember that this application embeds a Java Runtime and a lot of libraries that make your live easier developing in Kotlin.

In comparison to Electron.js + React.js I believe that the artefact size is similar, but there is a huge improvement in running a Kotlin / JVM app.

Getting Started with Flutter for Linux

Ubuntu

This post will not cover how to install Flutter on Linux, that can be easily done reading the following official guide. Instead, it will focus on building a Flutter app that runs on (Ubuntu) Linux.

Adding your Linux machine as a device

By default Flutter expects that you connect an Android or iOS device, or even a Chrome web browser to run the app. But in this case we would like to have a full-native Linux experience.

Flutter may report that you don’t have any connected device

$ flutter devices
No devices detected.

Run "flutter emulators" to list and start any available device emulators.

So in order to create and run a Flutter Linux app, not only the application must we created with Linux as a platform, but also we have to tell Flutter to enable Linux Desktop support.

$ flutter config --enable-linux-desktop
Setting "enable-linux-desktop" value to "true".

You may need to restart any open editors for them to read new settings.

If you are using a version of Flutter prior to 2.x, and upgrade is necessary at this point (flutter upgrade). Running Flutter doctor can tell us if there is something missing. A common situation in a fresh installed device may look like

$ flutter doctor
Doctor summary (to see all details, run flutter doctor -v):
[✓] Flutter (Channel beta, 2.0.0, on Linux, locale en_US.UTF-8)
[✓] Android toolchain - develop for Android devices (Android SDK version 30.0.3)
[✓] Chrome - develop for the web
[✗] Linux toolchain - develop for Linux desktop
    ✗ clang++ is required for Linux development.
      It is likely available from your distribution (e.g.: apt install clang), or can be downloaded from https://releases.llvm.org/
    ✗ CMake is required for Linux development.
      It is likely available from your distribution (e.g.: apt install cmake), or can be downloaded from https://cmake.org/download/
    ✗ ninja is required for Linux development.
      It is likely available from your distribution (e.g.: apt install ninja-build), or can be downloaded from https://github.com/ninja-build/ninja/releases
    ✗ GTK 3.0 development libraries are required for Linux development.
      They are likely available from your distribution (e.g.: apt install libgtk-3-dev)
    ✗ The blkid development library is required for Linux development.
      It is likely available from your distribution (e.g.: apt install libblkid-dev)
    ✗ The lzma development library is required for Linux development.
      It is likely available from your distribution (e.g.: apt install liblzma-dev)
[✓] Android Studio
[✓] IntelliJ IDEA Ultimate Edition (version 2020.3)
[✓] Connected device (2 available)

! Doctor found issues in 1 category.

Install all the necessary dependencies until Flutter doctor returns an output with all items checked, like this

$ flutter doctor
Doctor summary (to see all details, run flutter doctor -v):
[✓] Flutter (Channel beta, 2.0.0, on Linux, locale en_US.UTF-8)
[✓] Android toolchain - develop for Android devices (Android SDK version 30.0.3)
[✓] Chrome - develop for the web
[✓] Linux toolchain - develop for Linux desktop
[✓] Android Studio
[✓] IntelliJ IDEA Ultimate Edition (version 2020.3)
[✓] Connected device (2 available)

• No issues found!

Now, with Flutter telling everything is okey, and the linux desktop support enabled, we can query which devices are enabled

$ flutter devices
2 connected devices:

Linux (desktop) • linux  • linux-x64      • Linux
Chrome (web)    • chrome • web-javascript • Google Chrome 88.0.4324.182

Creating the app

By default Flutter creates an app with several targets as default: iOS, Android, Windows, Linux, macOS and Web. In this brief tutorial, we will only create an app with Linux as the unique target.

$ flutter create --platforms=linux --template=app hello_linux

Flutter create can receive a list of platforms when creating an app, in this example only Linux will be created.

Using the template argument, we can create applications, modules, a Dart package, or a plugin (iOS and/or Android).

Running the application can be doing using flutter run command from the application’s directory. We can specify the target device, using the device-id argument as follows

$ flutter run -d linux

The output of the command above is especially important in order to archive an agile development. It contains not only the URL of the debugger, but also some shortcuts that let’s you use Hot Reload and Hot Restart on this native app.

$ flutter run -d linux
Launching lib/main.dart on Linux in debug mode...
Building Linux application...                                           

** (hello_linux:23299): WARNING **: 17:23:44.883: Failed to set up Flutter locales
Syncing files to device Linux...                                    62ms

Flutter run key commands.
r Hot reload.
R Hot restart.
h Repeat this help message.
d Detach (terminate "flutter run" but leave application running).
c Clear the screen
q Quit (terminate the application on the device).
An Observatory debugger and profiler on Linux is available at: http://127.0.0.1:45697/9m8BapQhQeM=/

Flutter DevTools, a Flutter debugger and profiler, on Linux is available at: http://127.0.0.1:9102?uri=http%3A%2F%2F127.0.0.1%3A45697%2F9m8BapQhQeM%3D%2F

Development Environment

A recommend configuration for a Linux App Development is using Visual Studio Code with the Flutter plugin. On Ubuntu that can be installed using Snap.

$ snap install --classic code
$ code --install-extension dart-code.flutter

Bonus Track

Null Safety

When nullable safety is enabled types cannot be null be default. We have to explicitly say that they can be null.

The first benefit is that it prevents Null Pointer Exceptions (NPE) by throwing errors at compile-time instead of runtime. This is a huge improvement not only in stability but also in code quality.

For learning more about how Dart implements null safety, take a look at this post.

Null safety means that variables can’t contain null unless you say they can

Enabling null safety is done using dart migrate.

$ dart migrate --apply-changes

Scanning your Java/Kotlin project with SonarQube

In the past year as a Technical Leader at Santander I have seen several a lot of code challenges submitted by applicants. All of them were small apps with one explicit requirement. Submit an app that makes you are comfortable to put in production.

Most of these apps where written in Java, others in Node.js and a few in Kotlin. And most (if not all) of them, shared a single missing point. Static Code Analysis.

Submit an app that makes you comfortable to put in production.

Static Code Analysis

So what is Static Code Analysis and why it’s important for production-ready software development.

Static Code Analysis, a.k.a. Source Code Analysis, is a type of analysis that addresses weaknesses that may lead to vulnerabilities. It automates a task that can be done in a deep code review, but also allows a much richer analysis. For example, duplicate code detection, coverage and quality of unit tests, bad practices, known vulnerabilities, etc.

There is such thing as Dynamic Code Analysis, in contrast to Static, and the main different is when this analysis is done. Is Dynamic the analysis is done after the program runs (during testing, for example) and in Static it is done before (unit testing and source code).

The best thing about Static Code Analysis is that it’s super easy to integrate in your development processes, either in a CI (Continuous Delivery) or a Git Hook (pre-push, for example).

Straight to the point

We will see in this post how to add a Maven task for a Java project, and do some Static Code Analysis using SonarQube Community Edition. No fancy hardware is needed, no cloud computing or VPS, just your machine which should be able to run SonarQube locally using Docker.

Adding SonarQube to a Maven project

First of all we will add the Jacoco and Sonar plugins to the project’s pom.xml.

<plugin>
    <groupId>org.sonarsource.scanner.maven</groupId>
    <artifactId>sonar-maven-plugin</artifactId>
    <version>3.8.0.2131</version>
</plugin>
<plugin>
    <groupId>org.jacoco</groupId>
    <artifactId>jacoco-maven-plugin</artifactId>
    <version>0.8.6</version>
</plugin>

Also, we will add Maven Compiler Plugin (if it’s not already present)

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-compiler-plugin</artifactId>
    <version>3.8.1</version>
</plugin>

At the project’s level we will add a profile.

<profiles>
    <profile>
        <id>coverage</id>
        <activation>
            <activeByDefault>true</activeByDefault>
        </activation>
        <build>
            <plugins>
                <plugin>
                    <groupId>org.jacoco</groupId>
                    <artifactId>jacoco-maven-plugin</artifactId>
                    <executions>
                        <execution>
                            <id>prepare-agent</id>
                            <goals>
                                <goal>prepare-agent</goal>
                            </goals>
                        </execution>
                        <execution>
                            <id>report</id>
                            <goals>
                                <goal>report</goal>
                            </goals>
                        </execution>
                    </executions>
                </plugin>
            </plugins>
        </build>
    </profile>
</profiles>

Starting SonarQube

SonarQube will perform static code analysis based on some output files that Maven will produce when running a specific task, but prior to that, we need a working SonarQube Server.

Fortunately, that can be done easily using Docker. In case you don’t have Docker installed in your machine, follow this guide if you are under Linux, or this one if you are on Windows or macOS.

Then pull the latest SonarQube image, and bind the internal port 9000 to your local machine port 9000.

docker run -d --name sonarqube -e SONAR_ES_BOOTSTRAP_CHECKS_DISABLE=true -p 9000:9000 sonarqube:latest

Once it completes successfully, open your browser and login into http://localhost:9090 with the default credentials:

  • login: admin
  • password: admin

Create a new application and register for a Project ID and a Project Token. For more information about this, check this quick guide.

Putting all together

Once your application is created in SonarQube dashboard, and you have a valid token, run the following maven task.

mvn clean verify sonar:sonar 
  -Dsonar.projectKey=<ProjectID> \
  -Dsonar.host.url=http://localhost:9000 \
  -Dsonar.login=<ProjectToken>

Where <ProjectID> and <ProjectToken> where just created. When the target finished executing, you will see the link to your project’s report on the output.

ANALYSIS SUCCESSFUL, you can browse http://localhost:9000/dashboard?id=com.project

You will see in your project’s dashboard information about your code quality similar to the one in the following screenshots.

Headless Raspberry Pi configuration

Settings up a Raspberry Pi it’s easy when you have an HDMI display, a USB keyboard and mouse in oder to access the terminal or you favourite Desktop, but can be quite a challenge when you don’t have any of these or at least no keyboard.

Everything in this blog post should be done from you host computer, with you microSD card connected and mounted, assuming the microSD is mounted at /Volumes/boot.

cd /Volumes/boot
Raspberry Pi 4 with Hyperpixel 4.0 display

Enable SSH over LAN / WiFi

The SSH server is not enable by default as a user it’s expected that you access your Raspberry using a KVM. Not a Linux Kernel but a Keyboard, (VGA) Display and Mouse. This can be easily done by creating an empty file named ssh.

touch ssh

Preconfigured a WiFi network

If you are going to connect to Raspberry to your network using an Ethernet cable, you can skip this part. But if a WiFi network (even your mobile phone as a hotspot) it’s you only option, you can set your default WiFi network config in a file called wpa_supplicant.conf

sudo vi wpa_supplicant.conf
network={
    ssid="<< your WiFi's SSID >>"
    psk="<< your WiFi's password >>"
}

Take into consideration that this configuration uses your password in plain-text! If you want to encrypt your password in this configuration file do the following:

wpa_passphrase "<< your WiFi's SSID >>" "<< your WiFi's password >>"

On a Linux distribution wpa_passphrase is installed, but on macOS or Windows not. the following Ruby script will use OpenSSL in order to create the content of the configuration file.

vi /tmp/raspi_wifi.rb
require 'openssl'
ssid = '<< your WiFi SSID >>'
psk  = 'ID >>" "<< your WiFi password >>'
puts 'country=AR'
puts 'ctrl_interface=DIR=/var/run/wpa_supplicant GROUP=netdev'
puts 'update_config=1'
puts ''
puts 'network={'
puts '  ssid="' + ssid + '"'
puts '  psk=' + OpenSSL::PKCS5.pbkdf2_hmac_sha1(psk, ssid, 4096, 32).unpack("H*").first
puts '  key_mgmt=WPA-PSK'
puts '}'

Then run the script and copy the output to /Volumes/boot/etc/wpa_supplicant/wpa_supplicant.conf

ruby /tmp/raspi_wifi.rb > /Volumes/boot/wpa_supplicant.conf

Accessing your Pi using SSH

Now when your Raspberry completes its boot process, you will be able to access your device using SSH even without knowing its IP address.

Raspberry exposes a host in your local domain, so the following command with let you know it IP address.

ping raspberrypi.local

The default Raspberry user and password are pi and raspberry, so the following command will establish a SSH connection

ssh pi@raspberry.local 

Fibonacci Sequence (or when NOT to use recursion)

Fibonacci


For those of us who study computer science and algorithms, the study of recursion has an iconic example: The Fibonacci sequence.

I truly believe that this is clearly the worst scenario for using recursion. Some of you may think that it’s a great example due to the simplicity of the code. It is short, easy to remember, easy to read, even beautiful.

For example, I found this Swift implementation quite short, and I don’t believe it is possible to write one in fewer lines.

func fib(n: Int) -> Int {
    guard n > 1 else { return n }
    return fib(n-1) + fib(n-2)
}

It is clear the use of recursion, the function only has two lines and F(0) and F(0) are solved in O(1). While the rest of the numbers is resolved in O(2^n). (For more information about Big-O notation take a look at this Wikipedia post).

Now, let’s do some maths. We will see that it takes to calculate F(n) when N is less than 6

F(0) = 0
F(1) = 1
F(2) = F(1) + F(0) = 0 + 1 = 1
F(3) = F(2) + F(1) = (F(1) + F(0)) + F(1) = 2
F(4) = F(3) + F(2) = (F(2) + F(1)) + (F(1) + F(0)) = 3
F(5) = F(4) + F(3) = (F(3) + F(2)) + (F(2) + F(1)) = 5
F(6) = F(5) + F(4) = (F(4) + F(3)) + (F(3) + F(2)) = 8
A visual representation of how to calculate each number in the sequence

The first approach to optimise this function is to realise that some numbers are calculated several times. So… we can implement a cache! We can have a cache of F(n) and if it’s in the cache avoid the recursion. We are still on an approach that is not optimal neither in time nor in complexity.

This new approach can introduce a new problem when N is big. The algorithm now runs faster, but it takes too much memory!


But if we look a little bit further, every number in the sequence needs the two previous ones, and no more than that. So, what about having a small cache of the last two calculated number.

So we can have a non recursive approach of the Fibonacci sequence, solved in O(n) by saving the last two calculated numbers, and “switching them” as I show in the following Swift code.

func fib (n: Int) -> Int {
    if n < 2 { return n }
    var result = 0, fa = 0, fb = 1
    for  _ in 2...n {
        result = fa + fb
        fa = fb
        fb = result
    }
    return result
}
for i in 0...6 {
    debugPrint("f(\(i)) = \(fib(n: i))")
}

Testing it will produce the same result as doing the maths. But this time this algorithm is O(n) and it is very very optimal in terms of memory.

for i in 0...6 {
    debugPrint("f(\(i)) = \(fib(n: i))")
}
"f(0) = 0"
"f(1) = 1"
"f(2) = 1"
"f(3) = 2"
"f(4) = 3"
"f(5) = 5"
"f(6) = 8"

Listing all endpoints with Gorilla Mux Router

golang

When adding different handlers to a Router, each one with its own base path (sub routing), it comes handy to list all the endpoints declared, not only as a debug option but also when you want your API to expose all the methods it’s support.


Gorilla Mux

My favourite web toolkit in Go is the Gorilla Web Toolkit, which apart of the router and dispatcher Gorilla Mux, has also other packages for handling cookies in a secure way, using web sockets, and even RPC over HTTP.

If the following example you will see a common pattern I use when building services using Gorilla Mux. I start by creating a service layer (in this case then bots are my services), and the an API layer that defines endpoints and calls the service (controller). So imagine have two separate controllers and services, or more, each one with its own path.

func main() {
	fmt.Println("Starting messaging bot")
	//Create our Gorilla mux main router which path is "/api"
	mainRouter := mux.NewRouter().
		PathPrefix("/api").
		Subrouter().
		StrictSlash(true)
	//Create WhatsApp bot
	whatsAppBot := bots.NewWhatsAppBot()
	//Create a sub router at "/api/whatsapp"
	api.NewAPI(whatsAppBot,
		mainRouter.PathPrefix("/whatsapp").
			Subrouter().
			StrictSlash(true))
	//Create Telegram bot and API server
	telegramBot := bots.NewTelegramBot()
	//Create a sub router at "/api/telegram"
	api.NewAPI(telegramBot,
		mainRouter.PathPrefix("/telegram").
			Subrouter().
			StrictSlash(true))
	fmt.Println("WhatsApp bot running on /api/whatsapp")
	fmt.Println("Telegram bot running on /api/telegram")
        //Start the HTTP server on port 8000
	http.ListenAndServe(":8000", mainRouter)
}

Every Router has a Walk function that let’s us navigate all the endpoints we define. If we want to debug which endpoint has been declared in our main router, we can call a function like the one below:

func printEndpoints(r *mux.Router) {
	r.Walk(func(route *mux.Route, router *mux.Router, ancestors []*mux.Route) error {
		path, err := route.GetPathTemplate()
		if err != nil {
			return nil
		}
		methods, err := route.GetMethods()
		if err != nil {
			return nil
		}
		fmt.Printf("%v %s\n", methods, path)
		return nil
	})
}

In my example this function will print

Starting messaging bot
[GET] /api/whatsapp/login
[POST] /api/whatsapp/message
[GET] /api/telegram/login
[POST] /api/telegram/message
Listening on port 8000

How to make a WhatsApp Bot in Go [Part 3 of N]

WhatsApp Logo

Processing incoming messages, either text or media, is one of the fundamental features a Bot must implement. In this post we will explore how to do this.


Connection Handlers

go-whatsapp has a connection struct which exposes functions to perform login, handle session, and send messages. The reception of those messages is done by add a (message) handler.

The Handler interface contains the only function that we must implement, the ErrorHandler function.

type Handler interface {
	HandleError(err error)
}

Every other handler, like the TextMessageHandler inherits from Handler and adds its own functions. For example:

type TextMessageHandler interface {
	Handler
	HandleTextMessage(message TextMessage)
}

The most important part of this design, is that we can add as many handlers as you want. The connection struct contains a (private) list of handlers, and it checks on runtime to which handler it must dispatch each message.

Moreover, due to the implementation of this list of handlers, we can add two or more handlers of the same type. For example, one TextMessageHandler can log any incoming message into a database, and another one can perform text analysis and send responses based on that.

This implementation relies heavily on the very famous Go’s duck type (or structural typing).

If it looks like a duck, swims like a duck, and quacks like a duck, then it probably is a duck.

Incoming messages

A TextMessage is represented by a struct with three public fields. Info, Text and ContextInfo.

type TextMessage struct {
	Info        MessageInfo
	Text        string
	ContextInfo ContextInfo
}

Text contains the message we received (or sent), Info (MessageInfo) gives us information about who and when the message was sent, and it’s status (pending, sent, delivered, read or played). And ContextInfo gives us additional information like if the message is a quote of another message (we also have access to this original message), or if it was forwarded.

Receiving messages is a simple as declaring a struct and implementing the TextMessageHandler interface. In this example I declared a struct called whatsAppMessageHandler which implements all of the functions requiered.

type whatsAppMessageHandler struct{}
func (whatsAppMessageHandler) HandleError(err error) {
	fmt.Fprintf(os.Stderr, "%+v", err)
}
func (whatsAppMessageHandler) HandleTextMessage(message whatsapp.TextMessage) {
	fmt.Printf("HandleTextMessage: %+v\n", message)
}

If we send a message, for example, “Hello WhatsApp Bot!” the following line will be printed in our terminal.

HandleTextMessage: {Info:{Id:F75C24CE7098AB206B22DF6B275533B8 RemoteJid:54911**12****@s.whatsapp.net SenderJid: FromMe:true Timestamp:1581616175 PushName: Status:4 Source:key:<remoteJid:"54911**12****@s.whatsapp.net" fromMe:true id:"F75C24CE7098AB206B22DF6B275533B8" > message:<conversation:"Hello, WhatsApp Bot!" > messageTimestamp:1581616175 status:READ } Text:Hello, WhatsApp Bot! ContextInfo:{QuotedMessageID: QuotedMessage:<nil> Participant: IsForwarded:false}}

The TextMessage struct in a more (JSON) pretty printed version looks like this

{
   "Info":{
      "Id":"F75C24CE7098AB206B22DF6B275533B8",
      "RemoteJid":"54911**12****@s.whatsapp.net",
      "SenderJid":"",
      "FromMe":true,
      "Timestamp":1581616175,
      "PushName":"",
      "Status":4,
      "Source":{
         "key":{
            "remoteJid":"54911**12****@s.whatsapp.net",
            "fromMe":true,
            "id":"F75C24CE7098AB206B22DF6B275533B8"
         },
         "message":{
            "conversation":"Hello, WhatsApp Bot!"
         },
         "messageTimestamp":1581616175,
         "status":4
      }
   },
   "Text":"Hello, WhatsApp Bot!",
   "ContextInfo":{
      "QuotedMessageID":"",
      "QuotedMessage":null,
      "Participant":"",
      "IsForwarded":false
   }
}

A reply quoting this message will look like

{
   "Info":{
      "Id":"31D97076ED9DFC5C95D82A8B0B9FE00E",
      "RemoteJid":"54911**12****@s.whatsapp.net",
      "SenderJid":"",
      "FromMe":true,
      "Timestamp":1581616344,
      "PushName":"",
      "Status":4,
      "Source":{
         "key":{
            "remoteJid":"54911**12****@s.whatsapp.net",
            "fromMe":true,
            "id":"31D97076ED9DFC5C95D82A8B0B9FE00E"
         },
         "message":{
            "extendedTextMessage":{
               "text":"Reply on a message",
               "previewType":0,
               "contextInfo":{
                  "stanzaId":"F75C24CE7098AB206B22DF6B275533B8",
                  "participant":"54911**12****@s.whatsapp.net",
                  "quotedMessage":{
                     "conversation":"Hello, WhatsApp Bot!"
                  }
               }
            }
         },
         "messageTimestamp":1581616344,
         "status":4
      }
   },
   "Text":"Reply on a message",
   "ContextInfo":{
      "QuotedMessageID":"F75C24CE7098AB206B22DF6B275533B8",
      "QuotedMessage":{
         "conversation":"Hello, WhatsApp Bot!"
      },
      "Participant":"54911**12****@s.whatsapp.net",
      "IsForwarded":false
   }
}

As this reply was done on a specific message, the same information appears on the top-level ContextInfo and inside Info struct.

How to make a WhatsApp Bot in Go [Part 2 of N]

WhatsApp Logo

In this second step, we are going to analyse how to use the API of go-whatsapp described in the previous blog post, and what considerations we should take into account in order to create a bot.


Sending a text message

The WhatsApp Connection instance that is returned when creating a Session has several messages to send text messages, images, create groups, receive messages and even query for the contact list.

For sending a text message you have to create an instance of TextMessage indicating at least the text message an a receipt.

text := whatsapp.TextMessage{
    Info: whatsapp.MessageInfo{
	RemoteJid: "<receiver's phone number>@s.whatsapp.net",
    },
    Text: "Text message sent from Golang bot",
}
sendResult, err := waconn.Send(text)

where whatsApp refers to the module go-whatsapp imported as

import (
    "github.com/Rhymen/go-whatsapp"
)

whatsApp.TextMessage is an struct which fields are

type TextMessage struct {
	Info        MessageInfo
	Text        string
	ContextInfo ContextInfo
}

and whatsApp.MessageInfo contains

type MessageInfo struct {
	Id        string
	RemoteJid string
	SenderJid string
	FromMe    bool
	Timestamp uint64
	PushName  string
	Status    MessageStatus
	Source *proto.WebMessageInfo
}

The first important thing to notice is that MesssageInfo only contains on RemoteJid, so it’s not possible to a single message to multiple receivers (broadcasting).

Note: Trying to concatenate several RemoteJid (separated with comma, for example) leads to a crash in the mobile app where WhatsApp runs! unbelievable.

The response to the Send command is a string and an error. Where error is nil if the operation succeeded, and string contains either the MessageId if everything is okey, or the literal string ERROR in case of failure.


Other types of messages can be sent using this module. The existent implementation of go-whatsapp, by the time this post was created, allows a user to send text, photos, videos, documents (files), audio, locations, live locations contact information (vCards), and even Stickers!.

Image, Audio and Video messages receive its content in a io.Reader property. Several fields of these structs remain unexpected as they are needed for media upload/download and validations.

The definition of each of these structures are

Image Message
type ImageMessage struct {
	Info          MessageInfo
	Caption       string
	Thumbnail     []byte
	Type          string
	Content       io.Reader
	url           string
	mediaKey      []byte
	fileEncSha256 []byte
	fileSha256    []byte
	fileLength    uint64
	ContextInfo   ContextInfo
}
Video Message
type VideoMessage struct {
	Info          MessageInfo
	Caption       string
	Thumbnail     []byte
	Length        uint32
	Type          string
	Content       io.Reader
	GifPlayback   bool
	url           string
	mediaKey      []byte
	fileEncSha256 []byte
	fileSha256    []byte
	fileLength    uint64
	ContextInfo   ContextInfo
}
Audio Message
type AudioMessage struct {
	Info          MessageInfo
	Length        uint32
	Type          string
	Content       io.Reader
	Ptt           bool
	url           string
	mediaKey      []byte
	fileEncSha256 []byte
	fileSha256    []byte
	fileLength    uint64
	ContextInfo   ContextInfo
}
Document Message
type DocumentMessage struct {
	Info          MessageInfo
	Title         string
	PageCount     uint32
	Type          string
	FileName      string
	Thumbnail     []byte
	Content       io.Reader
	url           string
	mediaKey      []byte
	fileEncSha256 []byte
	fileSha256    []byte
	fileLength    uint64
	ContextInfo   ContextInfo
}
Location Message
type LocationMessage struct {
	Info             MessageInfo
	DegreesLatitude  float64
	DegreesLongitude float64
	Name             string
	Address          string
	Url              string
	JpegThumbnail    []byte
	ContextInfo      ContextInfo
}
Live Location Message
type LiveLocationMessage struct {
	Info                              MessageInfo
	DegreesLatitude                   float64
	DegreesLongitude                  float64
	AccuracyInMeters                  uint32
	SpeedInMps                        float32
	DegreesClockwiseFromMagneticNorth uint32
	Caption                           string
	SequenceNumber                    int64
	JpegThumbnail                     []byte
	ContextInfo                       ContextInfo
}
Stickers
type StickerMessage struct {
	Info MessageInfo
	Type          string
	Content       io.Reader
	url           string
	mediaKey      []byte
	fileEncSha256 []byte
	fileSha256    []byte
	fileLength    uint64
	ContextInfo ContextInfo
}
Contact Message (vCard)
type ContactMessage struct {
	Info MessageInfo
	DisplayName string
	Vcard       string
	ContextInfo ContextInfo
}

How to make a WhatsApp Bot in Go [Part 1 of N]

WhatsApp Logo

Such a long time since I’ve published my last post, I’m back with a simple and small post: How to make a simple Whatsapp Bot in Go.


Note about this post and its examples

The most important thing is to take into account that WhatsApp does not provide a public API that you can use to create a Bot, or any kind of program that interacts with it. Every open source (and free) solution found on the Internet relies on some sort of scrapping or reverse engineering. So the solution presented in this post may not be stable (or need an update) by the time you read this post.


Requirements

  • Go (Golang) version 1.11 or newer (Go modules are used)
  • An active WhatsApp account logged in a phone

Dependencies

Getting Started

From a terminal in your favourite OS (one of the beautiful things of a Multiplatform language as Golang) create a module as follow:

$ mkdir whatsapp-bot-test
$ cd whatsapp-bot-test
$ go mod init github.com/eaceto/whatsapp-bot-test
go: creating new go.mod: module github.com/eaceto/whatsapp-bot

In order to import Lucas Engelke’s go-whatsapp run the following command inside your project directory

$ go get github.com/Rhymen/go-whatsapp

Now the content of go.mod will look like this

module github.com/eaceto/whatsapp-bot
go 1.13
require github.com/Rhymen/go-whatsapp v0.1.0 // indirect

Connecting to WhatsApp requires establishing a connection and authenticating using a QRCode that is scanned with your (already authenticated) phone. This session can be started in your Go app by running

waconn, err := whatsapp.NewConn(10 * time.Second) //10secs of timeout
if err != nil {
    panic(err)
}

I have written a small function that given a connection handles the login process if there is no stored session, or if the stored one cannot be retrieved.

func login(waconn *whatsapp.Conn) error {
	var sessionError error = fmt.Errorf("no session")
	//try to find a session stored in the file system
	session, sessionError := readSessionFromFileSystem()
	if sessionError == nil {
		//try to restore saved session
		session, sessionError = waconn.RestoreWithSession(session)
		if sessionError != nil {
			log.Printf("error restoring session from file system: %v\n", sessionError)
		}
	} else {
		log.Printf("no session found on session from file system: %v\n", sessionError)
	}
	if sessionError != nil {
		//perform a regular login
		session, sessionError = loginImpl(waconn)
		if sessionError != nil {
			return fmt.Errorf("error during login: %v\n", sessionError)
		}
	}
	//save session
	sessionError = writeSessionToFileSystem(session)
	if sessionError != nil {
		return fmt.Errorf("error saving session: %v\n", sessionError)
	}
	return nil
}

The function loginImpl gets the QRCode from the API and renders it on the terminal. At this point, if you are writing a Web Application or Service, you can transmit the QRCode as an image, or if it is a bot, just render it on the terminal using qrcode-termina-go.

func loginImpl(waconn *whatsapp.Conn) (whatsapp.Session, error) {
	qr := make(chan string)
	go func() {
		terminal := qrcodeTerminal.New()
		terminal.Get(<-qr).Print()
	}()
	return waconn.Login(qr)
}

Where readSessionFromFileSystem and writeSessiontoFileSystem are declared as follows

func readSessionFromFileSystem() (whatsapp.Session, error) {
	session := whatsapp.Session{}
	file, err := os.Open(os.TempDir() + "waSession.gob")
	if err != nil {
		return session, err
	}
	defer file.Close()
	decoder := gob.NewDecoder(file)
	err = decoder.Decode(&session)
	if err != nil {
		return session, err
	}
	return session, nil
}
func writeSessionToFileSystem(session whatsapp.Session) error {
	file, err := os.Create(os.TempDir() + "waSession.gob")
	if err != nil {
		return err
	}
	defer file.Close()
	encoder := gob.NewEncoder(file)
	err = encoder.Encode(session)
	if err != nil {
		return err
	}
	return nil
}

Using the login function is as simple as calling like this

//login or restore your WhatsApp connection
if err := login(waconn); err != nil {
    log.Fatalf("error logging in: %v\n", err)
}

The QRCode will be printed in the console, and go-whatsapp is smart enough and saves the session so it is possible to restore it without authenticating again.

WhatsApp login QR Code

Building Android for BeagleBone Black with Kernel 3.2

beaglebone-black

Note: SGX (hardware-accelerated OpenGL) is supported only in Kernel 3.2

Get source (using TI DevKit)

mkdir ~/ti-android-bbb
cd ~/ti-android-bbb
repo init -u git://gitorious.org/rowboat/manifest.git -m TI-Android-JB-4.2.2-DevKit-4.1.1.xml
repo sync

Setup

export PATH=~/ti-android-bbb/prebuilts/gcc/linux-x86/arm/arm-eabi-4.6/bin:$PATH

Build U-Boot and X-Loader

cd ~/ti-android-bbb/u-boot
make CROSS_COMPILE=arm-eabi- distclean
make CROSS_COMPILE=arm-eabi- am335x_evm_config
make CROSS_COMPILE=arm-eabi-

Build Android and Kernel with SGX On

cd ~/ti-android-bbb
make TARGET_PRODUCT=beagleboneblack OMAPES=4.x
make TARGET_PRODUCT=beagleboneblack fs_tarball

Creating the SD Card

mkdir ~/android-bbb-image
cp ti-android-bbb/kernel/arch/arm/boot/uImage ~/android-bbb-image
cp ti-android-bbb/u-boot/MLO ~/android-bbb-image
cp ti-android-bbb/u-boot/u-boot.img ~/android-bbb-image
cp ti-android-bbb/external/ti_android_utilities/am335x/u-boot-env/uEnv_beagleboneblack.txt ~/android-bbb-image
cp ti-android-bbb/out/target/product/beagleboneblack/rootfs.tar.bz2 ~/android-bbb-image
cp ti-android-bbb/external/ti_android_utilities/am335x/mk-mmc/mkmmc-android.sh ~/android-bbb-image
cd ~/android-bbb-image
sudo./mkmmc-android.sh /dev/sdX MLO u-boot.img uImage uEnv_beagleboneblack.txt rootfs.tar.bz2