Listing all endpoints with Gorilla Mux Router

golang

When adding different handlers to a Router, each one with its own base path (sub routing), it comes handy to list all the endpoints declared, not only as a debug option but also when you want your API to expose all the methods it’s support.


Gorilla Mux

My favourite web toolkit in Go is the Gorilla Web Toolkit, which apart of the router and dispatcher Gorilla Mux, has also other packages for handling cookies in a secure way, using web sockets, and even RPC over HTTP.

If the following example you will see a common pattern I use when building services using Gorilla Mux. I start by creating a service layer (in this case then bots are my services), and the an API layer that defines endpoints and calls the service (controller). So imagine have two separate controllers and services, or more, each one with its own path.

func main() {
	fmt.Println("Starting messaging bot")
	//Create our Gorilla mux main router which path is "/api"
	mainRouter := mux.NewRouter().
		PathPrefix("/api").
		Subrouter().
		StrictSlash(true)
	//Create WhatsApp bot
	whatsAppBot := bots.NewWhatsAppBot()
	//Create a sub router at "/api/whatsapp"
	api.NewAPI(whatsAppBot,
		mainRouter.PathPrefix("/whatsapp").
			Subrouter().
			StrictSlash(true))
	//Create Telegram bot and API server
	telegramBot := bots.NewTelegramBot()
	//Create a sub router at "/api/telegram"
	api.NewAPI(telegramBot,
		mainRouter.PathPrefix("/telegram").
			Subrouter().
			StrictSlash(true))
	fmt.Println("WhatsApp bot running on /api/whatsapp")
	fmt.Println("Telegram bot running on /api/telegram")
        //Start the HTTP server on port 8000
	http.ListenAndServe(":8000", mainRouter)
}

Every Router has a Walk function that let’s us navigate all the endpoints we define. If we want to debug which endpoint has been declared in our main router, we can call a function like the one below:

func printEndpoints(r *mux.Router) {
	r.Walk(func(route *mux.Route, router *mux.Router, ancestors []*mux.Route) error {
		path, err := route.GetPathTemplate()
		if err != nil {
			return nil
		}
		methods, err := route.GetMethods()
		if err != nil {
			return nil
		}
		fmt.Printf("%v %s\n", methods, path)
		return nil
	})
}

In my example this function will print

Starting messaging bot
[GET] /api/whatsapp/login
[POST] /api/whatsapp/message
[GET] /api/telegram/login
[POST] /api/telegram/message
Listening on port 8000

How to make a WhatsApp Bot in Go [Part 3 of N]

WhatsApp Logo

Processing incoming messages, either text or media, is one of the fundamental features a Bot must implement. In this post we will explore how to do this.


Connection Handlers

go-whatsapp has a connection struct which exposes functions to perform login, handle session, and send messages. The reception of those messages is done by add a (message) handler.

The Handler interface contains the only function that we must implement, the ErrorHandler function.

type Handler interface {
	HandleError(err error)
}

Every other handler, like the TextMessageHandler inherits from Handler and adds its own functions. For example:

type TextMessageHandler interface {
	Handler
	HandleTextMessage(message TextMessage)
}

The most important part of this design, is that we can add as many handlers as you want. The connection struct contains a (private) list of handlers, and it checks on runtime to which handler it must dispatch each message.

Moreover, due to the implementation of this list of handlers, we can add two or more handlers of the same type. For example, one TextMessageHandler can log any incoming message into a database, and another one can perform text analysis and send responses based on that.

This implementation relies heavily on the very famous Go’s duck type (or structural typing).

If it looks like a duck, swims like a duck, and quacks like a duck, then it probably is a duck.

Incoming messages

A TextMessage is represented by a struct with three public fields. Info, Text and ContextInfo.

type TextMessage struct {
	Info        MessageInfo
	Text        string
	ContextInfo ContextInfo
}

Text contains the message we received (or sent), Info (MessageInfo) gives us information about who and when the message was sent, and it’s status (pending, sent, delivered, read or played). And ContextInfo gives us additional information like if the message is a quote of another message (we also have access to this original message), or if it was forwarded.

Receiving messages is a simple as declaring a struct and implementing the TextMessageHandler interface. In this example I declared a struct called whatsAppMessageHandler which implements all of the functions requiered.

type whatsAppMessageHandler struct{}
func (whatsAppMessageHandler) HandleError(err error) {
	fmt.Fprintf(os.Stderr, "%+v", err)
}
func (whatsAppMessageHandler) HandleTextMessage(message whatsapp.TextMessage) {
	fmt.Printf("HandleTextMessage: %+v\n", message)
}

If we send a message, for example, “Hello WhatsApp Bot!” the following line will be printed in our terminal.

HandleTextMessage: {Info:{Id:F75C24CE7098AB206B22DF6B275533B8 RemoteJid:54911**12****@s.whatsapp.net SenderJid: FromMe:true Timestamp:1581616175 PushName: Status:4 Source:key:<remoteJid:"54911**12****@s.whatsapp.net" fromMe:true id:"F75C24CE7098AB206B22DF6B275533B8" > message:<conversation:"Hello, WhatsApp Bot!" > messageTimestamp:1581616175 status:READ } Text:Hello, WhatsApp Bot! ContextInfo:{QuotedMessageID: QuotedMessage:<nil> Participant: IsForwarded:false}}

The TextMessage struct in a more (JSON) pretty printed version looks like this

{
   "Info":{
      "Id":"F75C24CE7098AB206B22DF6B275533B8",
      "RemoteJid":"54911**12****@s.whatsapp.net",
      "SenderJid":"",
      "FromMe":true,
      "Timestamp":1581616175,
      "PushName":"",
      "Status":4,
      "Source":{
         "key":{
            "remoteJid":"54911**12****@s.whatsapp.net",
            "fromMe":true,
            "id":"F75C24CE7098AB206B22DF6B275533B8"
         },
         "message":{
            "conversation":"Hello, WhatsApp Bot!"
         },
         "messageTimestamp":1581616175,
         "status":4
      }
   },
   "Text":"Hello, WhatsApp Bot!",
   "ContextInfo":{
      "QuotedMessageID":"",
      "QuotedMessage":null,
      "Participant":"",
      "IsForwarded":false
   }
}

A reply quoting this message will look like

{
   "Info":{
      "Id":"31D97076ED9DFC5C95D82A8B0B9FE00E",
      "RemoteJid":"54911**12****@s.whatsapp.net",
      "SenderJid":"",
      "FromMe":true,
      "Timestamp":1581616344,
      "PushName":"",
      "Status":4,
      "Source":{
         "key":{
            "remoteJid":"54911**12****@s.whatsapp.net",
            "fromMe":true,
            "id":"31D97076ED9DFC5C95D82A8B0B9FE00E"
         },
         "message":{
            "extendedTextMessage":{
               "text":"Reply on a message",
               "previewType":0,
               "contextInfo":{
                  "stanzaId":"F75C24CE7098AB206B22DF6B275533B8",
                  "participant":"54911**12****@s.whatsapp.net",
                  "quotedMessage":{
                     "conversation":"Hello, WhatsApp Bot!"
                  }
               }
            }
         },
         "messageTimestamp":1581616344,
         "status":4
      }
   },
   "Text":"Reply on a message",
   "ContextInfo":{
      "QuotedMessageID":"F75C24CE7098AB206B22DF6B275533B8",
      "QuotedMessage":{
         "conversation":"Hello, WhatsApp Bot!"
      },
      "Participant":"54911**12****@s.whatsapp.net",
      "IsForwarded":false
   }
}

As this reply was done on a specific message, the same information appears on the top-level ContextInfo and inside Info struct.

How to make a WhatsApp Bot in Go [Part 2 of N]

WhatsApp Logo

In this second step, we are going to analyse how to use the API of go-whatsapp described in the previous blog post, and what considerations we should take into account in order to create a bot.


Sending a text message

The WhatsApp Connection instance that is returned when creating a Session has several messages to send text messages, images, create groups, receive messages and even query for the contact list.

For sending a text message you have to create an instance of TextMessage indicating at least the text message an a receipt.

text := whatsapp.TextMessage{
    Info: whatsapp.MessageInfo{
	RemoteJid: "<receiver's phone number>@s.whatsapp.net",
    },
    Text: "Text message sent from Golang bot",
}
sendResult, err := waconn.Send(text)

where whatsApp refers to the module go-whatsapp imported as

import (
    "github.com/Rhymen/go-whatsapp"
)

whatsApp.TextMessage is an struct which fields are

type TextMessage struct {
	Info        MessageInfo
	Text        string
	ContextInfo ContextInfo
}

and whatsApp.MessageInfo contains

type MessageInfo struct {
	Id        string
	RemoteJid string
	SenderJid string
	FromMe    bool
	Timestamp uint64
	PushName  string
	Status    MessageStatus
	Source *proto.WebMessageInfo
}

The first important thing to notice is that MesssageInfo only contains on RemoteJid, so it’s not possible to a single message to multiple receivers (broadcasting).

Note: Trying to concatenate several RemoteJid (separated with comma, for example) leads to a crash in the mobile app where WhatsApp runs! unbelievable.

The response to the Send command is a string and an error. Where error is nil if the operation succeeded, and string contains either the MessageId if everything is okey, or the literal string ERROR in case of failure.


Other types of messages can be sent using this module. The existent implementation of go-whatsapp, by the time this post was created, allows a user to send text, photos, videos, documents (files), audio, locations, live locations contact information (vCards), and even Stickers!.

Image, Audio and Video messages receive its content in a io.Reader property. Several fields of these structs remain unexpected as they are needed for media upload/download and validations.

The definition of each of these structures are

Image Message
type ImageMessage struct {
	Info          MessageInfo
	Caption       string
	Thumbnail     []byte
	Type          string
	Content       io.Reader
	url           string
	mediaKey      []byte
	fileEncSha256 []byte
	fileSha256    []byte
	fileLength    uint64
	ContextInfo   ContextInfo
}
Video Message
type VideoMessage struct {
	Info          MessageInfo
	Caption       string
	Thumbnail     []byte
	Length        uint32
	Type          string
	Content       io.Reader
	GifPlayback   bool
	url           string
	mediaKey      []byte
	fileEncSha256 []byte
	fileSha256    []byte
	fileLength    uint64
	ContextInfo   ContextInfo
}
Audio Message
type AudioMessage struct {
	Info          MessageInfo
	Length        uint32
	Type          string
	Content       io.Reader
	Ptt           bool
	url           string
	mediaKey      []byte
	fileEncSha256 []byte
	fileSha256    []byte
	fileLength    uint64
	ContextInfo   ContextInfo
}
Document Message
type DocumentMessage struct {
	Info          MessageInfo
	Title         string
	PageCount     uint32
	Type          string
	FileName      string
	Thumbnail     []byte
	Content       io.Reader
	url           string
	mediaKey      []byte
	fileEncSha256 []byte
	fileSha256    []byte
	fileLength    uint64
	ContextInfo   ContextInfo
}
Location Message
type LocationMessage struct {
	Info             MessageInfo
	DegreesLatitude  float64
	DegreesLongitude float64
	Name             string
	Address          string
	Url              string
	JpegThumbnail    []byte
	ContextInfo      ContextInfo
}
Live Location Message
type LiveLocationMessage struct {
	Info                              MessageInfo
	DegreesLatitude                   float64
	DegreesLongitude                  float64
	AccuracyInMeters                  uint32
	SpeedInMps                        float32
	DegreesClockwiseFromMagneticNorth uint32
	Caption                           string
	SequenceNumber                    int64
	JpegThumbnail                     []byte
	ContextInfo                       ContextInfo
}
Stickers
type StickerMessage struct {
	Info MessageInfo
	Type          string
	Content       io.Reader
	url           string
	mediaKey      []byte
	fileEncSha256 []byte
	fileSha256    []byte
	fileLength    uint64
	ContextInfo ContextInfo
}
Contact Message (vCard)
type ContactMessage struct {
	Info MessageInfo
	DisplayName string
	Vcard       string
	ContextInfo ContextInfo
}

How to make a WhatsApp Bot in Go [Part 1 of N]

WhatsApp Logo

Such a long time since I’ve published my last post, I’m back with a simple and small post: How to make a simple Whatsapp Bot in Go.


Note about this post and its examples

The most important thing is to take into account that WhatsApp does not provide a public API that you can use to create a Bot, or any kind of program that interacts with it. Every open source (and free) solution found on the Internet relies on some sort of scrapping or reverse engineering. So the solution presented in this post may not be stable (or need an update) by the time you read this post.


Requirements

  • Go (Golang) version 1.11 or newer (Go modules are used)
  • An active WhatsApp account logged in a phone

Dependencies

Getting Started

From a terminal in your favourite OS (one of the beautiful things of a Multiplatform language as Golang) create a module as follow:

$ mkdir whatsapp-bot-test
$ cd whatsapp-bot-test
$ go mod init github.com/eaceto/whatsapp-bot-test
go: creating new go.mod: module github.com/eaceto/whatsapp-bot

In order to import Lucas Engelke’s go-whatsapp run the following command inside your project directory

$ go get github.com/Rhymen/go-whatsapp

Now the content of go.mod will look like this

module github.com/eaceto/whatsapp-bot
go 1.13
require github.com/Rhymen/go-whatsapp v0.1.0 // indirect

Connecting to WhatsApp requires establishing a connection and authenticating using a QRCode that is scanned with your (already authenticated) phone. This session can be started in your Go app by running

waconn, err := whatsapp.NewConn(10 * time.Second) //10secs of timeout
if err != nil {
    panic(err)
}

I have written a small function that given a connection handles the login process if there is no stored session, or if the stored one cannot be retrieved.

func login(waconn *whatsapp.Conn) error {
	var sessionError error = fmt.Errorf("no session")
	//try to find a session stored in the file system
	session, sessionError := readSessionFromFileSystem()
	if sessionError == nil {
		//try to restore saved session
		session, sessionError = waconn.RestoreWithSession(session)
		if sessionError != nil {
			log.Printf("error restoring session from file system: %v\n", sessionError)
		}
	} else {
		log.Printf("no session found on session from file system: %v\n", sessionError)
	}
	if sessionError != nil {
		//perform a regular login
		session, sessionError = loginImpl(waconn)
		if sessionError != nil {
			return fmt.Errorf("error during login: %v\n", sessionError)
		}
	}
	//save session
	sessionError = writeSessionToFileSystem(session)
	if sessionError != nil {
		return fmt.Errorf("error saving session: %v\n", sessionError)
	}
	return nil
}

The function loginImpl gets the QRCode from the API and renders it on the terminal. At this point, if you are writing a Web Application or Service, you can transmit the QRCode as an image, or if it is a bot, just render it on the terminal using qrcode-termina-go.

func loginImpl(waconn *whatsapp.Conn) (whatsapp.Session, error) {
	qr := make(chan string)
	go func() {
		terminal := qrcodeTerminal.New()
		terminal.Get(<-qr).Print()
	}()
	return waconn.Login(qr)
}

Where readSessionFromFileSystem and writeSessiontoFileSystem are declared as follows

func readSessionFromFileSystem() (whatsapp.Session, error) {
	session := whatsapp.Session{}
	file, err := os.Open(os.TempDir() + "waSession.gob")
	if err != nil {
		return session, err
	}
	defer file.Close()
	decoder := gob.NewDecoder(file)
	err = decoder.Decode(&session)
	if err != nil {
		return session, err
	}
	return session, nil
}
func writeSessionToFileSystem(session whatsapp.Session) error {
	file, err := os.Create(os.TempDir() + "waSession.gob")
	if err != nil {
		return err
	}
	defer file.Close()
	encoder := gob.NewEncoder(file)
	err = encoder.Encode(session)
	if err != nil {
		return err
	}
	return nil
}

Using the login function is as simple as calling like this

//login or restore your WhatsApp connection
if err := login(waconn); err != nil {
    log.Fatalf("error logging in: %v\n", err)
}

The QRCode will be printed in the console, and go-whatsapp is smart enough and saves the session so it is possible to restore it without authenticating again.

WhatsApp login QR Code