How to create a Twitter bot from scratch with Golang
Add to bookmarksWed May 15 2019
So a little background: I recently picked up Golang and decided to create a Twitter bot as a side project. Then came the problem. There is little to no documentation on using the Twitter API with Golang š(particularly the oauth1 and CRC encryption parts of it). So after some days of trial and error and finally completing it, I want to share the process. Hopefully, this helps someone out there.
What are we going toĀ build?
We will build a Twitter bot that will be served from our local machine. It will respond to any tweet that is tagged in with a āhello worldā.
Hereās a brief explanation as to what this go program will do. It will:
- Listen and respond to webhook CRC validation.
- Register a webhook URL that points to it.
- Listen for tweets and respond with āhello worldā.
What do youĀ need?
- Some basic knowledge of Golang
- An approved Twitter developer account. How to apply.
- You should have an account activity API dev environment set upāāācall it
dev
for this project - A Twitter app with generated consumer keys and access tokens (Read and write access)
- Golang installed on your development machine.
- Some determination.
Ready? LetāsĀ Go
First things first. Create your project folder in your $GOPATH/src/
Ā . Weāll be calling this project and our folder hellobot
Ā . In it create the intro file /hellobot.go
package hellobot
func main(){
}
The first thing we need to do is to create an endpoint for our app to listen to CRC checks and respond. Twitter sums up the requirements for the check pretty well.
Setting up aĀ server
package main
import (
"fmt"
"github.com/gorilla/mux"
"github.com/joho/godotenv"
"net/http"
)
func main(){
//Load env
err := godotenv.Load()
if err != nil {
log.Fatal("Error loading .env file")
fmt.Println("Error loading .env file")
}
fmt.Println("Starting Server")
//Create a new Mux Handler
m := mux.NewRouter()
//Listen to the base url and send a response
m.HandleFunc("/", func(writer http.ResponseWriter, _ *http.Request) {
writer.WriteHeader(200)
fmt.Fprintf(writer, "Server is up and running")
})
//Listen to crc check and handle
m.HandleFunc("/webhook/twitter", CrcCheck).Methods("GET")
//Start Server
server := &http.Server{
Handler: m,
}
server.Addr = ":9090"
server.ListenAndServe()
}
func CrcCheck(writer http.ResponseWriter, request *http.Request){
//TODO implement CRC check
}
The first thing we do is load theĀ .env
file. For that we are using the godotenv plugin. TheĀ .env file is usually in this format:
CONSUMER_KEY=
CONSUMER_SECRET=
ACCESS_TOKEN_KEY=
ACCESS_TOKEN_SECRET=
WEBHOOK_ENV=dev
APP_URL=
Note: We will be using basic
go get
to install all our dependencies considering the tiny size of our project
Then we set up our server using mux as our handler, and listen to the base route and webhook/twitter
Ā . If you install this using go install
and run hellobot
, when you run it and navigate to your localhost:9090 you should see the message
CRC validation
Now for the CRC, update your CrcCheck()
function with the following code:
package main
import (
...
"crypto/hmac"
"crypto/sha256"
"encoding/base64"
"encoding/json"
)
func main(){
...
}
func CrcCheck(writer http.ResponseWriter, request *http.Request){
//Set response header to json type
writer.Header().Set("Content-Type", "application/json")
//Get crc token in parameter
token := request.URL.Query()["crc_token"]
if len(token) < 1 {
fmt.Fprintf(writer,"No crc_token given")
return
}
//Encrypt and encode in base 64 then return
h := hmac.New(sha256.New, []byte(os.Getenv("CONSUMER_SECRET")))
h.Write([]byte(token[0]))
encoded := base64.StdEncoding.EncodeToString(h.Sum(nil))
//Generate response string map
response := make(map[string]string)
response["response_token"] = "sha256=" + encoded
//Turn response map to json and send it to the writer
responseJson, _ := json.Marshal(response)
fmt.Fprintf(writer, string(responseJson))
}
Here what we do in the function is:
- Set the header to āapplication/jsonā
- Get the crc_token URL parameter
- Encrypt it using Hmac sha256 and encode it
- Print it to the response writer
Make sure to replace the CONSUMER_SECRET
with the actual consumer secret key for your app. Now if you navigate to localhost:9090/webhook/twitter?crc_token=test
you should get a response similar to this:
Now that we have a working CRC route, time to register our webhook. Now a couple of things to note here. Twitter will not accept localhost
based URLs nor will it accept a URL with a port number or a non-https URL as a webhook. A way around that during development is to use a service like ngrok. Simply install it and start up a dev server pointing to your 9090 port:
ngrok http 9090
You should see a response similar to this:
Now if you go to the <id>.ngrok.io
URL you should see the same response as the localhost:9090. Donāt forget to add the URL to yourĀ .env file APP_ENV
Registering theĀ webhook
For this tutorial we are going to check for the presence of a register
flag in the arguments list. You can add this to your code:
//client.go
package main
func registerWebhook(){
}
...
func main(){
fmt.Println("Starting Server")
//Check for -register in agument list
if args := os.Args; len(args) > 1 && args[1] == "-register"{
go registerWebhook()
}
...
}
...
Here our bot is checking for the presence of -register
in the argument list. Then it runs registerWebhook()
as a goroutine. We are defining the registerWebhook()
function in a client.go
file which we will use for all Twitter requests. Now, for the functionās body:
package main
import (
"encoding/json"
"fmt"
"github.com/dghubble/oauth1"
"io/ioutil"
"net/http"
"net/url"
"os"
)
func CreateClient() *http.Client {
//Create oauth client with consumer keys and access token
config := oauth1.NewConfig(os.Getenv("CONSUMER_KEY"), os.Getenv("CONSUMER_SECRET"))
token := oauth1.NewToken(os.Getenv("ACCESS_TOKEN_KEY"), os.Getenv("ACCESS_TOKEN_SECRET"))
return config.Client(oauth1.NoContext, token)
}
func registerWebhook(){
fmt.Println("Registering webhook...")
httpClient := CreateClient()
//Set parameters
path := "https://api.twitter.com/1.1/account_activity/all/" + os.Getenv("WEBHOOK_ENV") + "/webhooks.json"
values := url.Values{}
values.Set("url", os.Getenv("APP_URL")+"/webhook/twitter")
//Make Oauth Post with parameters
resp, _ := httpClient.PostForm(path, values)
defer resp.Body.Close()
//Parse response and check response
body, _ := ioutil.ReadAll(resp.Body)
var data map[string]interface{}
if err := json.Unmarshal([]byte(body), &data); err != nil {
panic(err)
}
fmt.Println("Webhook id of " + data["id"].(string) + " has been registered")
}
So, a break down of the new code. The first thing is to create a CreateClient()
function. This function returns a pointer to an OAuth http.Client
that we can then use to make all Twitter requests on behalf of our botās account. Remember to run go get
in the project folder so it can fetch the neat go library we use for all OAuth requests. In the registerWebhook
function, we:
- Fetch a client
- Pass our webhookās URL as a parameter using
url.Values
- Make a post response to the register webhook endpoint then unmarshall (decode) and read the response
Next, we need our code to subscribe our webhook to events.
Note: You can use the account-activity-dashboard app created by Twitter for managing webhooks during development
Update your client.go
file as shown below:
...
func CreateClient() *http.Client {
//Create oauth client with consumer keys and access token
...
subscribeWebhook()
}
func subscribeWebhook(){
fmt.Println("Subscribing webapp...")
client := CreateClient()
path := "https://api.twitter.com/1.1/account_activity/all/" + os.Getenv("WEBHOOK_ENV") + "/subscriptions.json"
resp, _ := client.PostForm(path, nil)
body, _ := ioutil.ReadAll(resp.Body)
defer resp.Body.Close()
//If response code is 204 it was successful
if resp.StatusCode == 204 {
fmt.Println("Subscribed successfully")
} else if resp.StatusCode!= 204 {
fmt.Println("Could not subscribe the webhook. Response below:")
fmt.Println(string(body))
}
}
The code above is very straightforward. Cand heck after registering, subscribe to events and check for a status code of 204
Ā . Now if you run go install
on your code and run the code as hellobot -register
you should get the following response:
Starting Server
Registering webhook...
Webhook id of <hook\_id> has been registered
Subscribing webapp...
Subscribed successfully
Listening forĀ events
Now we need our webhook to actually to listen for events once the URL is called. Update your files as shown below:
..
//Struct to parse webhook load
type WebhookLoad struct {
UserId string `json:"for_user_id"`
TweetCreateEvent []Tweet `json:"tweet_create_events"`
}
//Struct to parse tweet
type Tweet struct {
Id int64
IdStr string `json:"id_str"`
User User
Text string
}
//Struct to parse user
type User struct {
Id int64
IdStr string `json:"id_str"`
Name string
Handle string `json:"screen_name"`
}
func CreateClient() *http.Client {
...
...
func main(){
...
//Listen to crc check and handle
m.HandleFunc("/webhook/twitter", CrcCheck).Methods("GET")
//Listen to webhook event and handle
m.HandleFunc("/twitter/webhook", WebhookHandler).Methods("POST")
//Start Server
server := &http.Server{
Handler: m,
}
server.Addr = ":9090"
server.ListenAndServe()
}
...
func WebhookHandler(writer http.ResponseWriter, request *http.Request) {
}
What we are doing in the hellobot.dev
is listening for post requests to our routes and passing them to the appropriate function. While in the client.go
we are adding the appropriate structs we would use to parse the JSON payload to our bot.
Now update your code so it sends the tweet on the tag.
...
func SendTweet(tweet string, reply_id string) (*Tweet, error) {
fmt.Println("Sending tweet as reply to " + reply_id)
//Initialize tweet object to store response in
var responseTweet Tweet
//Add params
params := url.Values{}
params.Set("status",tweet)
params.Set("in_reply_to_status_id",reply_id)
//Grab client and post
client := CreateClient()
resp, err := client.PostForm("https://api.twitter.com/1.1/statuses/update.json",params)
if err != nil {
return nil, err
}
defer resp.Body.Close()
//Decode response and send out
body, _ := ioutil.ReadAll(resp.Body)
fmt.Println(string(body))
err = json.Unmarshal(body, &responseTweet)
if err != nil{
return nil,err
}
return &responseTweet, nil
}
...
func WebhookHandler(writer http.ResponseWriter, request *http.Request) {
fmt.Println("Handler called")
//Read the body of the tweet
body, _ := ioutil.ReadAll(request.Body)
//Initialize a webhok load obhject for json decoding
var load WebhookLoad
err := json.Unmarshal(body, &load)
if err != nil {
fmt.Println("An error occured: " + err.Error())
}
//Check if it was a tweet_create_event and tweet was in the payload and it was not tweeted by the bot
if len(load.TweetCreateEvent) < 1 || load.UserId == load.TweetCreateEvent[0].User.IdStr {
return
}
//Send Hello world as a reply to the tweet, replies need to begin with the handles
//of accounts they are replying to
_, err = SendTweet("@"+load.TweetCreateEvent[0].User.Handle+" Hello World", load.TweetCreateEvent[0].IdStr)
if err != nil {
fmt.Println("An error occured:")
fmt.Println(err.Error())
} else{
fmt.Println("Tweet sent successfully")
}
}
The updates we added to our source files are simply to respond to webhook events. Check if it was a tweet_create_event and send a response as a reply using the SendTweet()
method in our client.go
file.
Note: Any tweet being sent as a reply needs to include the handle of the user it is replying to as the initial content of the reply
Now if you run this with the appropriate credentials your bot should respond to tags and reply with āHello Worldā.
Conclusion
Now thatās done, and since this is an extremely basic version of a bot, you can try adding a few things:
- Checking and ignoring retweet events
- Adding a name to the response
- Responding to the tweet in the case of an error anywhere on the app.
The code for this walkthrough is on Github here. Feel free to fork and play around with it
Cheers!