For the past 3 months, I've done a few development explorations with Twitter API. The results were (shameless plugs) my two projects, and . My experience has been less than stellar, to say the least. In my experience, the problem is that left a lot to desire. The biggest issue is , the new webhook concept that is quite challenging to understand and work with. In this post, I hope to explain how it works and build a fun Twitter bot to put these concepts in practice. Tweet Scheduler thank u, next Twitter Developer documentation Account Activity API Webhook and Subscription Here is how Twitter webhook works. When you use Twitter webhook to follow a user, you get notified whenever new events related to that user happen. Here is that you can subscribe to. the list of all events Simple enough, right? In many products, there is a nice and pretty UI where you can register the webhook URL. With Twitter you have to do everything programmatically. On top of that, Twitter also adds in the concept of subscription. Let me elaborate with this diagram about Twitter subscriptions. In many cases, in order to let a Twitter user to subscribe to your app, you need to handle OAuth to ask for permissions. Here are 2 use cases, one where you need to ask for permission and one where you don't. Use case 1 You're building a Twitter bot where it subscribes to **its** own activities. For example, [@this_vid](https://twitter.com/this_vid), a Twitter bot that will download the video for users on mention, subscribes to the mention event. In this case, when creating the Twitter app from that bot account, Twitter will give you an access token and secret that work for that account only. It is good enough for this use case, so you don't have to worry about OAuth. You will still need to use those token and secret to add a subscription to your webhook. Use case 2 You're building a Twitter bot where it subscribes to **others** activities. For example, in [thank u, next](https://tynext.app), the app subscribes to many different users direct messages events instead of its own Twitter account. In this cases, you need to ask for permission to gain access to their activities. This involves Twitter authentication using OAuth 1. After a user signs in to your app, you will receive an access token and a secret token. Using those tokens, you can then add that user's subscription to your webhook. To sum up Register webhook to Twitter with an API call For each account whose activities you want to subscribe to, first ask for permission with OAuth to retrieve an access token and a secret token Then add a new subscription with another API call Practice So, I'm gonna show you how Twitter webhook works by building . Whenever you mention the bot in your tweet, it will reply with a dad joke. @GimmeDadJoke The jokes are going to be fetched from . icanhazdadjoke I will write everything in JavaScript/NodeJS and deploy it on . Zeit Now Let's do it. Create Twitter App First, create a new Twitter account with the desired handle; in my case: `GimmeDadJoke`. Then, fill in to apply for access to Twitter API. It may look scary, but it seems like an automated process, so don't worry. a Twitter form Once applied and given access, you can create a new Twitter app from . this form And lastly, navigate to the `Keys and tokens` tab to get access to your app API keys. From there, click on the button that will create access token and access token secret. Let me explain what these tokens do: - `Consumer API Keys`, including `API key` and `API secret key`, are tokens that let Twitter know which app is calling the API. They are needed in every API call your app will perform. - `Access token` and `Access token secret` are tokens that identify which user is performing the action. For every user that gives your app permission, you will receive their access token and access token secret. In this case, these two tokens belongs to the app's owner user. In my case, the account `@GimmeDadJoke`. Next, go to the `Permissions` tab and change your access permission to `Read, write, and Direct Messages`. Even if you don't need to access DM, you still need to use this permission because the Account Activity API automatically sends the DM data to your webhook. Finally need to create a Twitter development environment to use subscription APIs. Navigate to the and click the "Set up dev environment" button in the Account Activity API / Sandbox section. You'll then be prompted to name your environment and select which app it belongs to. In my case, I name it `development` and select the app that I just created. environment dashboard That's it. We're finally done with Twitter set up and ready to write some code. Set up Navigate to a folder of your choice and start setting up our code: mkdir gimmedadjoke && gimmedadjoke cd Webhook URL If you haven't used Zeit Now before, you also need to install their CLI and login or create an account: npm i -g now now login Now, you're ready to set up the API serverless function: mkdir api && api yarn init -y cd Inside `package.json` file, add in the start and build scripts: // gimmedadjoke/api/package.json { : { : , : } } "scripts" "start" "now dev" "build" "now --prod" Now, let's create the webhook function: crypto = ( ); { hmac = crypto .createHmac( , process.env.TWITTER_CONSUMER_SECRET) .update(crcToken) .digest( ); ; } { crcToken = req.query.crc_token; (crcToken) { res.status( ).send({ : createCrcResponseToken(crcToken) }); } { res.status( ).send({ : }); } } { body = req.body; .log(body); res.status( ).json(body); } .exports = { { (req.method) { : getHandler(req, res); : postHandler(req, res); : res.status( ).json({ : }); } } (error) { .log(error.message); res.status( ).send(); } }; // gimmedadjoke/api/webhook.js const require "crypto" ( ) function createCrcResponseToken crcToken const "sha256" "base64" return `sha256= ` ${hmac} ( ) function getHandler req, res const if 200 response_token else 400 message "Error: crc_token missing from request." ( ) function postHandler req, res const console 200 module ( ) => req, res try switch case "GET" return case "POST" return default return 410 message "Unsupported Request Method" catch console 500 Some brief explanation of the purpose of the code: This route expects 2 types of request, a GET and a POST. A GET route is for authentication purpose. When you register the webhook, Twitter will send a to make sure that you're the one who controls the webhook URL. test request A POST route is for the actual events. Whenever a new event happens, Twitter will send a POST request to this route. Currently, we're not doing anything yet. To run function in development mode, you can run `yarn start`. You can try to make a POST request to `http//localhost:3000/webhook.js` to confirm your function is working. Deploy to Now To set up deployment, create a `now.json` file: // gimmedadjoke/api/now.json { : , : , : [{ : , : }], : { : , : , : , : } } "name" "gimmedadjoke" "version" 2 "builds" "src" "webhook.js" "use" "@now/node" "env" "TWITTER_CONSUMER_KEY" "@gimmedadjoke-consumer-key" "TWITTER_CONSUMER_SECRET" "@gimmedadjoke-consumer-secret" "TWITTER_ACCESS_TOKEN" "@gimmedadjoke-access-token" "TWITTER_ACCESS_TOKEN_SECRET" "@gimmedadjoke-access-token-secret" Next, you need to set up the environment variables: now secrets add gimmedadjoke-consumer-key TWITTER_CONSUMER_KEY now secrets add gimmedadjoke-consumer-secret TWITTER_CONSUMER_SECRET now secrets add gimmedadjoke-access-token TWITTER_ACCESS_TOKEN now secrets add gimmedadjoke-access-token-secret TWITTER_ACCESS_TOKEN_SECRET Don't forget to change `gimmedadjoke` to your bot name and use the correct tokens for their values. After this, you are ready to deploy your function that powers the Twitter bot. Run `yarn deploy`. Now, to test if your deployment is successful: Scripts Once your Now deployment is ready to go, you can start writing a few Node scripts to set up webhook and subscriptions. mkdir scripts && scripts yarn init -y yarn add dotenv request request-promise pwd # make sure you're at your bot root directory cd In your `.env` file inside `scripts` directory: =https://api.twitter.com/1.1 =your token app dashboard =your token app dashboard =your token app dashboard =your token app dashboard TWITTER_BEARER_TOKEN= =development ( whatever you name it when creating your dev environment) =https://gimmedadjoke.now.sh/webhook.js (your Now webhook function) TWITTER_API_URL TWITTER_CONSUMER_KEY from TWITTER_CONSUMER_SECRET from TWITTER_ACCESS_TOKEN from TWITTER_ACCESS_TOKEN_SECRET from TWITTER_WEBHOOK_ENV or WEBHOOK_URL Wait, what is that mysterious Bearer Token? is yet another way for your app to authenticate with Twitter. Quite confusing, I know. Don't worry, I'll walk you through the code we're about to write. Bearer Token To make the scripts files simpler, you will write a big `api` file that takes care of the Twitter API interaction. Here are everything we're going to do with our scripts: ( ).config(); request = ( ); TWITTER_API_URL = process.env.TWITTER_API_URL; TWITTER_CONSUMER_KEY = process.env.TWITTER_CONSUMER_KEY; TWITTER_CONSUMER_SECRET = process.env.TWITTER_CONSUMER_SECRET; TWITTER_ACCESS_TOKEN = process.env.TWITTER_ACCESS_TOKEN; TWITTER_ACCESS_TOKEN_SECRET = process.env.TWITTER_ACCESS_TOKEN_SECRET; TWITTER_BEARER_TOKEN = process.env.TWITTER_BEARER_TOKEN; TWITTER_WEBHOOK_ENV = process.env.TWITTER_WEBHOOK_ENV; oauth = { : TWITTER_CONSUMER_KEY, : TWITTER_CONSUMER_SECRET, : TWITTER_ACCESS_TOKEN, : TWITTER_ACCESS_TOKEN_SECRET }; authorizationHeaders = { : }; exports.getBearerToken = { [request.post](http: url: , : { : process.env.TWITTER_CONSUMER_KEY, : process.env.TWITTER_CONSUMER_SECRET }, : }); }; exports.getWebhook = { request.get({ : , : authorizationHeaders, : }); }; exports.createWebhook = { [request.post](http: url: , oauth, : { : webhookUrl }, : }); }; exports.deleteWebhook = { request.delete({ : , oauth }); }; exports.getSubscription = { request.get({ : , oauth, : }); }; exports.createSubscription = { request.post({ : , oauth, : }); }; exports.deleteSubscription = { request.delete({ : , : authorizationHeaders, : }); }; // gimmedadjoke/scripts/src/api.js require "dotenv" const require "request-promise" const const const const const const const const consumer_key consumer_secret token token_secret const authorization `Bearer ` ${TWITTER_BEARER_TOKEN} ( ) function return //request.post)({ "[https://api.twitter.com/oauth2/token?grant_type=client_credentials](https://api.twitter.com/oauth2/token?grant_type=client_credentials)" auth user pass json true ( ) function return url ` /account_activity/all/ /webhooks.json` ${TWITTER_API_URL} ${TWITTER_WEBHOOK_ENV} headers json true ( ) function webhookUrl return //request.post)({ ` /account_activity/all/ /webhooks.json` ${TWITTER_API_URL} ${TWITTER_WEBHOOK_ENV} form url json true ( ) function webhookId return url ` /account_activity/all/ /webhooks/ .json` ${TWITTER_API_URL} ${TWITTER_WEBHOOK_ENV} ${webhookId} ( ) function return url ` /account_activity/all/ /subscriptions.json` ${TWITTER_API_URL} ${ .webhookEnv} this json true ( ) function return url ` /account_activity/all/ /subscriptions.json` ${TWITTER_API_URL} ${TWITTER_WEBHOOK_ENV} json true ( ) function userId return url ` /account_activity/all/ /subscriptions/ .json` ${TWITTER_API_URL} ${ .webhookEnv} this ${userId} headers json true Here are all the functions we've written: getBearerToken getWebhook createWebhook deleteWebhook getSubscription createSubscription deleteSubscription We'll create 7 scripts that directly correlate with these functions . later In the mean time: Take a break Hey, well done for following along. I know that has been a lot of code, but that is the most of the code for this section. When you come back, I'll explain what these functions are really doing, and hopefully you can gain a better and more practical understanding of how Twitter webhook actually works. Now, go get a snack or grab a cup of tea. You totally deserve it. Bearer token First of all, let's write a script to retrieve your app bearer token: api = ( ); { api .getBearerToken() .then( { .log(response); }) .catch( { .log(error.message); }); } run(); // gimmedadjoke/scripts/src/bearerToken.js const require "./api" ( ) function run => response console => error console Then, in the terminal: node src/bearerToken.js pwd # make sure you're inside the scripts directory If things go well, you should see something like this: { token_type: 'bearer', access_token: 'some_token' } Copy that token and put it into your `.env` file. You're ready to write some webhook goodness. Webhook Let's warm up by writing a script that retrieve all current webhooks associate with our app. api = ( ); { api .getWebhook() .then( { .log(response); }) .catch( { .log(error.message); }); } run(); // gimmedadjoke/scripts/src/webhook.get.js const require "./api" ( ) function run => response console => error console Then, run `node src/webhook.get.js`. If the response is `[]`, you're on the right track. To add webhook to your app: api = ( ); WEBHOOK_URL = process.env.WEBHOOK_URL; { api .createWebhook(WEBHOOK_URL) .then( { .log(response); }) .catch( { .log(error.message); }); } run(); // gimmedadjoke/scripts/src/webhook.post.js const require "./api" const ( ) function run => response console => error console When you run it, you may notice it takes slightly longer than other commands you've run. That is because a lot is going on here: You asked Twitter to register your webhook URL. Twitter sent a test request to your function on Now. Once it's successfully tested, Twitter answered back to you with your new webhook information. And lastly, let's create a script that remove the webhook: api = ( ); { api .getWebhook() .then( { webhookId = response[ ].id; api .deleteWebhook(webhookId) .then( { .log( ); }) .catch( { .log(error.message); }); }) .catch( { .log(error.message); }); } run(); // gimmedadjoke/scripts/src/webhook.delete.js const require "./api" ( ) function run => response const 0 => response console "Successfully delete webhook" => error console => error console Now, with these 3 scripts, you can register your webhook, get its information, and remove it when you want to. Although you can register your webhook, it is not doing anything just yet. You also need to use `subscription` to make your webhook functional. Subscription Similar to the scripts you wrote for webhooks, you will now write another 3 scripts for subscriptions. I'm going to show you the code, and we can talk about it afterwards: api = ( ); { api .createSubscription() .then( { .log( ); }) .catch( { .log(error.message); }); } run(); // gimmedadjoke/scripts/src/subscription.get.js const require "./api" ( ) function run => response console "Successfully subscribe the app owner user to webhook." => error console api = ( ); { api .createSubscription() .then( { .log( ); }) .catch( { .log(error.message); }); } run(); // gimmedadjoke/scripts/src/subscription.post.js const require "./api" ( ) function run => response console "Successfully subscribe the app owner user to webhook." => error console api = ( ); { userId = process.env.TWITTER_ACCESS_TOKEN.split( )[ ]; api .deleteSubscription(userId) .then( { .log( ); }) .catch( { .log(error.message); }); } run(); // gimmedadjoke/scripts/src/subscription.delete.js const require "./api" ( ) function run const "-" 0 => response console "Successfully remove subscription." => error console Try subscribing the app owner Twitter user (the user associated with the developer account -- i.e in this case @gimmedadjoke) to your webhook. node src/subscription.post.js If your script runs without error, congratulations, your webhook is now in action. In my case, whenever the Twitter user `@GimmeDadJoke` receives any new event, I will know about it. You can confirm by sending a direct message to your Twitter bot and check the function's log. 🎉🎉 Respond with a dad joke Because this is not the focus of this guide, you can check out the actual handling of the webhook request in the . source code In general, this is how you can handle it: { body = req.body; (body[THE_EVENT_YOU_CARE_ABOUT]) { res.status( ).send(); } { res.status( ).send(); } } ( ) function postHandler req, res const if // do stuff return 200 else return 200 The end result: 🎉🎉 Extend Twitter bot to subscribe to other users' events As of right now, the bot is listening to its own Twitter account event. You can extend it to subscribe to other users as well. This use case is beyond the scope of this guide, but I'll give you a general approach to implement it. Maybe it can be a fun challenge for you. Here is the general approach: Implement "sign in with Twitter" functionality using OAuth. One way you can do that is by using [Firebase](https://firebase.google.com/docs/auth/web/twitter-login). Once signed in, your app will receive the access token and access token secret. Using those tokens to add a new subscription to your bot. This is exactly the same as when you add the subscription to your own bot user: request.post({ : , : { : process.env.TWITTER_CONSUMER_KEY, : process.env.TWITTER_CONSUMER_SECRET, : ----->YOUR_USER_ACCESS_TOKEN_THAT_YOU_JUST_ACQUIRED, : ----->YOUR_USER_ACCESS_TOKEN_SECRET_THAT_YOU_JUST_ACQUIRED, }, : }); url ` /account_activity/all/ /subscriptions.json` ${TWITTER_API_URL} ${TWITTER_WEBHOOK_ENV} oauth consumer_key consumer_secret token token_secret json true Once your user is subscribed, your webhook function will receive a POST request every time a new event occurs. Limitation When using the free tier of Twitter API, there are many limitations. Here are a few that directly relates to webhooks: You can only have 1 development environment for subscription. Within that environment, you can only have 15 subscriptions. That means if you app wants to subscribes to many users (use case 2), you can only have 15 users on the free tier. CLI I'm actively working on to smoothen this process of registering webhook and managing subscriptions. Stay tuned for updates, and I'd love your collaboration if you want to get involved. an open source command line tool Resources Twitter API Documentation : The Twitter dev team created a sample dashboard for you to handle this process. This is super helpful, and I'd highly recommend you check this out. This project was how I got started with Twitter webhook. Twitter Sample Webhook on GitHub Thank you for checking out my guide on Twitter webhook. Don't hesitate to ping if you run into any problem, and please let me know when you build something from this. I look forward to seeing all the cool stuff you build on top of Twitter. me
