Writing Messenger bot in Java with Spring Framework


Previously I wrote about setting Messenger bot environment on own VPS. Today I’m writing about setting basic bot with Java and Spring Framework. To do so I’m needed:

Aftere setting up basic Facebook page and creation of Facebook app, I had  to select Messenger product, which will be used with my application. After that there is no much to do. Wee need to get:

  • Page access token, which is needed to send message
  • Set webhook for Messenger to receive messages
  • Set submissions for: messaging and messaging subscriptions

First submission is neeed to reply on messages, which is possible only up to 24h after receiving one. Second submission is needed to reply after 24h and reply periodically. But let’s do some coding first.

I need web controller for receiving messages

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
@Controller
@RequestMapping("/webhook")
public class WebhookController {
    private final static String TOKEN = "SUPER_SECRET_TOKEN";
 
    @GetMapping
    public @ResponseBody String getHook(
            @RequestParam("hub.mode") String mode,
            @RequestParam("hub.verify_token") String token,
            @RequestParam("hub.challenge") String challenge,
            HttpServletResponse response) {
        if (TOKEN.equals(token) && "subscribe".equals(mode)) {
            response.setStatus(HttpServletResponse.SC_OK);
            return challenge;
        } else {
            response.setStatus(HttpServletResponse.SC_FORBIDDEN);
            return "";
        }
    }
}

Main class

1
2
3
4
5
6
7
8
@SpringBootApplication
public class Messenger {
 
    public static void main(String... args) throws Exception {
        SpringApplication.run(Messenger.class, args);
    }
 
}

And application.properties file in resources directory

1
2
3
4
5
6
7
server.port=4321
server.ssl.key-store=classpath:keystore.jks
server.ssl.keyStoreType=JKS
server.ssl.key-store-password=secret
server.ssl.key-password=secret
server.ssl.keyAlias=tomcat
logging.level.root=DEBUG

After building project and running it on VPS, comes time to set webhook in Facebook app configuration

And create page token to return messages

Now when hook is ready, there is time to create code for receiving messages in WebhookControlelr class

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
@PostMapping
    public @ResponseBody
    String onEvent(@RequestBody Event event, HttpServletResponse response) {
        if ("page".equals(event.getObject())) {
            messageFromEvent(event).ifPresent(m -> 
                messengerController.sendMessage(Message.of(m.getSender().getId(), reverse(m.getMessage().getText()))));
            response.setStatus(HttpStatus.OK.value());
            return "EVENT_RECEIVED";
        }
 
        response.setStatus(HttpStatus.NOT_FOUND.value());
        return "";
    }
 
 
    private static Optional<Message> messageFromEvent(Event event) {
        Message message = null;
        List<Entry> entry = event.getEntry();
 
        if (isNonEmpty(entry)) {
            List<Message> messaging = entry.get(0).getMessaging();
            if (isNonEmpty(messaging)) {
                message = messaging.get(0);
            }
        }
 
        return Optional.of(message);
    }
 
    private static boolean isNonEmpty(List list) {
        return list != null && list.size() > 0;
    }
 
    private static String reverse(String in)  {
        byte[] buf = in.getBytes();
        int pivot = buf.length / 2;
        int len = buf.length - 1;
        for (int i = 0; i < pivot; i++) {
            buf[i] ^= buf[len-i];
            buf[len-i] ^= buf[i];
            buf[i] ^= buf[len-i];
        }
        return new String(buf);
    }

Well, that escalated quickly. Let me explain code above. Facebook sends POST request to mine webhook endpoint, which is deserialized as an Event class object, which JSON representation looks like that

1
{"object":"page","entry":[{"id":"1774684282840000","time":1515003458200,"messaging":[{"sender":{"id":"6515631598500000"},"recipient":{"id":"17746742828000"},"timestamp":1515003457419,"message":{"mid":"mid.$cAAYU8xm2sYBm7oN3i1gvT4v","seq":1123008,"text":"hello bot"}}]}]}]

I needed to create POJO classes to handle all parts of message. Here what’s interesting for us, is sender->id and message->text. Rest can be easily ignored. Here I used messengerController bean to handle reply containing reversed message (test on ASCII chars only plx or it fails :<). There is also a contract with Facebook, that after properly receiving message, application should return HTTP status 200. To send message back to Facebook I used retrofit2 library again. Simple message must only contain recipient and text components.

1
{"recipient":{"id":"17746742828000"},"message":{"text":"hello human"}}

Message sender code looks like below

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
public interface MessengerService {
    @POST("/v2.6/me/messages")
    Call<Map<String, String>> sendMessage(@Query("access_token") String token, @Body Message message);
}
 
@Component
public class MessengerController {
    private final static String TOKEN = "PAGE_TOKEN_HERE";
    private final static Retrofit retrofit = new Retrofit.Builder()
            .baseUrl("https://graph.facebook.com/")
            .client(new OkHttpClient())
            .addConverterFactory(JacksonConverterFactory.create())
            .build();
 
    public void sendMessage(Message message) {
        MessengerService service = retrofit.create(MessengerService.class);
 
        try {
            service.sendMessage(TOKEN, message).execute();
        } catch (IOException e) {
            e.printStackTrace();
        }
    }
}

And that’s it. You can test your bot, by writing private message to facebook page connected with bot application as long as you are page / app admin. After that, when everything works properly, you apply for review and describe your bot commands. Later on one of Facebook non technical (!!!) employees will send described commands and some random ones, and if everything will work properly, your bot will be accepted and available for public.

Full project is available at https://github.com/felix-catus/simple-messenger-bot
Please put your keystore in resources directory. If you don’t have one yet, you can learn how to get it here.

Checking crossfit workout booking availability with jspoon and retrofit2


There are many pros associates with new technologies. You can easily chat with friends without leaving your bed, order things online and get them brought to your place without leaving home and even get pizza and pay for it, before it gets delivered. One of those super fancy features is possibility to register for sports activities. This way you can be sure, that there would be place waiting for you, but it also gives you convenient way to change your plans and change your booking activity. But there are also days, when your favorite workout is fully booked and you’re constantly refreshing the page waiting for someone to change their plans. This might be frustrating and it has happened to me many times. That’s why I decided to write a simple application, which will scrape booking page and notify me, when there will be place available for my favorite workout.

At first I wanted to use python with simple regex solution (you really shouldn’t use regular expressions to parse HTML. It will work for simple tag collection, but it turns out that many web pages are not so regular, as regular expressions are), but then I thought:
– hey, maybe it is possible to deserialize HTML to POJO, just like you deserialize XML. And guess what? Someone already did it, and there is already ready solution for webscraping. You just need to annotate your POJO fields with proper CSS selectors and voilĂ , any webpage you read, can be transformed to java objects. It’s called jspoon and uses jsoup to parse HTML code. Principle of operation is similar to jackson library.

There is also another interesting library I used in my little project. It’s called retrofit and allows you to create fast and simple HTTP client for any API. It uses OkHttp as a HTTP client and has jspoon dedicated convertor, which makes it perfect solution for our problem.

So, let’s do the job. At first we need to determine the structure of page, we want to scrape. We’re good, it turns out to be simple HTML table

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
<tr>
    <td class="hour">
        18:30
    </td>
 
    <td>
        <div style="width:100%;float:left">
            <div class="event" meta:id="10894364" style="color:#050505;background-color:#6feb1d;">
                <span class="eventlength">60 min</span>
                <span class="availability">
 
                    <span class="availability-number">0</span> wolnych</span>
                <p class="event_name">WOD Gymnastics</p>
                <p class="instructor">Artur</p>
                <p class="room"></p>
            </div>
        </div>
    </td>
</tr>

This is actually one workout from page containing every workout for certain day. So we need to actually only get two parameters: an hour and availibility number. But we’ll get also workout name, just to make this solution output clearer data. Our POJO code should look like following class

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
public class Workout {
    @Selector("td.hour")
    String hour;
 
    @Selector("p.event_name")
    String name;
 
    @Selector("span.availability-number")
    Integer available;
 
    public String getHour() {
        return hour;
    }
 
    public String getName() {
        return name;
    }
 
    public Integer getAvailable() {
        return available;
    }
 
    @Override
    public String toString() {
        return "Workout{" +
                "hour=" + hour +
                ", name='" + name + '\'' +
                ", available=" + available +
                '}';
    }
}

We mapped every needed property onto java POJO fields, but now we need to make a collection container, to read all workouts from page. So we do with following code:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
public class WorkoutDay {
    @Selector("table.calendar_table_day tbody tr")
    List<Workout> workouts;
 
    public List<Workout> getWorkouts() {
        return workouts;
    }
 
    @Override
    public String toString() {
        return "WorkoutDay{" +
                "workouts=" + workouts +
                '}';
    }
}

The catch here is that we don’t point onto workout container, which in our case is table > tbody, but we must indicate CSS selector for workout field, which in our case is table > tbody > tr. Because every table row is mapped to single workout entry.

Having above classes we could easily use simple jspoon invocation and deserialize our workout entries:

1
2
3
Jspoon jspoon = Jspoon.create();
HtmlAdapter<WorkoutDay> htmlAdapter = jspoon.adapter(WorkoutDay.class);
WorkoutDay day = htmlAdapter.fromHtml(htmlContent);

But we’re getting a little creative here and we’ll use retrofit library to get the page. To do so, let’s create our API service interface with properly annotated method:

1
2
3
4
public interface WorkoutService {
    @GET("/kalendarz-zajec?view=DayByHour")
    Call<WorkoutDay> getDay(@Query("day") String date);
}

We declared HTTP GET operation and URL path for getting our workout entries. There is also dynamic query parameter day which is part of URL query and is being set from method parameter. With such interface, it is time for controller code:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
public class WorkoutController {
    private final Retrofit retrofit = new Retrofit.Builder()
            .baseUrl("https://cf-krakow.cms.efitness.com.pl/")
            .addConverterFactory(JspoonConverterFactory.create())
            .build();
    private final DateTimeFormatter formatter = DateTimeFormatter.ofPattern("dd-MM-uuuu");
 
    public WorkoutDay getDay(LocalDateTime date) throws IOException {
        WorkoutService service = retrofit.create(WorkoutService.class);
        Response<WorkoutDay> response =  service.getDay(date.format(formatter)).execute();
 
        if (response.isSuccessful()) {
            return response.body();
        } else {
            throw new RuntimeException("Something went wrong: " + response.errorBody());
        }
    }
}

As you can see, with builder pattern we’re setting retrofit engine with proper URL and data converter. And later, we use it to create API service and make the call. As simple as several lines of java code. Now let’s use streams to enhance Workout class functionality and return workout for specified hour

1
2
3
    public Optional<Workout> getWorkoutByHour(String hour) {
        return workouts.parallelStream().filter(w -> w.getHour().equals(hour)).findFirst();
    }

and maybe create entry point class, to read user params and invoke controller code

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
public class CfNotify {
    private static final WorkoutController controller = new WorkoutController();
    private static final DateTimeFormatter formatter = DateTimeFormatter.ofPattern("HH:mm");
 
    public static void main(String... args) {
        LocalDateTime date = parseArgs(args);
 
        try {
            WorkoutDay day = controller.getDay(date);
 
            Workout workout =
                    day.getWorkoutByHour(date.format(formatter)).orElseThrow(
                            () -> new RuntimeException("No such workout"));
            System.out.println(workout);
        } catch (Exception e) {
            e.printStackTrace();
        }
    }
 
    private static LocalDateTime parseArgs(String... args) {
        if (args.length < 1) {
            throw new RuntimeException("Please execute with date parameter in following scheme 2011-12-03T10:15");
        } else {
            return LocalDateTime.from(ISO_LOCAL_DATE_TIME.parse(args[0]));
        }
    }
}

Above code altogether with previous classes will return workout for ISO date time format, which is smth like 2011-12-03T10:15 and will output similar to:

Workout{hour=16:00, name='WOD Beginners', available=2}

Of course you can change the code to output only availability and if it is bigger than 0 notify you in any choosen way. But this is outside of the scope of this post and will be described next time, when I show how to connect above mechanism to Facebook Messenger bot to set hook and get notified when again there is an possibility to register for the workout.

Whole solution can be downloaded / cloned from GitHub repo https://github.com/felix-catus/CfNotify