Some time ago, I encountered an issue when 10 million messages with emoji were written in the MySQL table with utf8 encoding. For those who don’t know: you should use the ut8mb4 encoding in MySQL to support emoji If it were a small table, I would alter it with the conversion of columns to utf8mb4 encoding. But the table contains hundreds of millions of rows in many shards, so it’s very hard to alter it in the production runtime without the system degradation. So the fast solution was to prevent the insertion of messages in a database on the back-end level. The service back-end is written in Golang. In this article, I’ll explain a few solutions to work with emojis in Go: the good way and the bad way. Not So Good Way 🙅‍♂️ Some libraries and forums suggest using regexp to find emojis in a text: package main

import (
	"fmt"
	"regexp"
)

func main() {
	var emojiRx = regexp.MustCompile(`[\x{1F600}-\x{1F6FF}|[\x{2600}-\x{26FF}]`)

	fmt.Println(emojiRx.MatchString("Message with emoji 😆😛")) // true
} It looks like it works, but some disadvantages make us don’t use the way: it detects NOT all emojis: fmt.Println(emojiRx.MatchString("Message with emoji 👏")) // false the regexp isn’t readable [\x{1F600}-\x{1F6FF}|[\x{2600}-\x{26FF}] regular expressions have poor performance, so it’s inappropriate for high-load apps Good Solution 🔥 The best way is to have a storage with all emojis and use it when you need to detect an emoji in a text. That’s how the works. GoMoji library Firstly let’s add the package to our project: go get -u github.com/forPelevin/gomoji Now it’s pretty simple to check whether a string contains emoji: package main

import (
	"fmt"

	"github.com/forPelevin/gomoji"
)

func main() {
	fmt.Println(gomoji.ContainsEmoji("Message with emoji 👏")) // true
} GoMoji Internals 🧐 Let’s deep dive into the library internals and figure out how it works. Firstly, look into the function: ContainsEmoji // ContainsEmoji checks whether given string contains emoji or not. It uses local emoji list as provider.
func ContainsEmoji(s string) bool {
	for _, r := range s {
		if _, ok := emojiMap[r]; ok {
			return true
		}
	}

	return false
} We see the lib iterates through string’s runes and checks whether it’s an emoji or not via . So the complexity of the function is , where is runes count. emojiMap O(N) N But what is ? It’s a map of Emoji models by their hex code: emojiMap // Emoji is an entity that represents comprehensive emoji info.
type Emoji struct {
	Slug        string `json:"slug"`
	Character   string `json:"character"`
	UnicodeName string `json:"unicode_name"`
	CodePoint   string `json:"code_point"`
	Group       string `json:"group"`
	SubGroup    string `json:"sub_group"`
} // Code generated by generator.go ; DO NOT EDIT.

package gomoji

var (
	emojiMap = map[int32]Emoji{

		42: {
			Slug:        "keycap",
			Character:   "*️⃣",
			UnicodeName: "keycap: *",
			CodePoint:   "002A FE0F 20E3",
			Group:       "symbols",
			SubGroup:    "keycap",
		},
        ...
    }
) So the pre-generated map gives us some advantages: it contains all existed emojis it returns an emoji by a hex code in which is an outstanding performance. 0(1) There are some benchmarks: BenchmarkContainsEmojiParallel-8   	94079461	        13.1 ns/op	       0 B/op	       0 allocs/op
BenchmarkContainsEmoji-8           	23728635	        49.8 ns/op	       0 B/op	       0 allocs/op
BenchmarkFindAllParallel-8         	10220854	       115 ns/op	     288 B/op	       2 allocs/op
BenchmarkFindAll-8                 	 4023626	       294 ns/op	     288 B/op	       2 allocs/op The reasonable question is where it takes all emojis data? An answer to the question is in the . It contains the CLI app that fetches all emojis from and saves them in the in format. It allows the lib to keep emojis up to date. generator.go file OpenAPI Emoji data.go file emojiMap map[int32]Emoji Conclusion 💡 As software engineers, we encounter problems every day. The best solution isn’t always the first that comes to mind or is founded in stackoverflow. So if you are stuck with emojis, consider using a simple and useful . It can help you not only validate texts but make great features in your chat app. GoMoji lib

Too Long; Didn't Read

Making security fun one episode at a time

How to Detect and Delete Emojis in Golang

Too Long; Didn't Read

Vladislav Gukasov

About Author

TOPICS

THIS ARTICLE WAS FEATURED IN...

How to Detect and Delete Emojis in Golang

Too Long; Didn't Read

Vladislav Gukasov

About Author

TOPICS

THIS ARTICLE WAS FEATURED IN...

RELATED STORIES