[Minimizing Startup Costs] We Built our Own Email Marketing App in 4 Weeks

Written by jaschaio | Published 2019/10/08
Tech Story Tags: firebase | serverless | email-marketing | javascript | nodejs | cloud | latest-tech-stories | hackernoon-top-story

TLDR At the end of last year email marketing had become by far the largest recurring expense in our business. We were spending 10x as much on Emails than on anything else. The worst was that costs grew at the same rate we were growing: Adding about 10,000 new email subscribers each month meant that even if we regularly removed no longer engaged subscribers from our list, the amount we were spending was still increasing way too fast to be healthy. We have switched email marketing providers three times and out of experience I can tell you it's always a major pain.via the TL;DR App

Thanks to Hiten Shah who generously reviewed an early draft of this article.
Some day at the end of last year I stumbled over a post by Justin Jackson about reducing startup costs.
And it made me think hard about our number one expense:

Email Marketing Software

Over the past few years me and my wife have build a brand new Blog from zero to almost 150.000 email subscribers.
And by the end of last year email marketing had become by far the largest recurring expense in our business.
We were spending 10x as much on Emails than on anything else.
The worst was that costs grew at the same rate we were growing:
Adding about 10.000 new email subscribers each month meant that even if we regularly removed no longer engaged subscribers from our list, the amount we were spending was still increasing way too fast to be healthy.
Especially taking in mind that our revenue doesn’t necessarily grows as fast as your email list.
Our main target market are Spanish speaking countries and only a small number of email subscribers actually end up producing revenue. And the typical revenue per customers is not as high as in other economies typical software prices are calculated on.
So this is what made me thinking:

What would I have to do in order to cut this expense?

Unfortunately the decision was taken from me almost right after new years eve.
I got an email from our Email Marketing Provider that they were increasing prices on a 12 days notice:
The worst thing was that they didn’t even mention how much prices would be increasing.
Instead they decided to promote switching to yearly pricing.
Which meant we were practically pressured into forking over almost 5 figures on a two weeks notice as "the only way" to keep our current pricing.
After some investigating and seeing an outburst of other loyal customers we found out that prices would be at least doubling.
What I didn’t know at that time was that while prices were increasing, sending limits were actually decreased. Meaning that we would end up paying more for sending considerable less emails.
In the end it came down to an almost 300% price increase just for us.
In concrete numbers it would have meant about 1,600$ per month.
And just as Justin Jackson says in his Article:
Which solopreneur couldn’t use an additional 20,000$ per year of guaranteed profit?

Why we didn't move to a different provider and decided to build our own App instead

Over the past few years we have switched email marketing providers three times and out of experience I can tell you it's always a major pain.
I loved our previous provider and their functionality but I was seriously disappointed by how they treated us and other customers.
They had just completely lost my trust.
And while Email Marketing is our number one expense its as well the most important part of our content based business.
It's something we literally can't live without.
After such a devastating experience I just didn't feel comfortable giving up again complete control over such an important part of our business to yet another company. 
Email Deliverability itself was another important consideration.
With most other Email Marketing Providers you have no control over which emails get sended out together with yours. According to Mailchimp keeping delivery rates high is actually one of the major challenges.
By building our own solution it was fairly simple to have dedicated servers for sending our own emails and have real time information about how exactly we are doing:
We could even do things like separate email sending by highly engaged subscribers and lower engaged subscribers to make sure that the ones who matter most are guaranteed to receive our emails.
Beside that it was a simple economical calculation:
At 1,600$ per month and a 10% increase every other month we would be spending nearly 300,000$ over the next five years:
(Taking in mind that we've already spent close to 30,000$ in the past two years)
If it took me just a month of my own time to build a viable solution then we would have a positive ROI within the first year.
And that would only compound over the next five years and expected lifetime of the App.
Having complete control over features and building something truly unique to our advanced needs (more about that in a minute) was just an added bonus.
Additionally:
It was an exciting challenge and real world project that would immediately benefit us where I could apply everything I have learned about building software over the past years.
So for us this just felt like the right decision, but it probably isn't for a lot of other businesses with lower email volume and needs.

So here is how

I knew right away that I had to limit the tools I used and cut some corners where possible to fit into the tight budget of spending just a single month on this project.
I had to concentrate on the most necessary and yet powerful features only and make use of well established supporting services in order to develop this in such a short amount of time.
Luckily picking essential features was an easy call as I have worked extensively with all mayor email marketing apps and their APIs building integrations to my products over the past few years.
This way I knew exactly what I wanted out of an email marketing app.
I had as well already a huge list with little and bigger annoyances of existing solutions that I could improve on right away.
Sending the weekly newsletters was actually the simplest thing we could have used even an existing open source solution for. (Developed by freecodecamp which found themselves in a similar situation)
What was way harder to do and incredible more important to us was the ability for extensive personalization features through all parts of the App.
For example building visual workflows which use dynamically passed in variables at subscribers optin time, sends different type of emails and does different types of actions at different times in a personalized way for each subscriber:
(This is an actual screenshot from the finished App – Its a single workflow we use for double optins and we previously had 100 almost identical copies off just to handle different tags and downloads)
Luckily technology wise I could start from a blank slate.
The only requirements were the list of features I had in mind, but I wasn't tied to any legacy code or specific platform.
This meant I could develop everything with state of the art technology like server-less cloud functions or an API first design.
It would scale as needed without the need of providing extra infrastucture or running into typical scaling issues, no matter if the App only had a single big customer (ourselves), or hundreds or even thousands of active customers of varying sizes.

The best possible time to be an indiehacker

What I found curious about this whole project is that just a few years ago you would have needed tens of thousands of dollars to invest, a small team and a very looong time to build something remotely similar.
Nowadays I was able to pull this off all by myself within just 4 weeks.
This is partly thanks to the amazing work of other people who came before us and advanced software and services which weren't available just a few years ago.
There has never been a better time for any indiehacker to build and launch their own projects.
This is really the spirit of all of it: We are able to pull things off that can compete with well funded, big teamed startups.
So for what its worth here is the complete Tech Stack I used:

Firebase Firestore

I knew first hand that I didn't want to handle running a highly available database with hundreds of thousands or even millions of rows of data added per month that gets written to in a very high burst fashion.
Firebase Firestore is exactly made to handle this completely hands off for you.
Beside that it offers some of the best developer experience I have ever seen.
They got SDKs for every major platform.
Everything is throughly documented.
The community is big and helpful.
The backend UI is state of the art and usable to edit things you might not have build an UI yourself for yet.
And one of the greatest things is that it integrates almost seamlessly with other google cloud platform services which came in very handy developing other parts of the App. (You will see what I mean further down this article)
It's pricing is exactly what I was looking for:
It scales indefinitely with you but is basically free to get started with.
It supports a wide range of different needs, from this App I've been building or even communities like indie hacker (which runs on top of it as well).
There are certainly other (both more open or expensive) options if you don't want to handle database servers yourself.
But I just love Firebase.
It certainly has some mayor down sides you should familiarize yourself with before going all in, but once you know what you are up against you can easily navigate around them. (Let me know in the comments if you would like a more extensive post about the pros and cons of Firebase)

UI Framework

Within just 4 weeks there was no way that I was going to design and write my own UI. So I wanted to give a mayor UI framework a try instead.
It has been years since the last time I really worked with any of the available ones, and even then the only true experience I had with was Zurb Foundation – mainly for their JavaScript parts and CSS grid classes in pre-flexbox times.
As I was more looking for ready made UI components I could just plugin and play I decided go for Bootstrap which turned out as a very good choice:
Almost all of the visible part of the UI was made out of the ready made components Bootstrap provides, with a couple of small adjustments here and there.
I gave googles material UI a quick try as well but found it way too difficult to get started with.
There is definitely a higher learning curve with Material Design.
Especially as it seems to be more tightly coupled with the JavaScript framework you use and particular focused on mayor frontend frameworks like Angular. (Something I didn't want to work with and will tell you why in just a second)
Bootstrap was quicker and easier to start with and turned out truly excellent.
All in all I only wrote about 500 lines of CSS myself for this project which is probably a record as I can't remember anything I've worked on in the past few years that had so little CSS. (Of course not factoring in the CSS provided by Bootstrap itself).

Javascript Frontend Framework

Nowadays the default choice seems to be React or Angular and people can get pretty religious about it.
While I certainly love working with React I've become increasingly vary of its usage for small projects like this.
It just feels to me that what might be a great choice for a billion dollar company is not necessarily a great choice for indie hackers.
One thing that happened to me before is that React and its entire Eco System is just moving way too fast. You don't touch a project for a couple of months and then when you try to update its dependencies you need days just in fixing breaking changes.
I wanted to make sure that the underlying framework I used for this was still relevant and easy to use two to three years from now as I was unsure how frequent I will do updates.
Its a very small framework based on pure web standards that is incredible fast and performant.
If you have used any other modern javascript framework before you will probably find it really easy to use as well.
Its simplicity makes sure that it won't be out of date a few months down the line, and beside that it encourages you to not bloat your code.
Instead of installing yet another dependency for anything you are more likely to develop smaller, simple stuff you actually understand yourself using just vanilla javascript.
And if you are not used to writing plain javascript from scratch, than you can find help within one of the excellent JS communities like Go Make Things from Chris Ferdinandi. (Who actually helped contributing a small method to this project)
The App only had around 6 to 7 different views I had to build which a lot of repeating components and it was really easy to do this with HyperHTML in combination with Bootstrap and some utility libraries like momentjs.
Using HyperHTML was mainly my choice because I know it really well by now and have come to absolutely love it.
In the end it doesn't really matter what you use to build you app and instead of worrying about that, you should just choose whatever you feel most comfortable with.

API Design

I've never designed a Rest API for myself but have extensive experience working with all kind of different APIs.
Therefore I thought that it shouldn't be too had to build some simple API endpoints together with Node.js and Express.
Well... unfortunately that wasn't entirely true.
I seriously tried building a Rest API but with my lack of experience in the area it just wasn't feasible to get it done properly within such a short amount of time.
Building a proper Rest API turns to be out way harder than you might think. (Which is probably the reason why there are so many low quality APIs out there…)
Fortunately I wasn't really forced to build a rest API but could use whatever worked best for me in this particular situation.
And for this project using GraphQL turned out to be the better choice.
It definitely has a learning curve but if you have never designed an API before its probably way faster than building your own Rest Endpoints from scratch.
Beside that its declarative approach encourages a lots of best practices you would need to make sure to think of yourself when building a REST API instead.
I know that I am might contradicting my thoughts about the JS Framework going with an almost "overhyped" choice backed by a huge company. But if you are doing something for the first time the size of the community actually matters as there are excellent resources available to learn from and ask questions.

Serverless Backend

For the API and as well all Cron Jobs I used Google Cloud Functions / Firebase Functions.
The reason is simple:
They allowed me to easily trigger things whenever something in the Firebase Database happened.
And they would scale indefinitely with the needs of the App without me ever having to worry about the servers that run the whole thing.
Using the same language (JavaScript) across the whole application stack makes your life a lot easier as well. Coming from a PHP Background I nowadays actually prefer JavaScript above all else as I can share code between the Frontend and Backend whereever I see fit.
Pricing wise you got to be really careful with the things you put within your functions as economies of scale can really hurt you here.
Simple example:
In the beginning I had a couple of functions that took 20s of CPU time at peak hours (because it was waiting for a response from another Rest API). Being executed millions of times this gets VERY expensive VERY fast because you are mainly billed for the CPU time you use.
Optimizing just a single function to take 500 miliseconds instead 20 seconds makes a 4000% difference across millions of executions.

Aggregations and Segmentations

The bad thing about building an email marketing app is that one of the most important parts of your application is actually running aggregations and segmenting your data.
And unfortunately this is exactly the thing Firebase Firestore (or any document based database for that matter) is not great in.
It really depends on what you are building if the aggregations and segmentation features of your database are sufficient, but for this App I unfortunately had to add something especially made for that to the Mix:

Elasticsearch

Elasticearch is basically a search engine for your database that is incredible fast in both searching, indexing and aggregating.
Unfortunately its as well the only part of the application that needs to run on servers. (I am using a couple of powerful Digitalocean VPS for this)
Even if I initially didn't want to touch any servers for this app, the reason I still decided to do this is because its one of the most incredible and robust pieces of software I have ever come across.
Building a highly available cluster of servers is incredible easy (if you have at least some experience setting up and running servers on your own) and once its up its just mind blowing how smoothly it runs.
I have yet to find something I can't do with Elasticsearch. (And I found many things I couldn't do easily with Firebase)
It is a mature software with a huge community and throughly documentation which is just plain fun to work with.

Email Sending

I have worked with another transactional email provider before called Mandrill (made by MailChimp) – unfortunately since a while now its for MailChimp Customers only.
So if you want to send a LOT of emails you basically are left with two choices:
Amazon SES or Sendgrid.
And while Sendgrid looks and feels like the sexier option, its actually as well a lot more expensive if you don't reach HUGE volumes that might warrant a special deal.
So we went with Amazon SES as it has essentially the same features and is pretty affordable right away.
Building on top of it was mostly smooth. It takes some time to set up properly but then you get real time notifications for sending, delivery, bounces, opens and click rates build right in – some information we weren't even able to get before.
The sending limits are pretty generous and we yet have to reach the daily quota:
As a matter of fact it was quite a challenge to actually reach our maximum send rate of 200 emails per second. (Which is 12.000 per minute or an email to 100.000 subscribers within roughly 8 minutes).
Which brings me to the last point:

Challanges I didn't think of

All in all building this went fairly smooth once I figured out what was and wasn't possible with Firebase.
We transitioned quite abrupt on beginning of Feburary to the new system without any mayor bugs or downtimes.
The biggest difficulty in building this was actually handling such a high amount of data produced in relatively short amount of time.
As a simple example imagine that you send an email to 70.000 subscribers.
For each subscriber you have to personalize each email. Not just the delivery name and address but as well the content and links within the email.
Then each sended email produces quite a few number of "notifications" for sending, delivery, rejections, bounces, opens, clicks and so forth. Each of those notifications needs to be handled and acted upon, changing a single piece of data within your database or triggering complicated, stacked automations in other parts of the App.
So a single "broadcasts" produces roughly 300.000 new pieces of information you need to handle within just a few minutes. Then there are of course other times where almost no emails are sended and not much happens:
Sending Emails based on the subscribers time zone helps mitigating this (so that not all emails are sended at exactly the same time).
Pipelining your events through something like Logstash and handling them at a slightly slower pace your database can easily handle is another solution.
An additional challenge was making sure that emails are not sended twice to the same subscriber. With multiple independent workers sending a lot of emails in a short amount of time this was easier said then done.
UI Wise the most difficult task was building the visual workflows. They have various degrees of recursion and a lot of edge cases that were hard to build and debug. Recursion is definitely a topic hard to wrap your head around and even harder to master.

Conclusion

I wrote this post because a lot of times while building this I would have loved the possibility to reach out to somebody who has done something similar before and ask them questions.
I always imagine this fairy tale office world where you work in an amazing team and got experts you can turn to for everything.
Indiehackers on the other side have to be generalists. We usually work all by ourselves and I think sharing our experiences, difficulties and thought processes with each other in communities like this one can help tremendously.
Of course for the moment I only build and launched this as a tool we can use internally.
If I would have had to build signup flows, customer on-boarding features, billing processes, a marketing website and the millions of things I haven't thought of it would have easily taken me double or triple the time.
I still think that it was one of the most interesting challenge I've ever done that trained me to build and launch something with both limited resources and time. This is exactly the situation most of us find themselves in all the time why trying to build and launch our side projects or businesses.
I hope you enjoyed the journey and let me know in the comments if there is a topic you would like to expand me upon.
– Jascha Brinkmann

Written by jaschaio | Indie Hacker
Published by HackerNoon on 2019/10/08