Now, these jobs are not core to the product but are necessary for a business. Ask yourself, if this is a good investment of your developer's time and energy in building these workflows and Cron jobs.

Wouldn't it be awesome if someone else can do these tasks so the team can focus on the core product?

Introducing you to Microbots, where you can make hundreds of workflows on your own database without writing a single line of code. Imagine the time and energy that you are saving!

Here are a few examples of these workflows could be:

1. Sending user communication via Email, SMS, Whatsapp, etc. using your own data
2. Periodical deletion or upload of data in the database

... and many more!

Let us discuss each of the mentioned workflows in detail if these have to run from your own database.

User Communication

Communicating with your users ensures better retention. Now, that does not mean spamming your users with marketing chants but personalized communication every now and then. This could be as simple as "We think creating Heatmaps will be useful for you but I don't think you are using it now. Why don't you go here and make one!".  

How are you doing it now?

1. Asking your developers to write Cron jobs for sending weekly reports, payment reminder emails, free trial expiration, etc?
2. Using a 3rd party tool and paying money per subscriber for replicating the data that you already have?  

Whether you are doing it via Method 1 or Method 2, you end up wasting money, resources, or both.

How you can do it with Microbots?

Microbots give you a simple UI to build these communication flows. The steps are pretty simple and can be done by Microbots interface without needing any developer's effort from your team.

The steps are simple:

1. Connect with your database and Email Service API Provider
2. Create a segment or set up rules to identify users
3. Choose a schedule or frequency at which you would like to run the communication flow.  

Delete or Upload data periodically

Every business has data dumps that they want to periodically delete from their own database and also sometimes they want to upload it to an S3 bucket or Google Cloud Storage, etc. Each and every team is supposed to write such Cron jobs that are tedious, bulky, and repetitive.  

How are you doing it now?

Currently, someone from the team is responsible for writing as well as maintaining these Cron jobs. The code becomes bulky over time and it is hard to maintain if the person responsible for this moves from the company.

It should not be so hard, right? I think most of the developers would agree with me on this.

How can you do it via Microbots?

Again, Microbots give you a simple UI to build, maintain and schedule these Cron jobs saving your developers time and effort.

The steps are simple:

1. Connect with your database
2. Build a query for data that you want to delete or upload
3. If upload, connect to an S3 bucket, etc
4. Choose a schedule at which you wish to upload or delete your data

Do you have use cases where you want to do certain Database operations and do not want to write code for it? Drop them in our comment box and Microbots will make them happen!

SaaS products, your users are going to be in a long-term relationship with you and not just a fling! To build a relationship, you need to communicate. You need to understand their needs as well as you need to clearly communicate what is in there for them.

And, yes, this long-term relationship would mean recurring revenue or subscription money for your startup.

I am Microbots and I will tell you how to set up an effective series of communication with your users! For SaaS customers, emails are the best channel to reach out. So, this article will be focussed more on emails.

Type of Emails:

1. Transactional Emails
2. Onboarding Emails
3. Drip/Marketing Emails
4. Feedback Emails

Transactional Emails

Transactional emails are the key emails that are sent to a user on a specific event like:

1. Personalized Discount Emails
2. Payment Emails
3. Free Trial Expiry Emails

… and many more.

These emails should be done on pure transactions and should not be associated with your marketing, onboarding, or feedback emails.

As these are the key emails that are supposed to reach users, you probably should not have an unsubscribe link on these emails.

Let’s see how you can set up Free Trial Expiry Emails with Microbots instead of investing the time and energy in writing Cron jobs for the same. Also, not only do you send emails but you also get a cool analytics dashboard for your transactional emails if you send them via Microbots!

Step 1: Connect Your Database

The first step is to connect your database with your Microbots account. Choose the required Database template like Mongo, MySQL, Postgre SQL, or many more.

Once you select the template, you need to fill in the required information for Microbots to connect with your database

As soon as, you connect your DB, Microbots will parse all the tables and their fields.

You can also add descriptions to the fields in the tables so that everyone on your team can understand the field types.

Step 2: Connect Your Email Service Provider

Once you have connected the database, you also need to connect with an Email Service Provider like Amazon SES, Sendgrid, Netcore, or Mandrill (now part of Mailchimp). You can just choose the template and fill in a couple of details to connect with your ESP

Step 3: Create a Segment

Once you have established the desired connections, you now need to create a segment of users whose free trial is about to be expired. Let's say your product offers a week's free trial. So, you can choose the range for user creation in the last week. Also, you can add as many rules as you want to refine this segment.

Step 4: Select or Insert Email Template

You can choose and modify an Email from Microbots' existing templates or you can add your own.

Step 5: Choose a Run Schedule

The last step is to assign a frequency at which you need to run these emails. Since our example free trial is for a week, we chose the segment of users who signed up on the date i.e. greater than or equal to the last 7 days and less than the last 8 days. We can run this automation once a day and this will send free trial expiry email to our users whose trial is about to end.

This is just one small example of a transactional email that you need to set up as soon as you launch your payment flow. You need to send signup emails, reminders before the free trial ends, payment reminders, plan expiry, discount codes, etc.

Hope this is helpful! Follow our medium channel to read about how to create a complete flow for your onboarding, marketing, and feedback emails without writing any code.

Looks pretty simple, so we set out to find all the options to make this happen.

Airbyte

The first option that we checked out was Airbyte.  At that point, for the MySql version in source, it did not support incremental sync but only incremental appends and updates but it sufficed for our use case. In order to learn the difference between these modes of syncing you may check the sections below.

Airbyte was the fastest option to try out and setup. It performed a fast initial sync but beyond that the updates started to fail a lot. Sometimes they were due to lost connection, and sometimes for unknown reasons. Also all the failed updates caused some residual data to be left in the destination database which quickly caused the disk to fill up. I believe Airbyte was still in a beta phase and all these processes were not very clean at that point. The team was supportive and I tried various methods but eventually, we moved to other solutions.

pt-table-sync

It's a tool in Percona toolkit which is very highly regarded. I learnt that for replication it needs the binlog format to be STATEMENT, otherwise it will try to set it via SUPER privilege for the session.  Unfortunately, we did not have the right permissions for the setup and therefore this was not an option.

ZongJi

Finally, we moved on to cooking up our own solution using ZongJi, which is a simple binlog reader for MySql.  It requires binlog format to be ROW, which was suited for us and this along with mysql-events gave us binlog events in the master which we could easily apply to our slave tables for reliable replication.

Initial Sync

All sync methods usually start with a full copy of the initial table to the destination. The changes beyond that point are usually copied to the destination via below methods.

Incremental Sync

Incremental sync means updating all the changes from the source to the destination including inserts, updates as well as deletes. This is the best option for syncing two sources as it ensures an exact match along with the least amount of data overhead.

Usually, to achieve this, a log of all transactions on the table is needed like the binlog in MySql and oplog in MongoDB.

Incremental Appends and Updates

This ensures that all the inserts and updates are synced with the destination but deletes are not. So all the data in the destination table is correct, except that it may have some extra rows.  This is also a good option for many use cases.

To achieve this you just need a cursor field in the source table. An example of a cursor field would be say an update_time field that updates for a row when any other field in the row is changed. If this is available, we can just check for records that have update_time newer than last update and copy just those rows to the destination. Since deleted rows cannot be discovered with it, this method fails to sync deletes.

The extra deletes can then be synced by an occasional full copy.