In business, there is a rule that having a good service or product is not enough if we can't bring it to market. We have to find a way to let people know about us and a system to satisfy the demand we've generated.
In world of software services and products it works in the same way. Therefore we need both marketing and sales. Regarding the sales, part of its system is eliminating obstacles that prevent potential clients from using what we are offering. And one of the main obstacles in B2B software is the often complicated way of connecting our product with client's other services. However this connection is often essential.
Dealing with the integration of your software in the B2B segment is just as important for the business of a given product or service as the door is for a brick-and-mortar store. Having a storefront on the square is great, but if customers have to walk three blocks to get in, how many of them will end up in the store?
Our clients are mostly from the e-commerce segment, where custom plugins are often created for integration into individual e-commerce platforms. This method is used in logistics, ERP systems, and business intelligence. It is also suitable because their logos appear in the marketplace of the platforms. This is, without a doubt, a perfectly valid approach.
How to prepare the right concept for integrations?
In this case, it is appropriate to support the right consideration with the right concept. This way, we will avoid future problems with administration and development of the solution. Integrations will create our strategic element, so we should approach them with the same diligence with which we prepare our own service.
Let's describe the technical issues we will face while integrating. Most of them apply to data integrations in general, but we would like to demonstrate especially the integration using plugins in various 3rd party platforms, because a quick plan for their buildup often leads to unconceptional shortcuts, which can cause problems in the future.
What should we think about and why include an integration service when we have a REST API that e-shop plugins can easily comunicate with? Why should the communication be asynchronous? Let's demonstrate everything with a simple case of sending an order from an e-shop.
Ideally, we will want to call our REST API right at the moment a user creates an order. Ignoring the fact that the plugin author may not have the event available, in a naive way we could call our REST API directly in some cases. However, if we don't want to solve tasks naively, we need to ensure that we make repeated calls when the called interface is temporarily unavailable.
We need to run scheduled tasks on the server for repeated calls, but such permissions will not always be available to plugin creators. This situation already calls for an integration service.
Rate limiting - yes or no?
By handling the unavailability of the called service, we have improved the naive solution a bit. Now let's look at the handling of rate limits. Our system and the plugin we created should be able to handle the rate limits of the other party. You don't need rate limiting on your side? Not really, unless you want to control the overloading of your system. In any case, most cloud service interfaces have limits on their interface and if you want to call them, you have to take them into account.
Now, Let's not deal with how to handle the limitation. We can deal with that question some other time. The point is that the original idea of a “simple plugin” becomes more robust and the plugin "fatter". And how many plugins did we want to create for our integrations? Do we want to write all that logic again and again?
Again the integration service as an in-between link comes to mind. It can handle rate limiting and unavailability of both parties. Of course, the unavailability of the integration service itself must also be handled. A smart solution like Applinth or Orchesty has several ways to do this for you.
Ways to avoid unavailability
The first way to prevent unavailability is throughput of the service entry point, which can handle many times more load than can be expected from a plugin written usually in PHP or from most REST APIs. It's not just about the technology, it's about the fact that even the processes inside the integration service are asynchronous, so the service just receives the request and forwards it to the queue.
Another safety feature that deals with integration service unavailability are batch processes in which the integration service calls a source. We use these instead of real-time signals or in parallel with them (if real-time has high value for us). In addition to source unavailability, we also have to deal with source data paging in this case. And above all, when calling the target, respecting the limits will be critical. Even if it was possible, it makes no sense to write all this individually for each plugin we want to create. We need an integration service.
Finally, a smart integration service has one more important feature. It serves as a buffer that smooths out extreme traffic spikes. It controls the output limits to your API across all calling systems. While the input from the outside is built to absorb thousands of queries every second without scaling, on the output it is configured to not overload your API. Even load means significant optimization of your resources. If you use a smart integration service, you will scale your system much more efficiently.
Solving integrations via plugins
So if you choose to handle integrations through plugins in third-party systems, a good approach is to build lean plugins in the systems and leave most of the logic to the integration service. This will also provide a solution for integrations of systems that do not support custom plugins. It's the most effective way to get the best entrance from the square for your business.
If you've read this far, you've learned some basic rules for building multitenant integrations. In the next article, we'll discuss what the word "multitenant" actually means in the context of integrations and what we need to consider when building them.
