Is Your API ACTUALLY Ready for User Traffic?

- In this video, I will show you how to test the performance of your API and the different types of tests you need to run to replicate real users behavior. I will also introduce K six, the best tool available for performance testing.

So how much would it cost you if you completely wrecked a client's new product release? So I was in charge of payments on the project, and on the release day we were expecting tens of thousands of users to submit payments through the platform. So needless to say, this was by far the most stressful moment in my career, because a failure would mean a massive loss in revenue and colossal damage to the company's reputation. But we were well prepared and everything worked because amongst other things, we heavily tested our platform using K six. And in this video I'll show you how to test the performance of your API and the different types of tests you need to run to replicate real users behavior. And to do that we will use K six, which is the best tool available for performance testing. The first thing to know about K six is that it's open source and you can use it for free. So you can run it on Windows, Mac and Linux, but you also have the option to run it on the cloud through Grafana cloud for free or a paid version if you really need to run a lot of tests with really high volumes. With K six you write your test in JavaScript and you run the test with a terminal command. For example, you can write a test that sends a request to one of your API endpoint, and when you run the test you get some performance metrics as output. But before implementing your test, you first need to ask yourself, what does performance mean in the context of your API? Because that will influence the type of tests you will implement. And I'll show you that there are four types of performance tests that you can implement and you might not need all of them. Defining performance will also help you decide the success and failure criteria for your tests. In general, you want to define your APIs normal traffic and acceptable response time. Once you have that, you can write test to ensure that your API meets those criteria. For example, I can test with ten virtual users and check that the API responses are under 200 milliseconds and I can run that test for five minutes. This will give you some initial indication of how your system performs under normal conditions, and you can add this test to your CI CD pipelines and assess the impact of any change on your API performance. This type of test is called load testing, but in order for your test to fail you'll need to set some thresholds. But it's not as simple as saying that if one request fails during the load test, then you can't deploy it to production because whenever there are network calls involved anything can happen and we need to give ourselves some flexibility. So that's why I the threshold is defined in terms of percentages. For example, you can say that if 99% of the request returns with a success status and the response time is under 200 milliseconds, then you're good to go and you can deploy it to production. But load testing alone is not enough. It only tells you how your system behaves under an arbitrary load, in this case the normal load condition. But in a real application, you might have times when you receive more requests than usual and you want to know how your API will behave. When that's the case, you can gradually push your API to certain levels and see how your system behaves. And for that you can use the concept of stages. For example, you can gradually ramp up the number of virtual users up to 100, and then stay at that level, and then load to 200 and stabilize at that level, and then up to 500 and stabilize at that level. And at the end of your test, you gradually go back to zero. That method of testing your API is called stress testing. In this type of test, you want to push your API to a certain level to see what happens. And you might want to increase the response time you're expecting or even the percentage of expected successful responses, because it's only fair to assume that when your API is under a lot of stress, the performance will be degraded. So what you want to assess is whether the performance is still acceptable under those extreme conditions. But let's imagine that you are building an API for a concert ticket selling platform. So when there is a very popular event, you usually get extremely high volumes of requests during a short period of time, and then nothing once the tickets are sold out. So in this case, you don't really have the concept of normal traffic. So load testing doesn't really make sense. And stress testing with a slow ramp up and a sustained traffic level doesn't make sense either, because you want to test your API when there is like a sudden increase in the number of requests for a very short period of time. What you need in that case is a spike test. So you start with a stage with a very low number of virtual users, then add a stage where you drastically increase the number of users over a very short period of time and you sustain that high load. Then you decrease the number of virtual users to simulate the end of the spike. As you can see, it's really important to define what you mean by performance in order to implement the correct testing strategy. So in some cases you might just need spike testing, in others you might need load testing and stress testing or you might need all three. But there is one thing that none of those three tests can spot. Imagine that there is a memory leak thats causing the performance of your API to slowly degrade over time, so you wont be able to spot it with a test that only runs over a few minutes. So what you need to do is run your test over a few hours to ensure that your API has a stable response time and resource usage. That type of testing is called SOK testing. Not only can you check the k six output at the end of the test to check the usual metrics on success rate, but if you also gather some metrics such as memory usage, cpu usage or you record request traces, you can get a very good picture of how your API behaves and potentially identify weak links and the area in your system that are performances bottlenecks. This is particularly important if you have a microservices architecture where different parts of your backend communicate via the network, and this software architecture can be ideal for dealing with large scale applications that can handle large volume of requests. And in this video I show you how to build such a software system. I go through the project setup, how to make your microservices communicate, and some best practices and common patterns to avoid total chaos.