A Novel Technique to Improve Latency and Response Time of AI Models using Serverless Infrastructure

Response time for AI-based models in time-critical applications is always a matter of concern. The situation is challenging when the model is deployed in a cloud-based infrastructure. To address this issue, a cloud-native development methodology called serverless infrastructure enables developers to create and execute applications without having to worry about managing servers. Generally, traditional systems require manual scaling, constant maintenance, and dedicated hardware resulting in high costs. To solve this, the AI model is deployed in serverless infrastructure services for hosting APIs that is to identify marine animals which have the potential to attack human beings at the seashores. The serverless infrastructure suffers from an initializing delay called cold start for occasional requests and hence the response time will be delayed even if the lambda function is free. The problem of cold starts is mitigated using the scheduler in the lambda function. The scheduler sends dummy requests to the server to keep the server warm and active. The AI model used as a test case utilizes Convolutional Neural Network Algorithms and Transfer Learning Technique, for detecting predators near the seashore. The model is deployed in a serverless infrastructure and has the benefits of automatic scaling, pay-per-use pricing, and decreased operational costs. The AI model is implemented in both serverless infrastructure and in Elastic Cloud Compute EC2. The performance of both systems was done for cost, latency, and response time. The proposed system provides promising results when compared to traditional server systems.