AWS Lambda vs EC2- Comparison of AWS Compute Resources
Compare two of the widely used compute resources in software development: AWS Lambda vs EC2. Understand and analyze infrastructure management, performance and cost comparison. Get to know the evolution of AWS computing resources from EC2 to AWS Lambda and their potential use cases.
Cloud Foundry Foundation, a non-profit organisation that overlooks the open-source cloud computing projects, conducted a global survey recently consisting of 550 users. The survey results made it evident that 31% which is 169 of them are already using serverless architecture.
When further asked which platform they are using, 77% said AWS Lambda. FaaS services like AWS Lambda are becoming popular since they enable an organisation to build agile and scalable applications than server-based applications, for example, EC2.
EC2 requires management and provisioning of the environment. Each EC2 instances runs not just a full copy of an operating system, but a virtual copy of all the hardware that the operating system needs to run. In contrast, what AWS Lambda requires is enough system resources and dependencies to run a specific program.
Also, AWS Lambda enables you to create portable code blocks for easy development, testing and deployment. That’s a winning trifecta!
If that’s all there was to AWS Lambda vs EC2 then I’d be writing a death note for EC2. But, there’s a lot more to it than just serverless-less/full. Let’s discover.
- What is AWS Lambda?
- What is EC2?
- An evolution: From EC2 to AWS Lambda
- AWS Lambda vs EC2: Infrastructure Management
- AWS Lambda vs EC2: Performance Comparison
- AWS Lambda vs EC2: Cost Comparison
- Use cases: When to use AWS Lambda?
- Use cases: When to use EC2?
What is AWS Lambda?
AWS Lambda is an on-demand cloud computing resource offered in terms of function-as-a-service by AWS. Over time, AWS Lambda has changed the way we used to create, architect and run our applications.
The main difference between AWS Lambda vs EC2 (virtual server-based resources) is the responsibility of provisioning and use cases to name a few. AWS Lambda pricing is one of the biggest factors as well.
Before the emergence of agile solutions like AWS Lambda, operations teams had to allocate the resources based on the forecasting. They had to make sure that the computer and memory demands don’t overflow the limits their system can handle.
With the computing resources like AWS Lambda, the computing resources can scale and descends automatically based on real-time demands. Presently, AWS Lambda supports multiple languages and can be used within an application in multiple manner or back-end as a service.
The architecture of applications built using functions like AWS Lambda is popularly called serverless architecture. AWS Lambda is a splendid example of how the overhead of the operation team is going to be a distant memory.
What is EC2?
Amazon Elastic Compute Cloud (EC2) is a virtual cloud infrastructure service offered by AWS. This provides on-demand computing resources through which you can create powerful servers in the cloud.
The entire hardware of EC2 is fragmented into multiple resources which are offered in the form of instances which are scalable in terms of computing memory and processing power.
It also provides you with the flexible options to host your application onto more than one platforms with tight security for multi-model multi-tenant architecture. These instances can be accessed by the HTTP or HTTPS (API) which lets developers create applications just like an on-premise infrastructure.
With Amazon EC2, you have the facility of provisioning virtual machines as per your applications’ requirements. Such facility is provided on a utility based subscription model where the user is billed as per their consumption of resources.
An Evolution: From EC2 to AWS Lambda
With the onset of EC2 as IaaS, AWS wanted to remove the overhead of managing the infrastructure. With the EC2, the time to allocate a server reduced to a fraction along with features like automatic scaling, scheduled provisioning and monitoring & alerting system powered by CloudWatch.
When EC2 was launched, it prevailed with the far more volatile environment than what it is now. Some of the initial issues were the sudden outage, multi-tenant model per machine, failure in scheduled provisioning and disappearance of virtual machines at all. Some of the popular sites that have been affected due to this are Reddit, Foursquare, Rapportive and Heroku.
Then came Elastic Beanstalk (EB), which provided all these facilities in a pretty package. EB is available to work with multiple languages and frameworks. With EB, developers can easily upload their code to the virtual machine itself through the AWS console.
Apart from that, EB directly spins up EC2 instances for you while automatically balancing the load and giving you the direct endpoint to use at the end. Even with this, developers can still log into the AWS console and manually configure or tweak the allocation of instances.
With the launch of IaaS, infrastructure management has been removed. It has benefited tremendously to the number of organisations. However, many of the promised advantages have not been fulfilled yet. For example, provisioning and capacity planning is still a major issue based on the manual forecast. To get ahead of this, FaaS was launched.
Amazon released AWS Lambda which is available to work with multiple languages and frameworks. Like Elastic Beanstalk, you can upload your code packages directly to your functions. Unlike Elastic Beanstalk, the infrastructure is not available entirely for the development teams to tweak or manage.
However, the scaling is fully automatic as per the requirements. As said, Lambdas use ECS and these containers are not available to configure manually. On the other hand, Lambdas are exposed through API Gateway which functions as a URL router to your Lambdas.
AWS Lambda vs EC2: Infrastructure Management
-
Setup & Management Environment
AWS Lambda: Whether you need to set up a multiple or single environment, you do not need to do much of a work. You are not required to spin up or provision containers or make them available for your applications, scaling is fully automated.
AWS Lambda might not appeal to someone already working with an on-demand development environment with containers and orchestration in place.
Amazon EC2: With EC2, setting up includes logging in via SSH and manually installing Apache and doing a git clone. Along with that, you need to install and configure all the required software in a manner which is automated and reproducible.
For EC2, instances come in two options, first are standards ones which serve data roughly the same as our desktop hard drive and second, advanced provisioning which will serve data much faster. Comparatively, this is a lot of work.
-
Security
AWS Lambda: Serverless architecture abstracts away the patching and OS update manual work. With Lambda, you have the higher flexibility of workflows but at the same time, it increases the surface attack. What you need to consider is how to secure communication happening inside and outside your application.
Another thing is that the granularity of functions is really high. With an increasing number of functions, monitoring becomes hard which in turn is a threat to decaying functions.
However, vulnerability breaches are less likely to happen considering the stateless characteristic of the functions. Malicious agents grow over the time which isn’t possible considering the statelessness. More to that, functions are automatically scalable which gives good protection against DDOS attacks. The protection is facilitated by auto-scaling functionality but it also increases your bill.
Amazon EC2: With EC2, you’ll have to take care of the security layer at the instance level. Security layer decides and controls the traffic allowed to communicate with each instance. Each instance can have multiple security layers which dictates the allowed inbound traffic through certain protocols like TCP, UDP, ICMP, etc.
Creating policies which will have correct permissions is tiresome and a work of trial and error. This is especially true when you’re working with a growing team. Handling permissions for every specific business needs mean changing policies often and you end up increasing the unwanted granularity.
Meanwhile, with Lambda, this is entirely taken care of by AWS along with OS patches and system maintenance.
If we consider DDOS attacks, you’ll either have to opt for other services of AWS Shield or you can do it manually by using ELB to scale under the attack or limiting the rate of requests/minute from a particular id. EC2 definitely have security groups and firewalls but unfortunately, those are not enough to monitor resource-based traffic monitoring.
AWS Shield helps in automatically detect the type of AWS resource behind the elastic IP address and apply the relevant DDOS protections.
AWS Lambda vs EC2: Performance Comparison
-
Timeout
AWS Lambda: As per the official documentation, AWS Lambda has the timeout of 300 seconds. This limits the type of tasks lambda can deal with. Long-running functions and complex tasks aren’t a good fit.
Another problem is the time limit imposed by the API Gateway for the function to invoke which presently is 30 seconds. Even if you manage to architect a solution which handles timeout sufficiently, you’ll have to constantly monitor and debug the issues when they inevitable happens.
Not all timeouts occur due to the 300 seconds limit. Some occur due to the bugs introduced or when you’re dealing with communication over external servers which takes too much time or answers the functions with the wrong data.
Amazon EC2: In comparison to AWS Lambda vs EC2, the later one have pretty flexible options. You can definitely work with long running tasks since instances are available for different types of requirements with different configurations. This makes EC2 a better option over Lambda.
However, EC2 service is not error free. Connection timeout is an issue due to overlapping security group rules and unidentified user key by the server. Another common error happens when you’re dealing with an insecure network over SSH.
This is especially applicable for newly created instances. For example, if you will try to SSH your new instance without waiting for its status checks to get complete, you will get an SSH timeout error. However, Lambda clearly lags while dealing with complex processing over EC2.
-
Dependencies
AWS Lambda: External libraries are inevitable for any project and it is true for AWS Lambda too. When you’re dealing with heavy processing functions like image processing, video conversion, you’re bound to have dependencies for the application.
However, Lambda comes with a limit. Uploading your function dependencies with the package itself seems to be the logical solution. But Lambda has a package size limit of 50 MB. Although, it gives you the option of downloading the dependencies once your function is executed from its “/tmp” file storage.
More to that, “/tmp” file storage has a limit of 512 MB. Also, the higher the number of dependencies, more time will be required to execute the function successfully.
Amazon EC2: Managing dependencies in EC2 isn’t a big problem since it doesn’t have constraints when it comes to temporary storage. Though what you should consider is the size of software packages and corresponding instance CPU. This is because your CPU may undergo burden if it’s not configured for the same.
For example, the base of Amazon Linux Containers is already preloaded with many software packages that are required for executing the basic functionalities.
-
Scalability
AWS Lambda: Scalability is one of the major benefits of Lambda as most of the part is automatic and is handles by AWS. It scales dynamically in response to the increased traffic. At the time of the surge, it increases the number of concurrent executing functions. The number of concurrent limits is set to 500 as of now.
But scaling isn’t as flawless as it is portrayed. In practical cases, we’ve faced errors during the creation of new Lambda functions. It is definitely great to have an automatic scaling system but it isn’t much fun when you can’t address these errors and work upon it.
For example, API Gateway timeout to invoke a lambda function is 30 seconds. And when you violate this limit, you’ll see a spike in 5XX error being returned from API Gateway. The only option you have is to retry the request until you succeed.
Another example, Lambda depends on Amazon EC2 to provide Elastic Network Interface for your VPC-enabled Lambda function. And hence, your function’s scalability also depends on EC2’s rate limits as they scale. You need to check whether your VPC-enables functions are allowed 500 concurrent invocations per minute or not.
Amazon EC2: With EC2, everything is under your control. Everything in control means you will have to manually configure the scalability features unlike Lambda and that might not be the novel idea. However, the positive side is you can make your scalability an error-free process. Also, EC2 Auto Scaling groups are there to help you out.
These groups help you to ensure that you have sufficient amount of instances available to handle the load. Simply create the Auto Scaling groups, add the instances and specify the number of minimum and maximum instances in each group.
AWS ensures that your group never goes above and beyond the mentioned number of instances by automatically launching and terminating them.
For example, this above mentioned Auto Scaling group has the minimum number of 1 instance (which will be active all the time) while the desired capacity is 2 instance (to meet the usual demands) and a maximum of 4 instances (to meet the unusual spike in traffic). More to this, there are no additional costs attach with Autoscaling Groups.
Conclusion: Fully automated scaling is definitely a dream come true but not at the cost of not being able to mitigate the errors. On the other side, manually configuring scalability environment is quite a work initially but once you have gained control through setup and tuning, it becomes automatic as well.
-
Availability (On-demand vs Always available)
AWS Lambda: As discussed in the scalability section, Lambdas are available all the time. These are brought up and spun down automatically depending upon the requirements of the event triggers. Since you’re not paying for the idle time with AWS Lambda, you can save a lot of money, about which we will talk in the next section.
However, the debate over the function’s availability is going on. Here is one of the example, “I’m running is US-East-1 and I am getting 500 errors no matter what I do. It was working a few hours ago and I haven’t change any code. Even a new ‘hello world’ from the templates returns this response. The service shows healthy on dashboards.”
Such outcries are common as we give away the ability to manage infrastructure for other added benefits. In such cases, if your Lambda isn’t available, a potential solution would be to deploy your functions into multiple regions. Since you only pay for the executed functions, you only pay for your fallback deploys if there is a region outage.
Amazon EC2: With EC2 instances, you don’t get the benefit of on-demand availability. As discussed in the above section, auto scaling groups help in scalability but in the end, you will have to keep at least one instance up and running.
However, EC2 has also faced many outages. One such was in the US-East-1 region due to the network connectivity which led startups like Foursquare, Rapportive, Heroku and Reddit to the temporary unavailability. A similar outage before this one lasted for 48 hours.
Such examples raise questions over the reliability of cloud platforms and consumer technology.
As discussed the Lambda, replicating the EC2 instances for the multi-region availability is quite difficult. The most possible solution is to deploy a blue/green strategy but that will lead to double the number of running EC2 instances and let alone the extra cost.
-
AWS Lambda vs EC2 Latency
AWS Lambda: The Cold startup is considered as one of the biggest issues with event-driven serverless functions. A cold start occurs when you trigger a function which has been inactive since a considerable time. This delay occurs from the cloud provider due to the time it takes to provision your selected runtime container and then executing your functions and hence this increases the function latency.
This may take more than 5 seconds which makes it impossible to guarantee less than 1 second triggering of function from API Gateway, DynamoDB, CloudWatch, etc. However, a potential solution can be pinging your function from time to time to keep it warm, as discussed we’ve discussed more about in the serverless performance.
However, this wouldn’t be a recommended solution as it comes at the cost of an increased bill. For example, if you have a scheduled function which consumes a lot of memory, CPU and time, Lambda’s cost might be higher even when compared to EC2.
Amazon EC2: Cold starts do not occur in EC2 instances typically unless you’re trying to start a new container. New instances have high latency and takes longer to be ready than existing instances because they typically run code on the first startup. For example, if you Stop the instance and Start it again, the instance will be available quite quickly.
However, the startup time invariable depends on some these issues- local filesystem mounts, remote file system mounts, regular OS initialization scripts, user data and cloud-init scripts, instance type and EBS volume type.
Moreover, the time taken to launch a new instance will be determined by running benchmarks for your particular use case and these points can only help in reducing the time.
Comparatively, this is a tiring task and Lambda seems to be the right choice. Also, consider an example, if you just have a couple of requests to process, running an entire EC2 instance would not be a wise choice.
AWS Lambda vs EC2: Cost Comparison
The setup costs for AWS Lambda vs EC2 is nil since both have free-tier available. However, getting started with AWS Lambda is comparatively easy. Saying that, let us examine what are the computing cost for two distinct use cases.
#1. For a low/infrequent traffic website/app use case, let’s assume the following specifications:
10,000 hits per day over 24 hours
Execution time: 200ms per hit at 512 GB
This will result in 432,000 requests per month and 2160 GB-sec of compute/month. Considering the above case, the total cost would be $0.31/month.
On the contrary, this is the 1/10th cost of what the smallest EC2 instance t2.nano will charge you. Higher EC2 prices is due to its running time for 24 hours (and hence the cost is multiplied with 24) which is not the case with AWS Lambda.
#2. For scalable architecture use case:
Lambda is independently scalable while the same is not the case with EC2. EC2 supports auto-scaling groups but with certain prerequisites.
To tackle the failover behaviour, the recommended number of instances to group together is three. Let’s say, we’re running two “nano” instances to handle the failover behaviour: $0.0059 * 24 * 30.5 = $4.31 for the virtual machine and $0.05 * 8 GB = $0.40.
So, the total cost of the two instances will be $9.42 (at $4.71 each). Autoscaling group would also need Elastic Load Balancer which would at least cost $0.0225 * 24 * 30.5 = $16.47 Can you compare now with AWS Lambda?
With Lambda, you pay per call and per GB second. Ignoring the call cost since it is $0.20/ 1 million requests. 1 million requests mean 0.4 requests per second in a month. AWS Lambda costs $0.00001667/GB-sec.
Both EC2 and Elastic Beanstalk will consume parts from of 512 MB memory of the OS and the container itself. Considering two instances similar instances like that would cost 2 * 256MB/1024MB * 60 * 60 * 24 * 30.5 = 1317600 GB-sec. The cost of this amount of GB-sec will be 1317600 * $0.00001667 = $21.96
This is more costly for obvious reasons. Apart from that, one should keep in mind that the traffic isn’t always distributed evenly and hence you might need more instances that you have anticipated. And this will further increase the cost.
Use Cases: When to use AWS Lambda?
#1. Low complex code: Lambda is perfectly suitable to execute code with no or few variables and 3rd party dependencies. It can quickly process easy tasks containing low complex code, for example, Alexa skills.
#2. Shorter executing time: There is no point for spinning up a container where your tasks are occasional and can be executed within minutes. For example, transforming image/video when uploaded to S3.
#3. Infrequent traffic: In typical scenarios, your servers would be sitting idle while you still pay for it. With the pay-per-function model, Lambda can help you in cutting down your computing cost.
#4. Real-time processing: AWS Lambda with AWS Kinesis works best for real-time batch processing. For example, writing batch data to dynamoDB, analysing logs, etc.
#5. Scheduled CRON jobs: You can use AWS Lambda function with scheduled events to function at a fixed scheduled time. You can specify this time or may use a CRON expression.
Use Cases: When to use EC2?
#1. High-performance computing: With the large number of EC2 instances you can create virtual servers to meet your demands, EC2 is perfectly suitable for processing complex tasks.
#2. Disaster recovery: EC2 can be used as a medium for disaster recovery for both active and passive environments. This is because it can be turned on quickly in case of an emergency which facilitates minimal downtime.
#3. Easy DevOps: DevOps processes are developed and are full-proof when it comes to EC2. On the contrary, when it comes to Lambda, it is still not mature and is an art only a few have mastered.
#4. Development/Testing: Since EC2 provides on-demand computing resources, organisations can deploy large scale testing environment without any upfront investment in the hardware.
#5. Secure Environment: EC2 has been known to provide outstanding security to some of the major clients from banking and healthcare. If security is your first priority, EC2 is for you.
Take Away!
Here we come to the final question: which one of these services is best suited for my application? Well, there is no direct answer. There are many factors which you need to analyze and one of the major ones is your specific use case.If you’re wasting your compute resources due to unpredictable traffic for your application but still want a scalable and cost friendly solution, AWS Lambda is for you. When not to use AWS Lambda? When you want to do complex processing and your process can’t be executed in the limited execution time.
Or maybe you want to run a complex application which has consistent traffic and want to operate in a tried and tested deployment environment, EC2 if for you. The only drawbacks are a complex setup environment and provisioning of servers.
The result- For either AWS Lambda vs EC2 or vice versa, both operate for a highly specific use case, however, one wasn’t sufficing the need which necessitated the invention of another.
Until then, let’s make the most of each of the services. If you’ve hands-on experience with either EC2 or Lambda or both, I’d love to hear from you.