The power of the Serverless Framework

The Serverless Framework (hereby mapping “Serverless Framework” to “Serverless” for this article) in my own definition is essentially a CloudFormation generator. It’s the first framework of it’s kind to be developed solely to build applications on AWS Lambda functions. However it is cloud-service agnostic, and can work on virtually any cloud service you can think of (Azure, Google Cloud Platform, Alibaba Cloud and Tencent to name a few). It’s also runtime agnostic, although that’s more of a feature of the cloud service. You can write your logic code in virtually any language, as long as the cloud service you’re using supports it. AWS Lambda for instance supports a whole array of runtimes and all of them can be used within Serverless.

I have been using the Serverless Framework for about two years now and I wanted to share my experiences with it. I will go over some of the pros and cons of Serverless and also my opinion on why people should adopt it if it makes sense for them.

CONS

I’ll start with the cons of Serverless, to start, if you don’t know what you’re doing then doing Serverless can end up being expensive. Let’s say you don’t really know what concurrency within lambda’s mean, and you set yours to some ridiculous high number. Let’s also say you’re building a big data ETL pipeline and you’re reading and writing to a database, while also making external API calls. This can get quite expensive if you don’t necessarily know the pricing model of AWS (or any cloud service). If something just “fixes” your problem and you don’t look into how much it’s going to cost you could run into a situation where simply using a EC2 would actually be cheaper.

I happen to believe that this is a skill you might have on top of other skills, but it will also narrow down your job search, since a lot of dev teams will probably not be using Serverless. Although it’s been adopted by a large number of developers and teams, it’s not like it’s used everywhere. So looking for a job, or even people to hire might not be as easy. With this being said, I personally wouldn’t not hire someone simply because they have no Serverless experience, if they were willing to learn and had a decent understanding of cloud services, it’s something I believe you can definitely learn on the job. Not a deal breaker in the slightest, but worth noting.

It’s not impossible to have good insights into your lambda’s, you can utilize tools like Cloudwatch Alarms and dashboards to get a good idea of how your lambda function is running. For example, at my current role we’ve utilized Cloudwatch quite a bit to get insights into our how often our lambda’s were failing. This is particularly important to us, because we have SQS triggers that hit these lambda’s, so failing will simply put messages back in the queue. This isn’t a bad thing, it’s our design, however we do like to have insights into hundreds of lambda’s being ran during a ETL pipeline. So although it’s not impossible, I think it’s worth noting its not as easy to monitor this. At the time of this writing, AWS just released some powerful lambda monitoring tools so this could actually be a moot point.

I almost feel like this next point is actually a pro of Serverless, but I have had cases where it felt like a downside. I’ve had times where I’ve actually had to go into the AWS UI, and sometimes the verbiage or just the UX of the services isn’t great or at least isn’t a good mapping to what I’m used to when writing my YAML files. So I’ve had times when I didn’t know how to create something in AWS UI, but I’ve done it dozens of times in my Serverless code. I do feel like this is partly a upside to Serverless because you actually have to learn more about the resources you’re creating, but that doesn’t mean the UI will be as easy. Really minor point, but again worth noting.

PROS

The best part of Serverless, in my opinion, is the fact that you manage all your resources within the same repository as your codebase. Even if you do microservices, all the resources that the microservice you’re building uses will be in the same code base. This is by far the biggest upside to using Serverless, if nothing else it’s way more streamlined this way. Without Serverless, you’d have to go into your cloud service and sort of just guess what resources belong to the codebase you’re working on. Perhaps you have an org setup for each project you work on so its easy, but nevertheless, I think one thing everyone can agree on is the fact that all of the resources coupled with the code that uses it is a huge upside.

I would also include the fact that because your resources are in your code base, this in a way will force the user to truly understand the resources they are making. Historically speaking, I’ve used cloud services before my current job. I put “AWS” on my resume thinking I actually knew what goes into each resource you need to make, and I was wrong. Typically you’d create the resource you need using the UI, for me, I would do this to but I didn’t know all of the required properties each resource needs and why. After two years of using Serverless, I understand these resources much better. The fact that I have to go look at AWS documentation to get my resource to deploy or it wouldn’t work. If I’m missing a required property for instance. It forces the user to go look at the documentation of it, and because you’re not just mindlessly using their UI flow, you have to actually know about the properties you’re setting. This really enabled me to learn a lot about AWS.

The Serverless community is huge, and there are tons of resources for anyone who needs help. I have yet to come across a problem that isn’t solved using a Serverless supported plugin or simple fixes on my end. For example, retrieving the AccountId of the current scope was always a pain point for us. One solution we had was to add a accounts property in our custom in serverless.yml. We’d then have a dictionary of

env = account_id .

For example,

dev: 111111111, qa: 222222222, uat: 333333333, prod: 444444444

and since we know the stage of the scope, we can retrieve the account id using something like this:

${self:custom.accounts.${self:provider.stage}}

but as you can see this is pretty messy and not as intuitive as one would hope. But Serverless has a supported package called serverless-pseudo-parameters that allow you to do something like this:

#{AWS::AccountId}

Of course this is limited to the resources files and won’t work everywhere, it’s still a valid solution to a rather annoying problem. This is just one example, but if you ever need any help there’s a rich ecosystem of helpful folks willing to tackle any issue you come across (if you can’t figure it out yourself).

I was able to step into Serverless right away as well. The learning curve was mostly getting used to yaml, rather than json. The intricacies of yaml were more difficult for me to learn than Serverless. This is simply because Serverless is really just the Serverless config file, you then define everything you want inside of it. Anything else is more or less the same as any other codebase or monolithic application. You still have a router for API’s, you still have actual code you write..etc. I would say it’s incredibly easy to understand Serverless, esepcially if you have even the slightest bit of understand of what Cloudformation does. In fact, we were able to port a old express app to Serverless with just a simple plugin.

CONCLUSION

I have tried to remain as unbias as I can for this article and took effort into making sure I try to see different perspectives. Just because Serverless has worked for our problems, doesn’t mean it’s going to solve everyone’s. But in the end I feel like Serverless is one of those frameworks that I’ll be using for quite some time. I feel like I’ve learned a lot using it, and this is valuable knowledge that translates directly into higher salary. Aside from the fiscal benefit, being more familiar with cloud services just helps you be a better developer. Again, it’s not going to solve every problem and not everything can be done in lambda functions. I have surely grazed the line between when something needs to be in an actual dedicated server or in a lambda function. But I can say, that as a full stack polyglot engineer, using Serverless has truly expanded my knowledge base and just overall an exciting technology to use.