Building a better monolith
The monolith gets a bad rap
When you hear that a company runs a monolith, you may think they're old-fashioned and they must have trouble scaling it, right? I'm here to tell you that some people (myself included) think monoliths are awesome for a whole lot of teams. That said, technology has advanced, and I truly think it's time to revisit the monolith with a new approach.
If you're new to back-end development, a monolith is a server-side system that runs as one.... thing. It's a single program that starts up, serves some network requests, and then terminates. Alternatives to 'the monolith' include service-oriented architectures (SOA), microservices, serverless functions, and probably a few that I haven't heard of. Each of these approaches has their time and place, and I salute anyone who has made an educated decision to build a system with these patterns. I'm of the opinion that a large number of web applications and services would be well served by a monolith.... but with some upgrades.
You need some services to lighten the load
The reasoning behind the alternative design patterns is very sound. By distributing the work among many different “things”, you make the system as a whole able to handle more. It is well known that by doing so, you introduce more complexity, which requires more effort, and therefore often more people-power and more money. The one possible exception I can see is serverless functions, which do indeed simplify many things, but whose downsides comes in the form of more difficult testing, vendor dependence, or the need for non-commodity tooling. When done right, the extra effort can lead to a very capable system whose benefits are truly remarkable. The classic example is Netflix, and they’ve done very well for themselves.
Finding a middle ground
If monoliths are hard to scale and microservices are too complex, then how do you design a system that can scale with your traffic and your development team without becoming a pain to operate, maintain and expand its functionality? Over the past few years, it has become clear that a middle-ground is needed. I don’t expect this solution to work for everyone, but most products aren’t serving the kind of traffic that really makes the microservice effort worth it.
Enter the SUFA design pattern
Before getting into the core of what SUFA is, I want to mention that this is not an entirely new way of thinking. Things like the actor pattern, the Neomonolith, and others have stipulated some similar ideas over the years, and SUFA is just one way of combining several concepts into one straightforward design pattern. So, what is it?
Simple, Unified, Function-based Applications.
Let’s break that down:
A system designed with SUFA can be run in the simplest of deployment scenarios. Auto-scaling groups have existed for a long time, and they’re made even easier by container orchestration systems. A SUFA system can be run on one ASG, or can be expanded with a service mesh to allow for capability groups (which we’ll talk about in a future post).
Rather than multiple services who each exist as something to be deployed, SUFA systems exist as one single deployable. This can be a Docker image, an AMI, or some other artifact, but there is only one thing that needs to be built. It should be built by CI/CD on a continuous or tagged release cadence, and it should be made available in an artifact registry such as a Docker registry or S3 bucket.
A standard monolith probably includes a handler layer which is responsible for taking API requests and making calls to a business logic or data storage layer to handle those requests. SUFA systems instead handle requests by chaining together a series of functions, each completely independent and unaware of one another. Functions should expect a particular input, perform some operations, and produce an output to be passed into functions further down the chain. Functions should be easily testable and reusable across different scenarios (such as for different API requests). SUFA systems should also be designed to consume and produce event-based traffic as a primary method of communication.
Well this seems like it should be straightforward, but in SUFA design, “Application” has a very particular meaning. A SUFA system should serve one single application, meaning that it should encompass all of the capabilities needed for a fully formed product. This can be up for some interpretation (such as whether a company should have one SUFA for their whole business, even if they have distinct product areas), but the point is to avoid having multiple “things” serving one application. If functionality needs to be shared across multiple applications, the functions comprising the SUFA system should be easily reusable and composed for other purposes.
You'll notice that this is all very technology-agnostic and vendor-agnostic. SUFA is meant to span across languages, cloud vendors, and deployment environments. SUFA is a way of designing your server-side system such that it is testable, scalable, and secure. You'll notice I haven't touched on scalability yet, so let's discuss that
SUFA at scale
The critical factor that allows a SUFA system to scale is that it is composed of independent functions. SUFA systems should rely on an underlying framework to orchestrate the execution of these functions such that it can scale effectively. By using a function runner or job scheduler to run the required functions, a SUFA framework abstracts away how the functions are executed, and the programmer writing the code only needs to indicate which functions to run, and in what order.
Additional scalability is provided by capability groups and meshing, which I plan on writing follow-up posts for, as they deserve to be explored at length.
The SUFA pattern was designed in concert with Atmo, which is an all-in-one framework upon which SUFA systems can be built. Atmo uses a file known as a 'Directive' to describe all aspects of your application, including how to chain functions to handle requests. You can write your functions using several languages to be run atop Atmo, as it is built to use WebAssembly modules as the unit of compute. Atmo will automatically scale out to handle your application load, and includes all sorts of tooling and built-in best practices to ensure you're getting the best performance and security without needing to write a single line of boilerplate ever again.
The awesome capabilities of WebAssembly and the design thinking behind SUFA are being harnessed by the open source Suborbital Development Platform to introduce a new way to build your web applications. I've been working for over a year to realize this goal, and I'm extremely happy with the results thus far. Riding the wave of new technologies and practices such as JAMStack and edge computing means that we have a opportunity to bring the best of the old and the new to the next generation of software makers to do incredible things. I hope you'll come and join me!