A Library of Node.js Microservices for Everyone — Looking Forward
Two weeks ago we at Polybit Inc. launched stdlib, “The Standard Library of the Internet.” To put it simply, the goal of stdlib is to make building microservices — modular functions that you deploy as standalone web services — as easy as possible. In addition, stdlib isn’t just a service-oriented platform. True to its name, it’s a library as well. By indexing and cataloguing services on stdlib we’re increasing their discoverability and reducing duplication of work.
I’m happy to say that we’ve been very fortunate to have experienced an awesome response from developers so far. Thousands have signed up on stdlib with hundreds having joined our Slack community. We were recently featured on TechCrunch and a headliner in issue 150 of Node Weekly, which we’re unbelievably grateful for.
That said, our goals are ambitious and there’s still a lot of work to be done. I’d like to take the opportunity to talk a little bit about our philosophy around the changing landscape of web services and applications. Containerization is changing the game of how we deploy and orchestrate systems, but as software developers (not infrastructure engineers) why is this important? How does it change how we write code, how do we build with microservices, and where does stdlib fit in?
The Changing Infrastructure and API Ecosystem
As the author of the Node.js framework, Nodal, I’ve been heads-down in the API space for quite some time. In the past couple of years — heck, the past couple of months — we’ve seen a rapid amount of progress in the infrastructure and API space.
Developers have been inundated with promises of indefinite scalability from the likes of technologies like Docker, Kubernetes, Chef, Terraform and many others. It’s quite literally overwhelming. The next wave of development seems to be everybody telling us that we’ve been deploying incorrectly and it’s time to change. Maybe they’re right, but do we really have to learn these complex toolchains in order to get the most utility out of recent advances in infrastructure technology?
Let’s take a step back. Let’s look at what’s already been built, and think about how we can change how we write software to fit into this new world. Instead of jerry-rigging new systems to support legacy application design patterns, why not tackle the space from the other end? What if we can simply make a few changes to how we think about building applications and all of our problems disappear?
Hello, Microservices
Welcome to the function-as-a-service ecosystem, spearheaded by AWS Lambda, where with the right perspective we can stop worrying about how to build fault-tolerant distributed systems, and begin assuming we already have them.
The easiest way to think about building microservices is to treat the entire cloud as if it were a single machine. The computational resources of the planet can be seen as an extension of your computer, and really, every computer. Just as you don’t know (nor do you need to know) the exact memory address of a function in JavaScript, you don’t need to know the exact physical (or virtual) location of a microservice. It’s just a function that runs in the cloud. You access it via a URL. That’s it.
Building With Microservices
There are three main concerns when developing applications using microservices, and I’m happy to address them:
- Network latency — every hop between a service introduces at least 10–100ms of additional execution time due to transit.
- Points of failure — every function call is a potential point of failure.
- Application design — best practices in the space are lacking.
Let’s tackle these one at a time;
Network Latency
There are a couple of concerns here — where are the functions / microservices located relative to each other (geographically) for initialization purposes, and how can I minimize transit time between them?
There are two solutions — one, make sure all functions are deployed in multiple regions by default so there’s always a data center close to wherever the execution request for your service originates. The second part of the solution here is to bundle commonly paired (coupled) functions in the same execution sandbox to give them zero-latency access to each other. We’ve already taken care of a lot of this for you at stdlib, so no need to worry.
Points of Failure
This is more of a “loss-of-control” point. Any function you write in an application is a potential point of failure, so that’s nothing new, but the communication between functions over a network is. The concern here is this: what if my application logic is fine, but the network goes down between two services? I don’t want to hand-wave this as a non-issue — it’s a big one.
This is one of the main reasons we launched stdlib in beta — reliability is a huge concern for us. We’re actively working with large, at-scale service providers to make sure when you build with us we have you covered. It’s our intention to commit to a great service, and we want to make sure we’re prepared to handle your demands. We encourage you to sign up, build, and let us know any questions you may have.
Application Design
Relying on microservices as “web functions” fundamentally changes how we build applications. For one, the concept of a web app as we currently understand it disappears. It transitions from an application instance running on one (or more) machines to become a loosely-defined collection of functions. Composition of projects moves away from MVC and towards a more strictly functional programming approach.
This is an entirely new area that’s going to see a lot of development, and we’re excited to work on it with you. Rest-assured, we’ll be launching tools and development kits in the coming months to make it as easy as possible to build the web application of your dreams relying only on stateless, scalable microservices. We’ve already started on GitHub. If you’re a developer interested in building in this space, we’d love to hear from you.
Where does stdlib fit in?
At Polybit we recognized a major pain point and hurdle to adoption of microservices and we acted. In spite of being on the tip of everybody’s tongue, using them in development environments actually isn’t very accessible — there’s a lot of room for growth. Microservices can be confusing to get started with and ultimately still seem superfluous for basic web applications.
We’re overcoming these problems, and microservices are already fundamentally changing how developers across the industry think about building applications. Let’s accelerate the process for everybody. We exist to help you build great apps, and our goals are simple:
- Make application development with microservices as easy as possible.
- Build a library of web functions to increase discoverability and accessibility of these services.
What’s Next?
We’re keeping stdlib in beta while we work on our security, documentation and sharing features. You’re free to join and help us test our systems — build whatever you want. We’d love your feedback! Users are restricted to three (3) namespaces while in beta, but you can run as many functions as you’d like.
It’s our mission to play a pivotal role in making sure developers around the globe can develop, launch and utilize web services as if the computational resources of the planet were merely their plaything. We invite you to join in.
Want a crash-course on how to use stdlib? Check out our most recent screencast.
Keith Horwood is the Founder and CEO of Polybit Inc., with the vision of connecting developers to the cloud and each other. Polybit is a proud alumni of the AngelPad accelerator.
You may be familiar with Keith’s other work, either as the lead author of the API framework, Nodal, or his Medium stories about Building Recommendation Engines using Graph Theory and Creating a DNA Sequence Alignment Tool in JavaScript.
Follow StdLib on Twitter, @StdLibHQ, or follow Keith @keithwhor.
