Serverless and Cost Duality.
Serverless and Cost Duality. Wolfram's paper "What is ChatGPT? and Why does it work" explains inner workings of ChatGPT and LLMs. Also, a re-org song that everyone should listen to.
Hope you are having great days. đ
This is a fortnightly newsletter in which I share interesting articles, talks, quotes from literature, comic strips đ and many such things that I found insightful around software and its makers.
Things this fortnight:
Serverless and Cost Duality.
What Is ChatGPT Doing⌠and Why Does It Work?
But first, we need to make this âRe-Orgâ rap as an official re-org anthem! đ
Serverless and Cost Duality.
Serverless often gets a bad rep in the industry. Mostly because if left untapped , it is very potent in drying out your entire capital. We also have commentaries on how teams are able to reduce costs by moving away from serverless.
"As a bootstrapped company, there was no way for us to come up with $72K.
By this time, I was well versed with Chapter 7 and Chapter 11 of Bankruptcy and mentally prepared of what could come next."
â Source
Forrest Brazeal has written about the risks of cloud billing citing horror stories of bleeding money across the three major cloud providers. Brazeal mentions having near real time billing and ability to have hard caps on cloud spend would really help mitigating the horror. Unfortunately, we're not there yet.
There are ways you can codify some controls in though, like limiting the concurrency limits of your lambda functions and being stringent about the memory limits. Yan Cui explains how AWS lambda pricing works and how you can optimise the costs. You can also use AWS Lambda Power Tuning tool which runs in your AWS account and helps you decide the optimum configuration of your lambda.
On the other side, serverless when used for prototyping is actually a huge cost saver. Infrastructure cost is not the only cost a team has to think about. There is development cost and maintenance cost. Every developer also comes at a cost. Add dedicated ops people to the mix and the costs start to rise non-linearly. At early stages of a company, there is also an opportunity cost. Miss the opportunity to land your product at the right time, and you may never recover from it. The ecosystem of serverless and various building blocks makes it easier to build and ship faster.
Adrian Cockroft has a great short video explaining how serverless first is like building with lego blocks. Building with serverless components is âquick and cheapâ. It shortens the feedback loop. If youâre trying to launch a new product, you get to launch sooner and test the waters before your competition does. But then why not start with monolith directly? Do you even need serverless at this time? Monolith first is generally a good advice, but there are a lot of nuances. I plan to cover this aspect soon, but in a different article to keep this one a bit shorter.
If everything goes right, your product will pick up more traffic, and will have more and more users coming in. Youâll start to notice the costs of lambdas are going higher and higher. Youâll start to think about bespoke optimisations. This is a right time to start taking out the concerned serverless components and start replacing them with bespoke services and deployments.
What Is ChatGPT Doing⌠and Why Does It Work?
ChatGPT, in simple terms, is doing a next word prediction.
The first thing to explain is that what ChatGPT is always fundamentally trying to do is to produce a âreasonable continuationâ of whatever text itâs got so far, where by âreasonableâ we mean âwhat one might expect someone to write after seeing what people have written on billions of webpages, etc.
So given an incomplete sentence: âThe best thing about AI is its ability toâŚâ, ChatGPT would rank the next set of possible words by their probabilities of occurrence from the training data (lots and lots of human produced text on Internet) and then choose a word to complete the sentence ârandomlyâ. Iâve quoted ârandomlyâ because given the same incomplete sentence, the completion wouldnât be the same.
So how is it able to complete the sentences in so human-like manner? How is it able to write lengthy essays on a given topic in such seemingly meaningful way?
Stephen Wolframâs What Is ChatGPT Doing... and Why Does It Work? is one of the best papers Iâve come across which describes the inner workings of ChatGPT from ground up.
etc.
Last week I published my summary and annotations of D.L. Parnas paper âOn The Criteria to be used in Decomposing System into Modulesâ. To which Naveen Negi suggested a follow up read: Volatility based decomposition.
The Value Flywheel Effect, by David Anderson is good read about how value chain mapping and serverless can compliment each other.
Thatâs a wrap!
Have good days! đ