Databricks is a software company that has established itself in a variety of sectors, with data warehousing, and AI-based solutions being their primary focus.
In recent times, we have seen the meteoric rise of ChatGPT, resulting in similar efforts from the likes of Meta, Google, and even Mozilla.
And now, Databricks is trying in their own way by open-sourcing its large language model (LLM) 'Dolly'.
Let's take a look at it.
Suggested Read 📖
Sourav Rudra
What is happening?: In a recent announcement, Databricks introduced what they term as 'a cheap-to-build' LLM that functions by using an existing open-source parameter model by EleutherAI.
The model has been slightly tweaked to give Dolly instruction following capabilities such as brainstorming and text generation.
When you compare the 175 billion parameters in GPT-3, Dolly's 6 billion parameters might seem puny in comparison.
But, the folks over at Databricks were surprised when they saw that even with this much data, Dolly was able to exhibit many of the same capabilities as ChatGPT.
Below is one of the examples they showcased:
The original model used data from Alpaca, the model built by Stanford using the LLaMA LLM by Meta as a base.
But, as you can see, the original model produced a very haphazard result, whereas Dolly, with its different model and tweaks, was able to produce a far usable answer.
Why now?: According to Databricks, they think that many companies would prefer to build their own model rather than sending data to some centralized provider who has locked their model behind an API.
Many companies might not be comfortable handing over their most sensitive data to a third party, and then there are the various tradeoffs in terms of model quality, cost, and desired behavior.
Do you want to check it out?
Sure, but there's a catch.
You will have to use their platform to use Dolly, they have open-sourced a Databricks notebook that will help you build it on Databricks.
Moreover, if you want to get access to the trained weights, you will have to contact them. I am uncertain whether they will provide access to it for free, though.
In a nutshell, this move to open-source their model should be good for companies to help safeguard their data, save on operating costs, and more by enabling them to create their own model.
You can check out the announcement blog to learn more about the technical details and other plans for it.
Suggested Read 📖
Ankush Das
Here's why you should opt for It's FOSS Plus Membership
- Even the biggest players in the Linux world don't care about desktop Linux users. We do.
- We don't put content behind paywall. Your support keeps it open for everyone. Think of it like 'pay it forward'.
- Don't like ads? With the Plus membership, you get an ad-free reading experience.
- When millions of AI-generated content is being published daily, you read and learn from real human Linux users.
- It costs just $2 a month, less than the cost of your favorite burger.
