Databricks is a software company that has established itself in a variety of sectors, with data warehousing, and AI-based solutions being their primary focus.
In recent times, we have seen the meteoric rise of ChatGPT, resulting in similar efforts from the likes of Meta, Google, and even Mozilla.
And now, Databricks is trying in their own way by open-sourcing its large language model (LLM) 'Dolly'.
What is happening?: In a recent announcement, Databricks introduced what they term as 'a cheap-to-build' LLM that functions by using an existing open-source parameter model by EleutherAI.
The model has been slightly tweaked to give Dolly instruction following capabilities such as brainstorming and text generation.
When you compare the 175 billion parameters in GPT-3, Dolly's 6 billion parameters might seem puny in comparison.
But, the folks over at Databricks were surprised when they saw that even with this much data, Dolly was able toexhibit many of the same capabilities as ChatGPT.
Below is one of the examples they showcased:
Dolly's performance in an open question and answer scenario.
The original model used data from Alpaca, the model built by Stanford using the LLaMA LLM by Meta as a base.
But, as you can see, the original model produced a very haphazard result, whereas Dolly, with its different model and tweaks, was able to produce a far usable answer.
📝
Fun Fact: The name was taken from the first cloned mammal, Dolly the sheep.
Why now?: According to Databricks, they think that many companies would prefer to build their own model rather than sending data to some centralized provider who has locked their model behind an API.
Many companies might not be comfortable handing over their most sensitive data to a third party, and then there are the various tradeoffs in terms of model quality, cost, and desired behavior.
Do you want to check it out?
Sure, but there's a catch.
You will have to use their platform to use Dolly, they have open-sourced a Databricks notebook that will help you build it on Databricks.
Moreover, if you want to get access to the trained weights, you will have to contact them. I am uncertain whether they will provide access to it for free, though.
In a nutshell, this move to open-source their model should be good for companies to help safeguard their data, save on operating costs, and more by enabling them to create their own model.
You can check out the announcement blog to learn more about the technical details and other plans for it.
It's FOSS turns 13! 13 years of helping people use Linux ❤️
And we need your help to go on for 13 more years. Support us with a Plus membership and enjoy an ad-free reading experience and get a Linux eBook for free.
To celebrate 13 years of It's FOSS, we have a lifetime membership option with reduced pricing of just $76. This is valid until 25th June only.
If you ever wanted to appreciate our work with Plus membership but didn't like the recurring subscription, this is your chance 😃
Sourav Rudra
