Open-Source Model 'Dolly' Claims to be a Cheaper Alternative to ChatGPT

Databricks is a software company that has established itself in a variety of sectors, with data warehousing, and AI-based solutions being their primary focus.

In recent times, we have seen the meteoric rise of ChatGPT, resulting in similar efforts from the likes of Meta, Google, and even Mozilla.

And now, Databricks is trying in their own way by open-sourcing its large language model (LLM) 'Dolly'.

Let's take a look at it.

Suggested Read 📖

What is happening?: In a recent announcement, Databricks introduced what they term as 'a cheap-to-build' LLM that functions by using an existing open-source parameter model by EleutherAI.

The model has been slightly tweaked to give Dolly instruction following capabilities such as brainstorming and text generation.

When you compare the 175 billion parameters in GPT-3, Dolly's 6 billion parameters might seem puny in comparison.

But, the folks over at Databricks were surprised when they saw that even with this much data, Dolly was able to exhibit many of the same capabilities as ChatGPT.

Below is one of the examples they showcased:

a screenshot of how dolly performs in an open question and answer scenario — Dolly's performance in an open question and answer scenario.

The original model used data from Alpaca, the model built by Stanford using the LLaMA LLM by Meta as a base.

But, as you can see, the original model produced a very haphazard result, whereas Dolly, with its different model and tweaks, was able to produce a far usable answer.

📝

Fun Fact: The name was taken from the first cloned mammal, Dolly the sheep.

Why now?: According to Databricks, they think that many companies would prefer to build their own model rather than sending data to some centralized provider who has locked their model behind an API.

Many companies might not be comfortable handing over their most sensitive data to a third party, and then there are the various tradeoffs in terms of model quality, cost, and desired behavior.

Do you want to check it out?

Sure, but there's a catch.

You will have to use their platform to use Dolly, they have open-sourced a Databricks notebook that will help you build it on Databricks.

Moreover, if you want to get access to the trained weights, you will have to contact them. I am uncertain whether they will provide access to it for free, though.

In a nutshell, this move to open-source their model should be good for companies to help safeguard their data, save on operating costs, and more by enabling them to create their own model.

You can check out the announcement blog to learn more about the technical details and other plans for it.

Suggested Read 📖