TL;DR:
- Uncensored, Mistral 7B based model that lets you write stories in collaborative fashion, but also works nicely for chat / (E)RP
- Hugging Face link: https://huggingface.co/dreamgen/opus-v0-7b
Hey everyone, I am excited to share with you the first release of “DreamGen Opus”, an uncensored model that lets you write stories in collaborative fashion, but also works nicely for chat / (E)RP.
Specifically, it understands the following prompt syntax (yes, another one — please don’t hate :D):
(Description of the story, can also optionally include information about characters)
...
(Instructions as you write the story, to guide the next few sentences / paragraphs)
You can find more details about prompting the model in the official prompting guide, including a few examples (like for chat / ERP).
The initial model is based on Mistral 7B, but Llama 2 70B version is in the works and if things go well, should be out within 2 weeks (training is quite slow :)).
The model is based on a custom dataset that has >1M tokens of instructed examples like the above, and order of magnitude more examples that are a bit less instructed.
How to try it out
The model should work great with any tool that supports the Mistral 7B base model. It will work well with oobabooga/text-generation-webui and many other tools. I like vLLM.
Using vLLM
- Install vLLM following the instructions in the repo
- Run python -u -m vllm.entrypoints.openai.api_server --host 0.0.0.0 --model dreamgen/opus-v0-7b
Using DreamGen.com website (free)
You can also try the model on dreamgen.com for free (but it requires a registration with email).
What’s next
I believe that for story telling & character creation it’s especially important to have access to the model weights, otherwise you run the risk of losing your plot or virtual companion (as already happened a few times before on various closed platforms that suddenly changed their rules or got shut down by their API provider). Hence DreamGen.
Here’s a high level overview of what I would like to do next under the DreamGen umbrella:
On the model side:
- (Soon) Larger story models
- Fine tune the model for even better character chat & roleplay
- Longer context windows, at least for smaller models (8-16K depending on how experiments go)
On the application side, I am thinking about these features:
- Character editor, chat & roleplay
- Ability to share your stories privately & publicly (not sure about this one, to be honest :))
- Image generation to go alongside with story generation & chat
- API so that you can use the model more easily if you don’t have a GPU
For all of these, I would love your input! You can vote on the roadmap here.
For more updates, join the community server or follow updates on Twitter.
You must log in or register to comment.
Is this the first fully uncensored mistral 7b?
but Llama 2 70B version is in the works
Might I suggest you use Yi-34B-200K instead? Or maybe later?
The problem I always have with storytelling models is slamming into the context limit, but Yi is already storytelling well out to 42K tokens for me, with just a basic Alpaca LoRA.
I was going to suggest you triain on Yi 34B 200K instead of Llama 70B, as my biggest issue with storytelling models is slamming into the context limit.
…But I just remembered that Yi has a stupid noncommercial license https://huggingface.co/01-ai/Yi-34B-200K/blob/main/LICENSE
Ugh. I hope a long context ~34B comes out that doesn’t have such an ugly license.
Why only 7B?
Any GGUF quantized download available?
Great news, the great /u/TheBloke is working on this!
There was a bug on the website where the first time the “Continue” would not work if you did not refresh, should work now even though the editor is quite janky still, sorry for that :(
(can’t wait for AI to take over React from me :P)
I have no idea how your model is but that prompting guide is probably the nicest one I’ve seen, so kudos on that.
Sooo, how’s the model?
is the context length 2048?
The training data had example of up to 4096 tokens. The model should also work beyond that, but I did not do a deep analysis of degradation.
This model is really good! By using the prompting template provided by the OP in the comments, I managed to get some really cool stories going. I haven’t really tried stories with other models before, so maybe this is the norm for Mistral-based models, but I’m heavily impressed regardless.
Not getting any of the common blocker phrases once you ask for something even slightly bad to happen to a character is also pretty nice.
Thank you so much for the kind feedback! If you have found some cool prompts, come share them with others on our discord.
Thank you for sharing! I am going to learn cooking with it.
I hope it will be something tasty! :)
Very cool website and model!
Thank you!
Awesome l
Becoming your own AI company has never been easier 😊
Looking forward to the release of 70b!
Currently seems very expensive. Use of 7b models is effectively available for free in many places, Openrouter, Agnaistic, etc. Seems ridiculous that you don’t get unlimited usage even with a subscriptions.
I agree, I hope I can make things cheaper with better utilization. You have to consider that a single GPU is not used 100% the time, so there’s a lot of waste. And due to lack of scale, I also do not get any special pricing on the GPUs. The more users, the closer the utilization will be to 100%, and the better GPU pricing. (For instance, I heard that on Google Cloud, enterprise customers can negotiate the on-demand GPU price down to the regular spot price for some of the GPUs)
