I seen a lot of posts here talking about this or that model being great for storytelling/writing but when I try them out the prose is…well…flat, boring and plain unfunny. I’m not interested in models to write NSFW (nothing against it just not my thing). I’m looking for models that can actually output stuff that sounds literary - for example, if I ask the model to write something in the style of x (where x is an author with a very unique style) that the output has some of the author’s style in it. Or if I ask it to write a poem, that the result isn’t like something out of some kid’s book written by Dr. Seuss.
With the exception of one model I tried [Storywriter 13b] (which sort of produced something literary after a little coaxing and leading). All the others produced results which sounded like entries from an encyclopedia or dictionary (lifeless, droning, emotionless, etc.). And the leaderboard hasn’t been much help in identifying anything that’s close to what I’m looking for - the top rated models I have tried are the worst when it comes to prose of the kind I’m looking for, in my limited experience.
Does anyone know of any models, that I can run on my local computer, that can produce “literary” prose (I.e. moving, detailed descriptions plus creative story writing)? Not looking for perfect just better… I’m hoping one of you might have come across a model I haven’t seen/tried so any and all suggestions will be appreciated.
You must log in or register to comment.
This post is an automated archive from a submission made on /r/LocalLLaMA, powered by Fediverser software running on alien.top. Responses to this submission will not be seen by the original author until they claim ownership of their alien.top account. Please consider reaching out to them let them know about this post and help them migrate to Lemmy.
Lemmy users: you are still very much encouraged to participate in the discussion. There are still many other subscribers on !localllama@poweruser.forum that can benefit from your contribution and join in the conversation.
Reddit users: you can also join the fediverse right away by getting by visiting https://portal.alien.top. If you are looking for a Reddit alternative made for and by an independent community, check out Fediverser.
Most models are going to require some heavy prompting to get them even close to the prose you’re looking for.
I’ve heard good things about Nous-Capybara-34B and Goliath 120b
I could write a book about this.
First no model will be able to write well in style x and then in style y. Not at the size we can work with. Not at the size of ChatGPT either, although it would generally work better (it usually well approximates what the style is and falls into a Hollywood imagination of that style)
With finetuning you can make the model do one of those things.
## Instruction:
Rewrite the following text as a poem: Lister sat near the fireplace, looking at his arms which were covered in ointment. I find it hard to believe that pain can form words. Is this something internal or external? On his left arm, the message read “Dying” while on his right arm, it said “Living”. He did not know what to think. A man who had been through a lot.
## Response:
Lister sat by the hearth and gazed
Upon the ointment-soaked clay
That coated his sinewy bare arms.
How could such agony frame
Such mystic characters? On one arm
The word “Dying” was graven clear;
On the other, “Living” appeared.
What could it mean? Ah! there,
A man whose life had been one tale
Of woes, of tortures, of crosses, of shame,
Who knew not what to hail as his own name.
Interesting result you got there. Good job! Finetuning is probably the way to go for consistent results for a given style.
I have been able to get closer to what I want via prompting with one model but not perfectly. For example, I asked a model to write a detailed description of an object in the style of Charles Dickens and although it deviated every so often from the style, overall it felt very “Dickens-esque”. Result quality varied depending on the author I prompted for. But most of the models I have tried (large and small) totally ignored the prompt and wrote bland encyclopedia entries. At the very least, I’d like a model that knows something about classic authors and their works - but I know it’s a big ask.
BTW, if you ever decide to write that book of yours, I’d be interested in reading it!
Noromaid 20b and dolphin-mistral 7b do fairly well. Neither have much of a context length though.
Here are my top faves atm, not perfect but sometimes they output some decent content:
https://www.reddit.com/r/LocalLLaMA/comments/181w2b1/what_is_your_top_3_fav_llms_with_settings_for/
Doing a full fine tune on Mistral 7B is the only way I’ve gotten human, literary text out of any of these models. Occasionally the vanilla Llama-2 70B will output something great. Yi-34B-Chat, while not by default a literary writer (it’s got that clunky, purple prose, GPT-4 feel to it) impressed me with its ability to write in a requested style.
The old Guanaco 33B and 65B models produced nice prose, but unfortunately they’re only 2048 context and they weren’t the best at following instructions.
As I understand LLMs basically write the average pattern of a billion books, so when you add gpt-4 and 3.5 data into the mix, which averages the average, things get boring very fast. For model suggestion, Yi-34b based ones look fine for literary purposes.
I think being very specific and editing (co-writing with the model) could help. Some LoRA training on specific books could be helpful to mimic a certain style.
High temperature and repetition penalty could help too.