Uncertainty in Defaulting to Dialogue - LLM vs. fine-tune

Examples for input prompt "my dog died.."

Some examples that show uncertainty with LLMs to pick up "chat domain generation" off the shelf. It is not that they are "incapable" of the task, but off-the0-shelf you cannot rely on them to consistently perform and "realize" that what should be generated is a dialogue response.

# LLMS (0 shot/few-shot)

## gpt-j-6B

screenshot of generation: https://i.imgur.com/7gJckK8.png

My dog died.. I know you’ve heard it before, but this time I feel it is different. It is not one of those dog deaths, the kind that you read about every day in newspapers. This was not caused by accident or natural causes. It was by murder.

She died on a Saturday. I was alone at home with her, the only person to notice she was not moving for the last 15 minutes. I got a call from the emergency vet saying that she was still alive and they were going to rush her in. That was on Sunday afternoon. She died on Monday at around 12.30 PM.

I called her vet to ask how she was. He told me she was breathing very fast. I did not think much of it. I was only slightly worried. We were not in a rush to get her to the hospital. We could not afford it. We could not afford to lose her.

## gpt-neoX-20B

screenshot of generation: https://i.imgur.com/m0LzMxK.png

My dog died.. it was one of the worst days of my life.... so I'll try to help anyone that I can to make the process easier. I'm so sorry for your loss.

Re: I am moving.

You're moving from a small town/city (not too big or too small) to a big city. If you don't have a car you will most likely move by train and not by air. You might want to take the metro or bus to and from the metro in the big city you are moving to.

You are not too young to live alone - lots of people move from an apartment to a house alone (or vice-versa) in their 40s.


---

# smaller fine-tuned models

some examples from models trained on conversational data

## ethzanalytics/distilgpt2-tiny-conversational
80M params

generation screenshot: https://i.imgur.com/6K4c7rG.png
demo link: https://huggingface.co/spaces/ethzanalytics/dialogue-demo

	GPT-Model: Aw that's unfortunate, I have a golden retriever. Do you know much about them? they are such great dogs!

## ethzanalytics/gpt2-xl-conversational

1.5B params
Generation screenshot: https://i.imgur.com/K9MAAbD.png
demo link: https://huggingface.co/spaces/ethzanalytics/gpt2-xl-conversational

	That's really sad. how long have you had the pet? What was his/her name? " I want to get a digital map of where he or she went from and maybe start an online memorial service later on down this road.

===

if anything, perhaps to second-guess the sincerity of people in conversation if empathy is this easy to spoof ;)