---
title: "3 Hard Truths about Generative AI"
pubDate: 2022-11-23
description: "At Quickchat AI, we think and talk about Generative AI a lot. There are some things that rarely get mentioned in popular content on the topic."
author:
    name: 'Piotr Grudzień'
    image: '/blog-assets/authors/piotr_grudzien.jpg'
image:
    url: '/blog-assets/posts/hardTruths_bg.jpeg'
    alt: "Background image for 3 Hard Truths about Generative AI"
tags: ['insights']
---

## 1) Are you generating a draft or generating live?

These days, most users will be happy to generate text, image or video a few
times and then go with the _best one_. Or take a few different generations as
drafts and mix & match to get their final result.

That imposes very relaxed requirements on models. The general public often
evaluates them based on their _Best of N_ performance. Incidentally, that has
another interesting consequence - it has become a [challenge even for biggest tech companies](https://www.technologyreview.com/2022/11/18/1063487/meta-large-language-model-ai-only-survived-three-days-gpt-3-science) to do a big
successful launch of a new model because it will always be compared to cherry-
picked examples from the past.

It’s a very different game when things are to be **interactive** and images
must be good enough to go live immediately, e.g. in interactive ads, games or
entertainment. We know it very well at [Quickchat AI](https://quickchat.ai/)
where we get no second try on what our AI says. That determines how our AI
engineers spend a huge chunk of their time and that is - on _testing_.

‍

## 2) Why is nobody talking about testing?

It might be because most products these days are generating a _draft_ \- and
that’s what users are ok with (for now).

How do you write a test for _“is this image generated well enough”_? How do
you write a test for _“does this text sound Shakespeare-like enough”_? These
questions are probably not that important if the user is always patient enough
to generate a new draft.

Going back to the Conversational AI space, consider testing for _“is this
answer correct?"_ or _“did this conversational experience go the way we
intended?"_. It’s what we think about a lot at [Quickchat
AI](https://quickchat.ai/). Performance must be **tested, measured and
improved upon iteratively**.

‍

## 3) Are you solving a real problem?

Every person on the planet wants to play around with AI for a bit - generate
some images, talk to an AI bot, make a video of X singing song Y while Z is
dancing to it. And that’s a huge market, or rather, one with a huge initial
spike that may make anyone feel optimistic.

When all is said and done though, the product must solve a **real problem** or
else people will stop paying for it. Don’t confuse a user excited to try out
your product (even if they forgot to cancel their subscription) with one
excited to _actually_ use and pay for your product _in the long run_.

‍