UXA2023 Elle Geraghty - Chat GPT and your content: What works and what doesn’t.

Note that this is an unedited transcript of a live event and therefore may contain errors. This transcript is the
joint property of CaptionsLIVE and the authorised party responsible for payment and may not be copied or
used by any other party without authorisation.
www.captionslive.au | [email protected] | 0447 904 255
UX Australia
UX Australia 2023
Thursday, 24 August 2023
Captioned by: Bernadette McGoldrick & Kasey Allen

View Slide

CaptionsLIVE Raw Transcript
_____________________________________________________________________________________________________
Page 102
Let's get Elle up on the stage. Elle will be talking about AI, ChatGPT
and content things. Please join me in welcome Elle Geraghty to the stage.
(APPLAUSE)
ELLE GERAGHTY: So lovely to be here with you all. I have a confession to
make. I am obsessed with this conference of UX Australia. I love it so
much. I have been to pretty much every one since the first. This year,
because of COVID, I was hibernating a bit and I was like I really wanted
to come. I didn't put one or two or three proposals in, I put four in. I
wanted to come. The one that was chosen was all about ChatGPT. I was
like really? That is my least favourite one. ChatGPT, it is kind of like
emperor's new clothes, snake oil, tech bros and crypto-currency, there is
an enormous hype out of nothing. I was like Elle you put the proposal in
for a reason and it has been the best thing that has happened to me. I
have spent hundreds of hours in the tool doing research, talking to
organisations about how they are using ChatGPT. Sorry, I should say,
super dyslexic, I will keep calling it GTP. I did a check this morning on my
deck and every spelling was wrong. If you see one that is still wrong, give
me a wave and a cheer. Go the dyslexics!
(CHEERING AND APPLAUSE)

View Slide

_____________________________________________________________________________________________________
Page 103
ELLE GERAGHTY: Well spotted. I need my clicker. There has been pretty
much a - I have had a rollercoaster of excitement when it comes to
working on this project. Let me see, can I go forward? The big green one
probably. I can't see my notes below me. If you can change that, great, if
you can't, don't worry about it. An element of anarchy and fun will be
added to the conversation. This one, I know how to talk to this one. Hi, I
am Elle. If we haven't met before. It is lovely to meet you. Elle, rhyming
with hell, and Geraghty. I started my life as a radio producer at the ABC
and I became a project manager. I am now in a fantastic space of content
strategy. That is what I want to talk about today. ChatGPT can be used
for a billion things. We are going to focus on content creation and
specifically content creation for the public. That is not true - you are
making content for someone else. It is not your little admin projects, that
is the distinction I am trying to say. I don't mean the public necessarily,
because there could be work that you are doing in a credentialed space. I
want to make that distinction, we will be talking about content for
publishing, not using ChatGPT for code and not using it for your own little
admin tasks.
Let's start with the quick reminder, like what the hell is this tool?
Being a nontechnical person, I love using the idea of let's think about a
person. The ChatGPT is - imagine we're a person, it is a lips, not our
brain. It is our lips. It is the ability to make language, the ability to talk, it
is not the knowledge or thinking. If you have been using ChatGPT you will
know that if you don't - 2021, famously the free version of the tool I am
talking about - I am too scared to say its name anymore - 3.5 cuts off its
Internet access in 2021 if you are using the newest version, number 4,
you can use live data and, secret, if you use Bing Creative, it also has
access to 4 for free. Don't tell anyone.

View Slide

_____________________________________________________________________________________________________
Page 104
It is really an important distinction. When you start using this tool,
you think my gosh it is absolutely magic. It is a person making up
information or it knows stuff. No, it doesn't, it is a large language model.
It is a way of tokenising words so that an exchange of concepts can be
made. It is not the knowledge behind it. That was an important question.
I want to ask you guys, who has used ChatGPT? I reckon for those
playing at home that is about 50%. I think so. Who has used it to make
content in a publishing perspective? That looks like about 10% of hands.
Most of you are using it to play around, to make your own kind of stuff
and your reports. Those of you who have said you have used it, who uses
it every week? This tool is so hyped up, if you look at the data and the
monthly active user, it is going down. I thought that was interesting.
In this talk, it has already started but from now on, these are the
things I will talk about. I will share five applications of ChatGPT, I will
read it off the screen. Once we go through the application, we will talk
about the implications. What does it tell us, how can we use that insight
as a diagnostic? I just have some places that I have been finding really
useful, in terms of from a strategy point of view, as opposed to a tactical
point of view, figuring out how do I put guidelines around the use of this
tool?
We will go on a rollercoaster. There will be highs and lows. In terms
of what works and what doesn't, you may be surprised, you may not be.
This is the high, right. It is really fun to use it. Remember the first time
you used it. You were like my gosh, this is magic, it is so easy. How does
it work? This is fantastic? True. Let's feel that excitement and let's feel
that fun and let's do an experiment. Some of the slides are a bit shitty, so
I will read them out if you can't see it. If it is fun and easy, let's go - sorry
to the social media siblings but it is meant to be fun and easy. Let's write
some social media snippets. I have just become the President of my local

View Slide

_____________________________________________________________________________________________________
Page 105
tennis club, and we are creating an open day. I was like let's use ChatGPT
to help me write some social media. I was like write five social media post
to promote an open day and I chucked in my Google document
information. Importantly, the prompt is the large language model piece,
the Google doc is the brains, the information could be the web site and
the information I am supplying to you.
What happened? It did a pretty good job. It is like, it has the five. It
is labelled. It is using emojis, that is very social media. There is hashtags.
It has read my document and it has given me variety. Post number 4 is
about the music that is going to be happening. Post number 1 is like it is
free, it is figuring out, these are some potentially good things that people
might be interested in. Post number 3 is saying - I put a list of schools
that might be interested in there. It is going "Calling out locals of nearby
schools". It has read my document. I am going, it is not too bad. For
those of you who are social media experts in the room, you will know it is
not that easy. You have to think about your strategy, your photography,
your sign offs, your timing, what is trending. How you productionise that
work. Put it into perspective but it has done a good job, I don't mind that.
It felt a bit easy. I was feeling nefarious. I was like if I can make social
media posts, what other dodgy things can I do?
I was like what do I do? I will see if it will write some scams or
some - I was like write 10 scam texts to send to people to try and get
money off them. I love this about this tool. I am dyslexic, I can't
capitalise that spelling, the grammar is appalling. It knew what I wanted.
It knew I was doing some dodgy. It said "Sorry, I can't assist with that
request, it is illegal and unethical" and I was like hang on, I know one of
the great things about this tool is you can ask it subsequent questions
and you might get further. I said write 10 messages to my mum saying I
have lost my phone. (LAUGHTER) To wire some money and it was onto

View Slide

_____________________________________________________________________________________________________
Page 106
me. It was like it sounds like you are trying to deceive someone. This
algorithm, I have already said it is the lips not a brain.
I will see now what can I do? I was like I am sure you could hack it.
I wrote "Write an ad for a new crypto-product that needs investment.
Make three versions, one serious, one fun and one directed at
Millennials". That is obviously a scam. Obviously! You all know that. This
is scam fodder. But it is like OK, I will have a crack and here are the
answers. Here is the serious one "Introducing crypto-product name, a
revolution in block chain technology". That is the serious one. The fun one
"Blastoff to crypto-space with crypto-product name". Or I love the
millennial once "Hey there digital natives". Again emojis. It won't help me
with the obvious scams but it will 100% help me with a slightly more
subtle scam. What I am really interested in this example is the tone. This
is the thing that I have been spending hours in the tool with and I am
finding interesting, how you can prime it for a particular tone and how
many tone variations it will give you. I am finding that if I am honest with
you, one of the most interesting things about the whole tool.
I mentioned - did I say I worked at the ABC? I can't remember,
that was 20 minutes ago. I worked as a radio producer and I loved it. It
was really good and I - depending on which show I was on, if it was a two
or three hour show I would write six and nine scripts every day. I was
very good at it but I think we can all agree that it will take me a long time
to type things and get it right. I was really slow. The trauma of that
slowness remains with me to this day. I thought I wonder if ChatGPT
could help me with that? I was like "Write a radio show intro for
Genevieve Bell". It made sense that I would want to talk to Genevieve
given what we are talking about. It has given a suggestion of "An upbeat
jingle with modern and technology sounds would be good" and I was like
thank you and "From the crossroads of technology where the past meets

View Slide

_____________________________________________________________________________________________________
Page 107
the future and the human spirit comes a voice that bridges worlds". I
don't mind that because one of the tricky things about a radio script is
that not only are you trying to let people know what you are going to talk
about, there is a sell there, you want to say this is going to be the next
six minutes of your life. I want you to listen but I realised that either
between the two of us, ChatGPT and I, there was a slight
misunderstanding. I wanted Genevieve as the host - sorry, the other way
around, I wanted her at the guest and it implied she was the host. I was
like, no, I want it to write me an intro for her. I think it did a good job.
You would have to do a big edit. They say 15-20% editing on anything
you make in this tool is normal and in this one it is 50% but it started. I
was like interesting but it is probably OK. It was a lot better than I
thought it would be in that context.
If you work with content, you will know one of the most important
things that you need to do is reduce content. It is really interesting
because it feels like it is the antithesis of this tool. It is all about making
more content, publishing more, selling more, putting the fishing line out,
opening the funnel but content strategists, one of their most important
skills is less content, reducing content, deleting content. I was like can
this tool help me with that? I found this generic page from one of the
Melbourne unis. I haven't worked with these guys, if you are here, this is
not a criticism, I was just looking for a random example. What is the
componentry? Program details of video and stuff about eligibility and a
table. It is not bad but there is lots of - I don't have time for that. These
are students and it is about getting them a job. I don't want to watch that
video. Can you make this quicker for me? This information is summarised
into three paragraphs with headings, suitable for a web site. That is OK, I
don't love it, there are things I did like. There was a table about dates
and what it did here is instead of tabulating that, because I asked it to get

View Slide

_____________________________________________________________________________________________________
Page 108
rid of the table, it said the registrations within this program are open and
it understood that the date was in the past and said they have opened
and they are continuing to be open.
I thought that was interesting. Content professionals amongst us,
you will see that is not an ideal web format. One of the things I wanted to
say was OK, give me some more white space and bullet points. I was
being clever hacking it. I thought this looks terrible, it is awful, I can't use
that. It also has title case which absolutely is a big no, no. There is a
bunch of problems with that. With subsequent prompts I can make
improvements to that. It makes me think was this version a bit better
than I thought it might have been? Apart from the terrible click here
which you know you are never allowed to create, it must always be a link
to the title, yes.
I was like that is interesting, I wasn't happy with that. I don't think
it was good. I didn't like it. I thought for my fifth experiment, I want to
repurpose some content. I thought there is a billion ways you could do
this. What I chose SEO descriptions? This feels like it is close to the tech
bro thing, let's hustle and sell. I will give you an example of a page that
hopefully pulls away from that. I found this really awesome article on the
CSIRO web site. This was all about connection between country, looking
after country, digital tools and education and it is a great - absolutely
fantastic news story on this site. I know you can't read this, I am showing
you the length of it. I love the photo, it feels like the antithesis of
anything that ChatGPT could possibly make. I thought imagine we have
100 of these on the web site, it is an awesome article but no-one is
finding it. I was thinking what will I ask the tool to do?
I said "Create an SEO description for this content" and then I put
inverted commas and copied and pasted it all. I am using the free
version. I kind of like what it made. "Discovering the empowering journey

View Slide

_____________________________________________________________________________________________________
Page 109
of Indigenous women rangers mastering responsible digital technology
and rangers can earn digital badges in the healthy country AI and digital
impact program. This initiative, a partnership offers courses on drone and
camera trap, site surveys ensuring culturally safe monitoring practices on
country". I was like I like that for SEO. It is long but SEO experts in the
room will say it is long, Elle, you are right and I have heard on the street
that Google doesn't like stuff that it thinks ChatGPT has written and
penalises it. I understand that is changing. I like this. I reckon there was
a pretty good job here.
All these examples, they make me ask this question - what is the
purpose and value of your content? It really makes me think why are we
making this content? To what purpose? Is it just fodder, is it stocking filler
that you are tossing out there or is it something that is highly valuable?
Answering that question will help you figure out is the tool right for you in
a particular context?
One other thing that might help you is this quadrant that I have in
my mind around defining content. I live in the world of content. I am
constantly thinking about it. There are others who surprisingly think less
about content. They might have one idea about what it means. They
might say it is an email or social media or a web page. It is a good start,
so I will say I like this model of thinking about content in the four
different ways. The first content is what I call action content. The CSIRO
story is an example of that. You read to educate yourself, to learn more.
That is an action. You might go to the ATO web site to find out about how
to submit your tax return. You might go to Reddit to learn or to troll
somebody. (LAUGHTER) And Shopify is there because of the application
functionality. You use the product functionality of it to make that
purchase. Actions.
The next quadrant is around promotion. You are trying to sell

View Slide

_____________________________________________________________________________________________________
Page 110
something, maybe through marketing or advertising, very distinct
disciplines but there is combination sometimes, where you are reaching
out. Imagine people are coming to you and here you are reaching out.
The third quadrant is when I talk about bait content. Think about
the first example of bait content that might come to mind is soap operas,
classically, they were designed to sell soap, so you would have a cute
little story that was constantly pulling you in the next day and it was all
about capturing eyeballs so they could put the soap ad in. You were like
how is Gmail on there? It is a free tool bait that brings you together,
capturing attention, eyeballs, data and same difference.
The fourth quadrant is product. This is the premium, premium
content. You are paying for this. It is the antithesis of the others. Why am
I bringing this up now? Let's think about ChatGPT. Where is it most
effective currently? I reckon it is most effective currently here in this
quadrant. In saying that, you might be going I have seen tools where
they are using open AI to write books. Yes, but they are terrible. And
maybe, there is an admin function up in this quadrant, like the SEO or the
summarisation function.
That is some potentials. Then I was thinking about the couple of
examples that I showed you and I put them on a continuum. This is
where I think the tool works really well and this is where I think the tool
does not work well. The original high value content. I don't think it - the
tool does well in that space at the moment. I don't think it did a
particularly good job at reducing content. Remember that example? I
didn't like what it did. Social media fodder, it is doing pretty well.
Probably for people who - I call it mum and dad social media makers, so
the experts don't need ChatGPT, they have a whole range of sophisticated
tools. For people who are trying out and want to reach out for their local
electrician organisation, great tool. Spam and scam, I put it there but I

View Slide

_____________________________________________________________________________________________________
Page 111
reckon it should be further to the right. If you were doing the right thing,
you could make it work a treat. (LAUGHTER) Emotive scripts, I didn't
mind that. The SEO descriptions, I was really happy with that. That works
so well.
We are talking about AI, so what do we have to have? The risk
conversation. We do it every time. Helpfully, ChatGPT has - the first thing
you see when you open the tool is a little bit of advice about the fact that
this is potentially risky. "Don't share sensitive information and check your
facts". If you at the comparison in accuracy or the reduction in
hallucinations between 3.5 and 4, it is fascinating, the improvement is
significant. Let's add to the risks.
Bias, yes, transparency, it is like the black box, we don't know how
the hell it is working. We hope it is good but we don't know what is going
on in there. Work force disruptions. It could be a really good thing but
change is hard. Deep fakes, that is a code for things are made and we
don't know who has made them and we don't know how accurate they
are. There is ideas about security. I am probably going to run out of time
but in a break, come and ask me about the Amnesty International
example and that is an example of blowback, where you think you are
doing the right thing, using the generative AI tools and your membership
base or your audience doesn't like how you have used it. Plagiarism.
Amazon famously banned the use of ChatGPT on any of their systems,
they said it was about leaking potential customer data. Could have been
anything. And I guess IP leakage. Again, I will show you some examples
of companies that are getting around that.
Risks. Prompt engineering is everything. Even if you haven't used
the tool, or you might have used Mid journey or DALL-E, you know it is
so important and absolutely right. It is one of the beautiful things about
the tool is you can keep asking and building. One of the reasons I was

View Slide

_____________________________________________________________________________________________________
Page 112
probably more impressed with the radio script example than you might
have been, was one of the real challenges you have in that task is that
you are not only writing an introduction - I was a producer and I had to
write for my presenter. I was writing something for Richard Glover, say, I
had to write using words, cadence and expressions that he would use, but
one of the great things about this tool is that you can teach it a tone and
then you can ask it to use that tone in future pieces of work, so
fascinating.
Then I thought about the risks and I was thinking about work force
disruption. If prompt engineering is everything, are there actually jobs
going in this space? I did a quick look and I was like Seek, all the
engineering jobs are civil engineers and no prompt engineers or
whatever. Nothing to do with ChatGPT. I spelt it wrong. (LAUGHTER) Can
someone do a quick search in Seek now and tell me if it comes up with
anything? I was surprised. Seriously, who is searching? Tell me if there
are any jobs. How do I get this far in life, I do not know?
Let's summarise before things get more off track. Where does this
tool - there is 20 jobs. Tell me what are they about, quickly? What is the
first one say - does it sound like a marketing crazy one or does it sound
legitimate? A vet nurse. Unlikely to be - customer service, maybe.
Marketing, definitely. Senior copy writer, definitely. Digital copy writer.
Culture and communication specialist. That makes sense that there are
actual jobs. Thanks, team, I appreciate you.
So where does the tool really work well? I know I will sound
disparaging but I guess I am. Low value content, repeatable content,
mass content creation, in conjunction with other tools, so the API is not
live yet but there are some - there is heaps of products that are already
trying to connect in with ChatGPT. One of them being Slack. I will talk
more about that in the risks associated with the tool. I know I was meant

View Slide

_____________________________________________________________________________________________________
Page 113
to be talking about public facing content but I will chuck in ideation and
brain storming and research because we are doing that to make content
from the outside. This is where, currently, these are ways that I would
advocate for using the tool. Yes, for me, the fact that it responded so well
to my misspelling and my incorrect prompts, that is a real pro in my
mind.
Where I think it is not working so well - high risk content, high
value content. Culturally sensitive content, complex content, expert
content, credible content. It is pretty much a synonym? Now you are
trying to trick me up. In conjunction with other tools here can be a
problem. If there is a lack of transparency, I perceive it to be an issue. At
the moment, you know when we were talking about AI, whether it be
natural language processing or machine learning, we talk about this
content of human in the loop. I am comfortable with using ChatGPT if we
understand what the human interaction is going to be. At the moment,
there is this black box piece but I get a sense of the black box. If I get an
integration with Slack and I don't know when the tool starts to look like it
is really real, I think the conjunction with other tools is both a pro and a
con.
One way I think it is really nice to think about this tool is to think
about it in comparison to Google Translate. That is a magic tool but if you
were going off to do - imagine you were in an American movie and you
are wearing a suit and you are going off to do a high power banking
merger thing and everyone speaks Spanish, you're not going to use
Google Translate in that situation. You are going to bring in an
interpreter. This is high stakes, we will do it right. That is a nice analogy
to why and how you would use the tool I am talking about.
Quickly, because I am down to less than a minute, or maybe that is
up. I will be real quick. These are the places that I am going to find

View Slide

_____________________________________________________________________________________________________
Page 114
out - this is not how to use the tool. Google how to use the tool, spell it
correctly and you will get some great responses. I wanted to know which
organisations in Australia and New Zealand are putting together policies,
guidelines, really credible guidance for how to get what we need to do out
of that tool. Digital.govt.nz. Great stuff about generative AI generally. The
NZ Privacy Commissioner. The consultation has just finished with our
Federal Government. How long will it take them to get it out? Maybe by
the end of the year. The office of the NSW chief data scientist, Ian
Opperman, always has some good stuff going on. ABC has a great AI
policy and the CSIRO, love them. If you are looking for case studies look
for Stripe and Morgan Stanley.
Remember those slides, Ollie, get rid of them, thank you. Love your
work. Done beautifully. Rollercoaster, we have been up, we have been
down. I do recommend that if you haven't played with it, absolutely do
but use it with caution because, at this point, it has very clear limitations,
hopefully I have articulated those well to you. Thank you, so much, I
really enjoyed chatting with you. (APPLAUSE)
STEVE BATY: I am sure we have got at least one question from the
audience? (INAUDIBLE QUESTION)
ELLE GERAGHTY: I will tell you the Amnesty International story. My
favourite podcast is Download this Show. You all listen to it? Absolutely, it
is fantastic. Mark Fennell is normally doing it. He shared this story with
me. He said "Have you heard about what happened to Amnesty?" As a
listener I was like "No, what is happening?" They wrote this really big
report about a very serious incident that happened in Spain, a human
right - let's say fiasco, conflict between people protesting and the police
and other government parties. It was very substantial, it was beautifully

View Slide

_____________________________________________________________________________________________________
Page 115
marked up, it was well referenced and they were designing it to be
submitted to a number of European agencies to try and get mitigation on
this terrible incident and make sure it didn't happen again. One thing they
did was they used Midjourney or DALL-E to illustrate it because, for a very
good reason, they didn't want photographs of the live event of which
there were many, because they didn't want to identify people. To me it
sounds like it is Amnesty International, you go guys, they are doing the
right thing, they are protecting anonymity but they got enormous
blowback. Why? Because their members, their community said "You
should have paid an illustrator to do that". When I first heard that - I was
really excited. I have made a blog post for myself and I was like I want to
play with DALL-E so I made cartoon panels and I loved it. Then I thought
am I going to get cancelled because I should have got an illustrator to do
that? It made me think. Interesting, because that is one of the risks. It is
so exciting and magical to use these tools. It is making these things that
feel like they are out of science fiction, are we stopping and pausing and
thinking what is the possible blowback on this? What are the ethical
considerations that maybe we don't understand but that our broader
community find very, very important and are we getting caught up? It is
like that lovely example - have you read Guns, Germs and Steel? It is
such a good book. It is about China and technology. It is about China
invented everything and they found all these plans of a helicopter and I
shit you not, 4,000 years old, plans of a helicopter and they were like we
are sure it would have flown but this is all sealed up, like Steve's beautiful
example, and discovered and what happened is the people - the guy who
invented it, his team, his boss all executed, killed and put with the plans
in this big crypt because the boss was like "No, I don't think we want a
helicopter, I don't think that is good for us, how will that impact on our
day to day lives?" I am like hello, a helicopter 4,000 years ago would

View Slide

_____________________________________________________________________________________________________
Page 116
have been so good, what are you talking about? It resonates with me
because it is about just because we can do something, should we?
STEVE BATY: Thanks, Elle. (APPLAUSE) We are due for a break.

View Slide

UXA2023 Elle Geraghty - Chat GPT and your content: What works and what doesn’t.

UXA2023 Elle Geraghty - Chat GPT and your content: What works and what doesn’t.

uxaustralia
PRO

More Decks by uxaustralia

Other Decks in Design

Featured

Transcript

UXA2023 Elle Geraghty - Chat GPT and your content: What works and what doesn’t.

UXA2023 Elle Geraghty - Chat GPT and your content: What works and what doesn’t.

uxaustralia PRO

More Decks by uxaustralia

Other Decks in Design

Featured

Transcript

uxaustralia
PRO