Learning tip: using ChatGPT to aid understanding

[mikej]

Here’s a tip for adding another tool to your learning and understanding arsenal.

I’ve been using SSiW for a couple of months now and am up to Purple/Black and picking it up reasonably well - the spaced repetition works well for me. But the other thing that helps me is understanding why.

For this I’ve started using ChatGPT and just asking it questions like:

• Why do some verbs use dwi and some use wnes?
• For I didn’t, I have seen wnes I ddem and Don I ddem. Can you explain when to use which?
• Why are there so many ways of saying yes. So far I’ve come across do, oes, ydw
• Wnest t’ir cinio means did you make the dinner, but where in that sentence is make?
• So oes means yes, so how does oes da chi mean have you got?!
• The words like mae, oedd, byddain etc they don’t really have a literal translation do they? From what I can tell, they set the tense for the sentence. Is that right?

Here is an example of the thread I have open with it: ChatGPT - Welsh Teacher

It’s responses are helpful! It explains it’s answers clearly and offers further teaching. It’s been useful for the moments when I can say it, and I can work out when to say it, but I don’t really understand why I need to say it that way.

Mike

Edit: as has been pointed out and I should make clear, you can’t blindly rely on AI. It can and will make mistakes. You still have to use your brain and other resources.

[Hendrik]

Oh boy. Here’s a better learning tip: ask what you want here on the forum and you’ll get answers by real people with real knowledge, and you’ll get answers that you can actually trust.

The main problem with AIs such as ChatGPT is that they don’t understand language. Sure, its output sounds like it really knows stuff, but it doesn’t. The fact of the matter remains that sometimes, AIs “hallucinate”, and then you get answers that are ranging anywhere from half-true to utter word salad.

And unless you ask the AI only questions you already know the answer to, you have no way of weighing the veracity of the answer without consulting another source anyway.

I’d strongly urge you not to use ChatGPT (or Google translate, for that matter) as a language learning tool, because no matter how many answers it gets right, as long as it’s not right all the time, you simply can’t trust it. Good luck, and please remember we are here to help you.

[Hendrik]

I now actually took the time to look at the linked transcript, and it confirms everything I said above. Many answers are good, even the majority, but that is completely useless, as long as it comes up with nonsense such as

The first one is plain and simple factually incorrect. Do is the “yes” answer to any question in the past, if the main verb is in the “short form” past:
Est ti i’r sinema ddoe? – Do. – Did you go to the cinema yesterday? Yes.
(The correct answer in ChatGPTs example would be “Ydw”, by the way)
The second picture shows a construction that I have never seen before, but at least the part that “Ga” is a form of “gallu” is wrong. Ga i is the standard form to ask May I or Can I have, and as such it is a form of cael:
Ga i ddŵr? – Can I have water?
Ga i barcio yma? – May I park here?

Another egregious error: iddyn is not a form of bod, it’s an inflected form of the preposition i, to be used with nhw. And all given examples are complete nonsense.

There are still more errors, but I hope this selection is enough to deter you from trusting ChatGPT as a language learning tool in the future.

ETA, for further reading, here is an excellent article (not specifically related to language learning, but rather a general overview what LLMs are – and what not – and why they hallucinate):

Why DO large language models hallucinate?

[mikej]

Fair enough — it’s helpful to be corrected. I personally think saying it’s completely useless is a bit of an overstatement. As long as you’re aware it is capable of producing incorrect answers, and you use it alongside other resources, it can still be a helpful tool. I don’t recall suggesting it as a replacement for the forum or for other learning resources.

As a side note: opening your response with ‘Oh boy’ comes across as a bit patronising. I’m assuming good intent, but disagreement with a method doesn’t mean it’s inherently and absolutely terrible — it just means it might not work for everyone and might not be perfect.

[Hendrik]

The crux of the matter remains: as long as you don’t know whether the AIs output is true or not, you might as well just forego asking it at all. Ask here on the forum, or in the Slack group. Or read a grammar book.
It doesn’t help you the slightest bit if you are aware that it might produce wrong answers – you’d have to fact check every single answer with an independent and reliable source anyway. If you see worth in that, fine. I do not.

[suw]

It’s really important to remember that genAI does not know anything. It is a giant text prediction machine, so in essence it breaks down your question into tokens, analyses them, then spits out the most likely tokens that it has calculated would be in a response. It has no concept of ‘fact’ or ‘fact checking’ so it simply cannot be relied upon for anything factual.

There are also moral and environmental reasons to not use genAI. The vast majority of its training data was stolen from writers and artists without any recompense (the rest coming from uncopyrighted material). That makes at at the very best ethically suspect, at worse, completely unethical.

Secondly, it uses a huge amount of power and water to run queries, and we are at the point where we really, really need to deal with global warming and this is doing quite the opposite.

So genAI is inherently and absolutely terrible: it’s got no knowledge so will frequently feed you incorrect information and if you’re using it to learn then you will learn incorrect things; it’s unethical; and it’s environmentally damaging.

[mikej]

Alright. I’m going to request to delete this thread. What was intended as a helpful idea has spiralled into a political, social and environmental debate about whether AI is good or not.

[Hendrik]

I would respectfully request that this thread remain for other people, who think that ChatGPT is helpful for language learning. I am not saying that any and all forms if AI are inherently bad, but for the reasons stated, in my opinion there is no point in using generative AI in the context of language learning.

I took time – the better part of an hour – to prove to you where the AI is faulty, so that other people can form their own opinion. If you want to continue to use AI, I am not stopping you. I simply wanted to point out that it might not be as helpful as you make it out to be.

[mikej]

That’s fair @Hendrik. Leave it open. And you’re right, it’s important people are aware that it can make mistakes and you still have to use your brain and not just rely on it without challenge.

But please let’s not let it degrade into a debate about the ethical, environmental and social impacts of modern technology as @suw tried to. That isn’t the purpose of this forum.

[christopher-14]

A simple “be careful with AI, it can make mistakes” would have been enough instead of turning it into a political or moral lecture. This is a forum for people learning a new language, not a space to criticise or shame others for using a tool. Unless you are completely off the grid, none of us are without environmental impact, whether it is cars, phones, energy, agriculture, or anything else, so let’s not act morally superior.

Also, I have noticed a few people on this forum often respond to perfectly valid questions with sarcasm or condescension. If you cannot reply to someone’s language question in a positive and constructive way, perhaps it is better to say nothing.

[martin-harte]

Well, that escalated quickly.

I plan to use AI when I return to Chinese, which I hope to do soon. I’ve long wanted to try out the Arthur Cotton method; but it wasn’t financially feasible. AI makes it feasible.

Of course @Hendrik is absolutely correct that you mustn’t just trust everything AI gives you and that you should always verify its output with someone who speaks the language; but that doesn’t mean it’s not incredibly useful.

In my case, asking a friend to spend dozens of hours creating thousands of Chinese sentences of 2-4 words maximum and using only a restricted set of words is not a reasonable request. It would be incredibly boring and time-consuming for them and frankly not likely to be something they are all that good at. Paying a professional to make them would be much too expensive for me.

On the other hand, generating the sentences with AI and asking a friend to check them is perfectly feasible. Unlike creating sentences with a strict maximum length from a restricted wordlist (which is hard work), checking grammaticality is pretty effortless for native speakers and even a few thousand short sentences could be done in a reasonable amount of time.

So AI makes feasible something that wasn’t previously feasible. But of course, Hendrik is right that you have to get everything checked.

[Hendrik]

@mikej

Edit: as has been pointed out and I should make clear, you can’t blindly rely on AI. It can and will make mistakes. You still have to use your brain and other resources.

Thank you for conceding that point. But then let me ask you why ask ChatGPT in the first place?

Imagine you have a friend, let’s call him Bob. Bob is incredibly friendly, and incredibly willing to help. You can ask Bob any question, and he will give you an answer and he will even sound absolutely convinced that his answer is correct. The problem is: Bob can and will make mistakes, and he doesn’t even know it. And you don’t know how often he is correct, and how often he is not.

So you realize you can’t rely on Bob alone. You have to ask someone else, or refer to other sources. So why do you insist that asking Bob at all is a good idea? Why not skip asking Bob and stick to sources you can rely on?

I want to reiterate that there may well be use-cases where generative AI might prove valuable as a time-saving tool – like in the case @martin-harte outlined – but for the reasons I have explained, I hold firmly to my opinion that using ChatGPT in the way you did and proposed is not a good idea.

Please, don’t ask Bob, ask us, we are all here to help you learn.

[suw]

I stand by my post.

Anyone who wants to use genAI needs to understand the broader context so that they can make an informed decision.

It’s not political that genAI tools were trained on stolen content, it’s fact. It’s not political that they use huge amounts of power and water, it’s fact.

If someone wants to use genAI, they are free to do so, but it’s very important that anyone new to the technology can understand the situation so that they can make an informed decision about what fits with their own priorities.

I’d also add, if we’re talking about politics, that there are many, many people for whom the learning of Welsh is itself a political act.

Personally, I would find it difficult to learn Welsh and not be aware of the language’s politics and history. I’d go as far as to say that learners have a duty to be aware of them, even if they aren’t explicitly discussed here (although any discussion of Welsh language TV, for example, contains politics, and those have gone off here without a hitch).

[martin-harte]

I don’t necessarily disagree with anything substantive that anyone has said here; but if I was Mike J, I’d be feeling pretty hurt right now. He came in here excited to share something he finds useful and got every flaw in his method dissected followed by a political and moral lecture.

With Hendrik, I know that German culture is very different on this matter and that correcting someone often implies respect (I’m taking you and your idea seriously enough to discuss it honestly) rather than disrespect and I understand how important you feel it is to let any less experienced learners who might read this know about potential pitfalls. I can also see how passionately Suw feels about her political views, so I’m not criticising either of you here (as I said, I don’t disagree with anything Hendrik said and I don’t have a dog in the fight either way where the politics and morals of AI are concerned); I just want to publicly thank Mike J for his interesting suggestion (which I think could potentially be useful with a few modifications). I hope you still feel good about sharing it.

[christopher-14]

Suw, with respect, this is a forum for people learning Welsh (and other languages), not a pulpit for political sermons or moral lecturing.

You say it’s “not political” to bring up stolen content, fossil fuels, and climate change, but come on, these are hot-button issues. Dropping them into a thread where someone’s just using a tool to understand Welsh turns a language discussion into a battleground, and that’s not what this space is for.

Let’s be real. Everything we use - phones, laptops, even the servers keeping this forum alive, rrelies on rare earth metals, fossil fuels, and no shortage of ethical murkiness. If we’re going to draw purity lines based on environmental or moral perfection, we may as well unplug, toss our devices into a bog, and start carving ogham into driftwood.

It’s totally fair to say, “AI isn’t perfect, double-check its answers.” No one here disagrees with that. But framing it as an ethical crisis or scolding people for using a tool to help with their Welsh just shuts down conversation and makes people feel unwelcome.

There are plenty of places online to debate the ethics of AI. This forum isn’t that. It’s a place to wrestle with mutations, not morality.

Some of us are just trying to learn how to say “Ga i goffi?” without being told we’re complicit in the end of the world.

What you’ve actually done is create a heavy, uncomfortable atmosphere, the kind that pushes people away from the forum and maybe even from SSiW entirely. That doesn’t help the Welsh language. It helps it about as much as Putin helps world peace.

And let’s be honest: AI isn’t going back in its box. It’s here. Just like cars replaced horses, electricity replaced oil lamps, and missiles replaced cannonballs. Technology pushes forward with or without your approval. We can have thoughtful discussions about it, but not at the cost of driving learners away from the very thing they came here for.

[mikej]

@Hendrik thanks for your original response, genuinely. The post was constructive, albeit I felt the tone was a little off as I mentioned, but as I said originally I assume good intent.

@suw I’m not going to bite and devolve this thread into a debate. I don’t think this forum or my post is the place for it, but who am I to tell you where you can and can’t air your opinion. I’m not even in disagreement with you in some respects, I work in tech so I’m very familiar with AI as a technology and it’s impact - good and bad - and you’ve presented some fair points but, in my opinion, made a black and white argument for something which is far more nuanced. Hopefully you’ve made your point now and can hit the mute button, or please go and make your own thread instead of derailing this one.

@martin-harte @christopher-14 thanks both - appreciate your input! As you say, I was just trying to help making my first post in a forum which I thought was quite a friendly place.

I’m going to respond to @Hendrik follow-up question in a separate post as it’s a completely fair question and I want to address it.

[mikej]

Hey @Hendrik - fair points.

For me, a question pops into my head and I want to answer it - and if a tool like ChatGPT can give me an instant answer that can guide me a bit I find that really helpful. Especially because I can keep probing it. vs a forum where you wait for a response, then you respond, then you wait etc.

I know it won’t be 100% accurate, and I have to approach it with some caution. But, for example, the various information it gave me about the different tenses of bod was really helpful for me. And because I’ve been using SSiW and could place the why on those words that I’ve already heard and said many, many times was very helpful for my learning.

As you say it’s got some things wrong, but I like think that as I learn more from SSiW I’ll work out when it’s correct and when it’s answers are slightly misleading.

It’s not a replacement for proper learning, but it’s another tool in the learning arsenal that I find helpful.

If it’s not a good idea for you, or for anyone else, that’s fine. I’m not telling you to use it. But perhaps for some people, like me, it’s a useful tool. So I thought it useful to share my experience with it. And for what it’s worth, I think you were absolutely right to point out its flaws to warn people who might be less aware.

[Hendrik]

@mikej

For me, a question pops into my head and I want to answer it - and if a tool like ChatGPT can give me an instant answer that can guide me a bit I find that really helpful.

I understand where you’re coming from, but for me, the risk of getting a wrong answer far outweighs the perceived benefit of a quick answer.

Aside from this forum and the Slack group, there are also Facebook groups that you could join, and those groups literally never sleep, so you’d usually get an answer within minutes. (Then again, just because of the sheer number of people who are willing to answer, you also run the risk of getting wrong answers, but those are usually corrected quite quickly.)

And in all seriousness, there are a number of excellent books out there that might help you find the answer yourself, and I find that very rewarding in its own right. If that is an avenue you’d be willing to follow, I am sure we could come up with a list of titles suited to your needs.

[mikej]

@Hendrik fair - let’s agree to disagree

I didn’t know about the Slack group, I’ll probably jump on that. I don’t use Facebook because I despise social media (almost as much as some people despise AI )

I may well get into books later, but as much as I’d love the time to study properly a more-than full time job and a busy family life don’t grant me a lot of time so 1 - 2 hours of SSiW while getting ready / doing other chores and posing AI (and maybe now Slack) the odd question when I’m in need of a bit of guidance is about what I manage.

Given that dwi’n hapus iawn gyda faint dwi 'di ddysgu mewn amser byr

[martin-harte]

“I understand where you’re coming from, but for me, the risk of getting a wrong answer far outweighs the perceived benefit of a quick answer.”

This is a very interesting point. I must admit that I have different tolerances for different languages regarding this. Really, I’m just playing at Italian and Japanese and I have a high degree of tolerance for low-quality information (whether from an AI or just from Googling). I figure it will all work itself out in the end.

On the other hand, I have a very different tolerance level for languages like Chinese or German. Both languages where I hope to reach a high level one day. I don’t trust AI or Google and usually look for reputable or trustworthy sources (e.g. Gareth King for Welsh) exactly as you recommend.

Goodness, it seems weird to describe Italian and Japanese as “just playing” when I’ve logged 74 hours on the Japanese app and 146 hours on the Italian app since this thing was launched in December ha ha! I really am doing it just for fun and out of curiosity, though.