Learning with AI

ChatGPT and other LLM tend to disallow people to think and learn. My main concern is, if you use LLMs all the time, you run on auto pilot, and you tend to not learn and not think anymore (see more on Will AI Replace Human Thinking).

Here’s a study The effect of ChatGPT on students’ learning performance, learning perception, and higher-order thinking: insights from a meta-analysis by Jin Wang & Wenxiang Fan which tends to say otherwise:

• A large positive impact on improving learning performance (g = 0.867)
• A moderately positive impact on enhancing learning perception (g = 0.456)
• A moderately positive impact on fostering higher-order thinking (g = 0.457)

Rather than reducing thinking, the evidence suggests ChatGPT can actually enhance it when used appropriately.

The analysis shows that ChatGPT’s effectiveness in fostering higher-order thinking varies by:

1. Type of course - with the strongest impact in STEM-related courses
2. Role of ChatGPT - with the most significant impact when it functions as an “intelligent tutor”

This higher order thinking is something I could understand, if you deeply interested in something, you can actually learn more by asking ChatGPT. But if you always do it, I would argue that over a long time, your brain won’t learn to think/learn anymore. As we didn’t learn to remember phone numbers anymore, as we don’t train that anymore.

But the questions is: How long was the study applied too? Because I believe, the effect comes only true if you used LLMs for months and months, as your thining pattern changes, as we don’t train the muscle of learning/thinking anymore; but instead just type into ChatGPT.

The studies were categorized by duration as:

• Less than 1 week: 8 studies
• 1-4 weeks: 12 studies
• 4-8 weeks: 17 studies
• More than 8 weeks: 7 studies

The majority of studies (36 out of 44) were conducted for 8 weeks or less, with only 7 studies examining effects beyond 8 weeks. This is a limitation when considering potential long-term impacts on thinking patterns in my opinion.

when the usage period exceeds eight weeks, the positive effect of ChatGPT on students’ learning performance slightly declines

This is what I mean, that the prolonged use of ChatGPT can lead students to become overly reliant on the AI tool, and thus they may neglect to reinforce the knowledge they have learned.

Tweet, Full article and source paper.

Cal Newport is also analysing this trend on Overstimulation Makes You Dumber | Cal Newport - YouTube. More on Will AI replace Humans.

Note-taking and Its Impact on Learning, Academic Performance, and Memory

Related: [[]]

• Your Mystery: Have Attention Spans Been Declining? - AI Summary:
  • The article investigates the widely discussed mystery of whether individual attention spans have declined since 2000, particularly influenced by the internet and social media.
  • Despite widespread concern, direct, long-term research on individual attention span decline is notably scarce and often lacks robust methodology.
  • Attention is categorized into sustained, selective, and alternating types, with Continuous Performance Tests (CPT) identified as the most promising, though not universally agreed upon, measurement instrument.
  • Indirect evidence, such as Gloria Mark’s research showing decreased screen duration (from 150 to 47 seconds on average before switching) and James Cutting’s findings on shortening film shot lengths, suggests a decline in sustained attention and increased propensity for task-switching.
  • The author concludes that a decline in individual attention spans is “likely (~70%)” but emphasizes the need for dedicated, well-designed longitudinal studies, proposing inexpensive methods like repeated CPT administration.

Origin: Marcel Salathe on LinkedIn
References: Don’t use AI for everything, you stop thinking-learning AI Use
Created 2025-05-13