Reviewing ways of analyzing speech without writing it down

The goal was to identify accessible ways to assess communication of people with aphasia in the clinic and next steps to improve the process.  

There is one main research question and five subquestions. 

The main research question:  What tools or procedures are available that don’t require every word spoken to be written down and assess communication abilities of people with aphasia? 

The five sub-questions:

1. What types of measurements are taken with these tools?
2. What information is being measured?
3. What are the characteristics of the people with aphasia included in the studies?
4. What are the characteristics of the researchers in the studies reviewed?
5. How well does the method work and how consistent is it? Has whether it could be used in a clinic been investigated?  

A scoping review, which is a review of published papers using specific search terms.

Who the Tools Are Used On:

• The tools were tested on people with aphasia who spoke different languages, including English, German, and Swedish. Most were adults recovering from strokes, with different levels of severity in their language difficulties.

Who Used the Tools:

• Naive raters: Regular people with no special training.
• Learners: Students studying speech therapy.
• Experts: Speech therapists and researchers.
• Most tools were used by trained raters, but the article points out that these tools could be made easier for everyday use.

Strengths and Weaknesses of the Tools:

• The tools work well for analyzing different parts of speech, but many don’t fully cover the bigger picture of communication. For example, they might miss how people use body language or tone of voice.
• Some tools are better for short tasks, like storytelling, while others work better for conversations.

Overall conclusions:

• The researchers also found that the tools need more testing to prove they’re accurate and reliable.
The article suggests that transcription-less methods are effective in determining the communication abilities of participants with aphasia, which makes it much simpler to treat their needs.
• These tools are still new, and researchers need to make sure they are reliable (give the same results every time) and valid (accurately measure what they claim to measure).
• Therapists also need more training to use these methods effectively in their practice. Few of the tools have been used in clinical settings, however. 

When researchers study the language use of people with aphasia, a lot of the time the measurements are made outside of everyday contexts. A lot of researchers want to measure language use in natural contexts but face a lot of problems when trying to do so. One big problem is the lack of tools to take measurements. Another big problem is that the time it takes to write down everything someone says and analyze it takes too long for a clinician to do. Despite this, analyzing discourse is important because it gives information about language structure (for example, grammar) as well as language use (for example, if the language used is appropriate for the situation). 

Speech pathologists / therapists have limited time and resources. Identifying effective ways of analyzing discourse without having to transcribe it would potentially address these barriers. 

21 research articles were reviewed by researchers to identify ways to measure communication of people with aphasia. To be included, the article needed to feature a way to assess communication based on the listeners interpretation and without the need to write every word said. Researchers looked through the articles to find five things:  1. The type of tool being used.  2. The type of information that is taken from the process.  3. The characteristics of the people with aphasia in the studies being reviewed.  4. The characteristics of the people using the tools.  5. How well the tool measures communication and how consistent it is.  

No treatment was evaluated for this article. The article provides details about the assessments used: https://osf.io/af8zk/ 

Researchers are trying to find easier ways to study how people with aphasia communicate in everyday life. Right now, transcription takes too much time and isn't practical for most clinics.

The tools they study aim to measure many aspects of language, like how clear someone’s ideas are, whether their sentences are grammatical, and if they stay on topic during a conversation.

Using these tools can help therapists focus on real-life communication skills instead of just testing isolated abilities, like naming objects.

There were no risks involved.

This is a review, which summarizes many studies. There were speech therapists, graduate students in speech therapy, and naive raters (those with no training in speech therapy) included in studies as raters. There were persons with aphasia, mostly in the chronic stage, included in the studies as participants.

This is a review, which summarizes many studies. There were speech therapists, graduate students in speech therapy, and naive raters (those with no training in speech therapy) included in studies as raters. There were persons with aphasia, mostly in the chronic stage, included in the studies as participants.

A scoping review is a type of research that helps to map out what is known about a topic. Instead of trying to answer a very specific question, it looks at the "big picture" by exploring a broad area to identify gaps, trends, and key ideas. In this article, the scoping review was used because there is limited research on transcription-less tools for aphasia. It helped the researchers map out the current tools, understand their strengths and weaknesses, and suggest what needs to be done next.

The scoping review took place from 2022-2023.

The review included studies from across the world. The review primarily included studies that analyzed English speakers, but it also included some studies involving speakers of German and Swedish. These were the main languages represented in the studies reviewed. However, the authors noted that there was limited diversity in the linguistic and cultural contexts, which could affect how well the tools generalize to speakers of other languages.

No funding was used for the current study.

The current study highlights several challenges or problems with the research on transcription-less tools for analyzing speech in people with aphasia. These include:

1. Limited Testing of Tools

Many of the tools studied haven’t been tested enough to prove they are reliable (producing consistent results) or valid (measuring what they are supposed to measure).

Without strong psychometric data, it’s hard to trust the tools for clinical or research use.

2. Focus on Specific Features

Most tools focus heavily on linguistic and propositional features (like grammar and clarity of ideas), but they often miss out on important aspects like:

Macrostructural features: The overall flow and structure of speech.

Pragmatic features: Real-life use of language, like taking turns in conversations.

Non-verbal elements: Gestures, facial expressions, and tone of voice.

3. Time and Training Barriers

Some tools are still too complicated or time-consuming for everyday use by clinicians.

Therapists often lack the training or resources to use these tools effectively, which limits their implementation in real-world settings.

4. Small and Homogeneous (similar) Samples

Many studies included only small groups of participants, often from similar language or cultural backgrounds (e.g., mostly English speakers).

This makes it hard to know if the tools will work well for diverse populations or languages.

5. Rater Variability

The people using the tools (raters) ranged from naive listeners to trained experts. This variety can lead to inconsistent results, especially since not all tools are designed to work with untrained raters.

6. Limited Generalization

Some tools are designed for very specific tasks, like storytelling or describing pictures, and may not work well for other kinds of speech, such as casual conversations or interviews.

7. Unclear Definitions and Methods

Terms like transcription-less analysis and perceptual analysis are not used consistently across studies. This makes it harder to compare results or replicate the studies.

8. Gaps in Psychometric Testing

Few studies reported data on key psychometric properties, such as:

Inter-rater reliability: Do different raters score the same sample similarly?

Test-retest reliability: Do the same scores occur when tested at different times?

Validity: Does the tool accurately measure discourse ability?

9. Limited Clinical Relevance

Many tools were developed in research settings and may not address the practical needs of clinicians, such as ease of use and the ability to assess functional communication (e.g., conversational skills).

Yes. All of the studies included received ethical approval to obtain their information.

Conduct larger, more diverse studies to test the tools in real-world settings.

Improve training materials and resources for clinicians.

Develop tools that are simpler, faster, and more focused on everyday communication.

Provide more detailed psychometric testing to ensure tools are trustworthy.

The full article can be found at: https://doi.org/10.1111/1460-6984.13028