Evaluation of the usability, content, readability and cultural appropriateness of online alcohol and other drugs resources for Aboriginal and Torres Strait Islander Peoples in New South Wales, Australia

Abstract

Objectives

This study aimed to analyse the usability, content, readability and cultural appropriateness of alcohol and other drugs (AODs) resources for Aboriginal and Torres Strait Islander Peoples in New South Wales (NSW), Australia.

Outcome measures

The content of 30 AOD resources for Aboriginal and Torres Strait Islander Peoples was analysed according to the following criteria: general characteristics; elements of graphical design and written communication; thoroughness and content; readability (Flesch-Kincaid grade level (FKGL), Gunning Fog index (Fog), Simplified Measure of Gobbledygook and Flesch Reading Ease); and cultural appropriateness.

Results

Most resources displayed good usability, depicted by the use of headings and subheadings (n=27), superior writing style (n=19), relevant visuals (n=19) and use of colour support (n=30). However, some resources used at least one professional jargon (n=13), and many did not provide any peer-reviewed references (n=22). During content analysis, 12 resources were categorised into the alcohol group and 18 resources in the other drugs group. Impact of alcohol during pregnancy and breast feeding (n=12) was the most common included topics in the resources related to alcohol, while the physical impact of drugs (n=15) was the most discussed topics among the other drugs group. Based on the FKGL readability score, 83% of resources met the recommended reading grade level of 6–8 by NSW Health. Many resources (n=21) met at least half of the cultural appropriateness elements of interest. However, less than one-third were developed in collaboration with the local community (n=9), used local terms (n=5), targeted the local community (n=3), included an Aboriginal voice (n=2) and addressed the underlying cause (n=1).

Conclusions

Many AOD resources are developed specifically for Aboriginal and Torres Strait Islander Peoples, but their usability, content and readability differed, and they were not culturally appropriate for all communities. Development of a standardised protocol for resource development is suggested.

Correspondence to Professor Amit Arora; [email protected]

STRENGTHS AND LIMITATIONS OF THIS STUDY

• This study has used rigorous methodology to conduct a comprehensive appraisal of the resources based on a wide range of criteria: resource characteristics; elements of graphical design and written communication; thoroughness and content; readability; and cultural appropriateness.

• Four commonly used and validated indices, Flesch-Kincaid grade level, Gunning Fog index (Fog), Simplified Measure of Gobbledygook and Flesch Reading Ease), were used to evaluate the readability of the resources.

• The evaluation methods used in this study were developed in discussion with a steering group with expertise in alcohol and drugs (AODs), digital communication and codesigned with an Aboriginal and Torres Strait Islander representation.

• Only resources written in English language were included in this study, which may exclude resources that are written in local language and developed specifically for a local community.

• Thoroughness and content between AODs groups differ depending on the commonly discussed topic, which may exclude some elements that may influence the intended readers’ opinion on harmful AOD use and treatment available.

Introduction

Harmful use of alcohol and other drugs (AODs) is a global public health issue, that is, responsible for many preventable chronic illnesses, hospitalisations and unintentional deaths.¹ According to the WHO, alcohol consumption is responsible for at least three million deaths each year and harmful use of alcohol accounted for 5.1% of the global burden of disease.² In 2019, the United Nations reported that drug use disorders were responsible for half a million deaths and 18 million years loss of healthy life.³

In Australia, recent data show that 7.5% of the total disease burden is attributable to AOD use disorders.⁴ Moreover, this issue disproportionately affects certain population groups in Australia, such as Aboriginal and Torres Strait Islander Peoples and individuals with mental health conditions.⁵ In 2018, harmful use of AOD collectively contributed to 17.4% of the total disease burden among Indigenous Australians, almost 10% more than non-Indigenous Australians.^{4 6} In this paper, the term ‘Aboriginal and Torres Strait Islander Peoples’ and ‘Indigenous Australians’ refer to the First Nations Peoples of Australia. The term ‘Indigenous peoples’ refers to First Nations Peoples globally.

Aboriginal and Torres Strait Islander Peoples comprise 3.3% of the Australian population.⁷ While the gaps in burden of disease between Indigenous and non-Indigenous Australians have decreased in recent years, Indigenous Australians still experience worse health than their non-Indigenous Australians counterparts.⁶ Unfortunately, these gaps in health are likely to start before birth and remain throughout their lives.⁶

For Indigenous Australians, the concept of health is not limited to physical, mental and social well-being, but also cultural, spiritual and ecological well-being of the individual and the community.⁸ Factors posing risk of harmful use of AOD among Aboriginal and Torres Strait Islander Peoples are associated with psychological distress from ongoing impacts of colonisation and intergenerational trauma, substance use by peers, family members or partner, and availability of substances.⁹ In addition, the cultural connections of Aboriginal and Torres Strait Islander Peoples are often misunderstood, misinterpreted or undervalued in the current policies and approaches in Australia, which may contribute to systemic discrimination, isolation, social exclusion and limit the access to healthcare.¹⁰ On the other hand, health knowledge, cultural engagement and supportive environments are protective factors against AOD use disorders and its related harms.⁹

Health education, through evidence-based written materials (such as leaflets and posters) and verbal communication, is a vital part of the AOD demand reduction initiatives.^{5 11} However, health literacy plays a fundamental role in understanding such information.¹² Health literacy is characterised as the ability to obtain, read and make sense of health information and use the health information in ways that promote and maintain their health.^{13 14} This ability is essential to understand and complete healthcare forms, such as consent forms or other health-related resources, and to make decisions and manage one’s health.^{11 15} The inability to read and understand resources can have a negative effect on health outcomes and quality of care provided by family or carers.^{16 17} Individuals with low health literacy are less likely to be involved in their health management, thus affecting their access to healthcare and quality of care.^18–20 The 2018 Australian National Health Survey reported that 17% of Australians were unable to appraise health information.²¹ It is suggested that Aboriginal and Torres Strait Islander Peoples may be at higher risk of low health literacy as educational attainment and school-based numeracy and literacy scores have been found to be significantly lower among Indigenous peoples compared with the general population.²²

The concept of health literacy has been broadened beyond an individual’s capability, into a more complex interaction with the healthcare systems (eg, healthcare providers, health policies, health education and resources).^{14 23} For instance, healthcare providers are recommended to refrain from using medical jargon when delivering health information to aid the patient’s understanding. Prior research noted that the mode of information, such as spoken and written information, may influence a patient’s retention.^{12 24} While patients with low literacy may struggle with written information, it has important benefits compared with other modes of information. Written information may promote treatment adherence as it remains available for later reference and for review by family and carer.^{12 24} In addition, including visuals aids or informative images may improve usability and quality of written information.^{24 25}

To date, the evidence on the readability of AOD resources for Aboriginal and Torres Strait Islander Peoples appears to be limited. Studies conducted previously mainly focused on the readability analysis for the general population and were not specific to AOD resources.^{16 26–28} Thus, analysis of AOD resources for Aboriginal and Torres Strait Islander Peoples is necessary to identify important elements when developing resource materials, which will help future health resource development and potentially prevent and delay harmful AOD use as well as to reduce the gaps in health outcomes between the Indigenous and non-Indigenous Australians. This study aimed to analyse the usability, content, readability and cultural appropriateness of readily available online AOD resources for Aboriginal and Torres Strait Islander Peoples in New South Wales (NSW), Australia.

Methods

A desktop search was conducted between October and December 2021 to obtain all AOD resources that were readily available online in NSW. These resources were obtained from key organisations working in the health and well-being of Aboriginal and Torres Strait Islander Peoples or key organisations that offer support and information services about AOD. Initial search terms included “alcohol”, “drugs”, “Aboriginal and Torres Strait Islander Peoples” and “Indigenous Australians” to capture all relevant resources.

Eligibility criteria

The eligibility criteria for this audit were as follows:

Inclusion criteria

• Written in English language.

• Resources developed specifically for Aboriginal and Torres Strait Islander Peoples with or at risk of AOD issues and/or their carers, families and communities.

• AOD resources produced by governmental and not-for-profit organisations in NSW.

• AOD resources which are readily available via the internet.

Exclusion criteria

• Resources that do not contain key messages related to AOD use and/or its management.

• Resources that only include information on how to access AOD service.

Development of evaluation method

The content of each included resource was appraised based on five main criteria: resource characteristics, elements of graphical design and written communication, thoroughness and content, readability, and cultural appropriateness. Topics in each criterion were developed in discussion with a steering group where there was not a standard assessment tool available. Four experts from three different fields of expertise (alcohol and drugs, digital communication, and an Aboriginal and Torres Strait Islander person) were invited and agreed to join the steering group. All experts had no part in the original development of the educational brochures.

Resource characteristics

Characteristics of each included resource were documented, including the title, publisher, publisher type, target group and format. The format of AOD resources was recorded according to the type of resources (eg, booklets, brochures, fact sheets, postcards, posters and storybooks) and the number of pages (ie, in a PDF format).

Elements of graphical design and written communication

Elements of graphical design and written communication were appraised based on the use of headings and subheadings, typography, the percentage of bulleted text, the percentage of visual to written text, writing style, active voice, the use of professional jargons, type of illustrations, relevance of the illustrations, infographics, and use of colour supports.

For the evaluation of some elements of graphical design and written communication, the Suitability Assessment of Materials (SAM) evaluation criteria was used. Developed to systematically assess printed materials and illustrations in a timely manner, the SAM evaluation criteria allow healthcare providers to assess reading materials based on 22 factors and rate them as ‘2=superior,’ ‘1=adequate’ and ‘0=not suitable’ according to classifications established by Doak et al.²⁹ As many topics from the SAM evaluation criteria overlaps with the existing topics used in this study, only some elements were chosen or adapted from it. Three topics, including ‘typography,’ ‘writing style’ and ‘type of illustrations’, were analysed as per the SAM evaluation criteria and were documented as ‘superior,’ ‘adequate’ and ‘not suitable’.²⁹ For example, ‘writing style’ was rated as ‘superior’ if the resource utilised mostly conversational style and active voice, and simple sentences. Three topics of ‘use of headings and subheadings’, ‘active voice’ and ‘relevance of the illustrations’ were also adapted from the SAM evaluation criteria. These were marked as ‘yes’ if the resource used any headings and subheadings, used any active voice, or provided relevant illustrations to the topic. Kool’s macrocoherence and microcoherence model of communication also was used to evaluate the use of headings and subheadings and percentage of bulleted text in each resource.³⁰

Topics of ‘the percentage of visual to written text’, ‘infographics’ and ‘use of colour supports’ were used to evaluate the use of visual stimulation to achieve the principle of dual code theory. The dual code theory postulates that simultaneous verbal and visual cognitive systems stimuli will help readers to understand the presented information better.²⁵ Each included resource were marked as ‘yes’ for any use of medical or technical jargon as it could impede the reader’s understanding of the corresponding health information.^{31 32}

Thoroughness and content

To evaluate the thoroughness and content, the AOD resources were further categorised based on whether the primary focus was on alcohol or other drugs. Resources that covered both were classified into the group, to which the majority of its content belonged.

Each included resource was analysed based on the presence or absence of common elements of its group’s classification. For instance, the subtopic of impact of alcohol during pregnancy and breast feeding was analysed in the alcohol group, but not in other drugs group, as most resources in the alcohol group discussed the detrimental effect of alcohol among pregnant and breastfeeding women. Resources that had alcohol as the main content were evaluated based on the presence or absence of five key topics. These topics were background information, impact of alcohol, safe limit, contact information for getting help and supporting evidence. The topic of ‘impact of alcohol’ overarched into four subtopics: physical health, mental health, social impacts, and during pregnancy and breast feeding. The ‘other drugs’ as the main content were evaluated based on the presence or absence of eight key topics: background information, impact of drugs, overdose and withdrawal, treatment options, contact information for getting help, associated laws, information for family and carers, and supporting evidence. The topic of ‘impact of drugs’ was categorised into three subtopics of physical health, mental health and social impacts.

Readability

Four readability indices, the Flesch-Kincaid grade level (FKGL),³³ the Gunning Fog index (Fog),³⁴ the Simplified Measure of Gobbledygook (SMOG)³⁵ and the Flesch Reading Ease,³⁶ were used to evaluate the readability of the resource (ie, understandability of health information in each included resource). The formulas for the indices are outlined in table 1. These four indices are widely accepted to assess the readability of health education resources and are validated for calculating readability.^{29 37}

Table 1

Formulas of the four readability indices

Indices	Formula
Flesch-Kincaid grade level	(0.39×ASL)+(11.8×ASW)– 5.59
Gunning Fog Index	0.4 (ASL+percentage of PSW)
Simplified Measure of Gobbledygook	3+√PSW count
Flesch Reading Ease	206.835–(1.015×ASL)–(84.6×ASW)

ASL, average sentence length; ASW, average syllable per word; PSW, polysyllable word.

To obtain the readability score from the four readability formulas, the title and content of each included resource was converted to plain text (ie, without abbreviations, bullet points, illustrations and text boxes) and entered into an automated online programme.³⁸ In addition, the word and sentence counts produced by the online programme were manually matched to the plain text.

The FKGL, Fog and SMOG readability indices yield a numerical value that represents the reading grade level or the level of education required to understand the corresponding text.³³ On the contrary, the Flesch Reading Ease formula produces a score between 0 and 100. A lower score suggests a more difficult text to comprehend; whereas a higher score suggests that it is easier to comprehend the corresponding text.³⁶ For instance, Flesch Reading Ease score of 85 indicates an easy text, in which a year 6 student can understand. During the analysis, the Flesch Reading Ease score of included resources was converted into the average of reading grade levels based on table 2. Subsequently, reading grade levels obtained from four readability indices were compared with each other. The reading grade levels from these indices are based on the US grade levels and are equivalent to grade levels in Australia.³⁹

Table 2

Interpretation of the Flesch Reading Ease³⁶

Reading grade level	Verbal description	Reading ease
5	Very easy	90–100
6	Easy	80–89
7	Fairly easy	70–79
8–9	Standard	60–69
10–12	Fairly difficult	50–59
13–16	Difficult	30–49
Above 16	Very difficult	0–29

Cultural appropriateness

Culturally targeted messages have been shown to influence the receptivity of preventive programmes and health promotion among Indigenous Australians and Indigenous peoples worldwide.^{40 41} The cultural appropriateness aspect of each included resource was appraised based on the presence or absence of 10 criteria: avoids stereotypes, addresses underlying cause, uses local terms, includes an Aboriginal voice, targets local community, developed in collaboration with local community, directing people to culturally appropriate services, cultural match in logic, language and experience, diversity of imagery, and culturally relevant artwork or images. All criteria were modified from the adapted DISCERN instrument by the Cultural and Indigenous Research Centre Australia (CIRCA) research team, the SAM evaluation criteria and in discussion with an Aboriginal and Torres Strait Islander person from the steering group.⁴²

Patient and public involvement

Patients and the public were not involved in this research.

Results

Included in this study was a total of 30 AOD resources readily available in NSW produced from governmental and not-for-profit organisations that are specific to Aboriginal and Torres Islander Peoples.

Resource characteristics

Characteristics of included resources were documented in table 3. Most resources (n=14) were published by governmental organisations such as Australian Government Department of Health, NSW Health and NSW Police Force. Eleven resources were published by not-for-profit organisations such as Aboriginal Drug and Alcohol Network of NSW, Aboriginal Drug and Alcohol Residential Rehab Network, Aboriginal Health and Medical Research Council of NSW, Alcohol and Other Drugs Knowledge Centre, and The University of Sydney Matilda Centre. The other five resources were collaborative publication between a governmental organisation and a non-governmental organisation (Australian Government Department of Health and Continence Foundation of Australia; Australian Government Department of Health and The University of Sydney Matilda Centre and NSW Health and Australian Drug Foundation). The most common publishers noted in this study were NSW Health (n=11) and Alcohol and Other Drugs Knowledge Centre (n=8). Of the 30 included resources, 12 had a target group such as the community, families and friends, individuals who are using drugs, men, parents and carers, women, and young people. All included resources ranged from 1 to 16 pages in length, with an average of 4.4 pages. Twelve of the included resources were published in the format of fact sheets, followed by four brochures, four storybooks, three postcards and three posters.

Table 3

Characteristics of Aboriginal and Torres Strait Islander Peoples’ specific alcohol and other drugs resources

No	Resource title	Publisher	Publisher type	Target group (if any)	Format
1	Harm Minimisation and COVID-19	Aboriginal Health and Medical Research Council of NSW, Aboriginal Drug and Alcohol Network of NSW	Not-for-profit organisations	Individuals who are using drugs	1-page fact sheet
2	Fetal alcohol spectrum disorder (FASD) among Aboriginal and Torres Strait Islander people	Alcohol and Other Drugs Knowledge Centre	Not-for-profit organisations	–	1-page fact sheet
3	Kava use among Aboriginal and Torres Strait Islander people	Alcohol and Other Drugs Knowledge Centre	Not-for-profit organisations	–	1-page fact sheet
4	Methamphetamine use among Aboriginal and Torres Strait Islander people	Alcohol and Other Drugs Knowledge Centre	Not-for-profit organisations	–	1-page fact sheet
5	Key facts: Alcohol use among Aboriginal and Torres Strait Islander people	Alcohol and Other Drugs Knowledge Centre	Not-for-profit organisations	–	1-page fact sheet
6	Key facts: Illicit drug use among Aboriginal and Torres Strait Islander people	Alcohol and Other Drugs Knowledge Centre	Not-for-profit organisations	–	1-page fact sheet
7	Key facts: Volatile substance use (VSU) among Aboriginal and Torres Strait Islander people	Alcohol and Other Drugs Knowledge Centre	Not-for-profit organisations	–	1-page fact sheet
8	Grog is no good for our babies	NSW Health	Government	Men	1-page poster
9	Stay strong and healthy: It’s worth it	NSW Health	Government	Women	1-page poster
10	What is FASD?	NSW Health	Government	Young people	1-page poster
11	Recovery and isolation	Aboriginal Health and Medical Research Council of NSW, Aboriginal Drug and Alcohol Network of NSW	Not-for-profit organisations	Individuals who are using drugs	2-page fact sheet
12	Detox, rehab and COVID-19	Aboriginal Health and Medical Research Council of NSW, Aboriginal Drug and Alcohol Residential Rehab Network	Not-for-profit organisations	Individuals who are using drugs	2-page fact sheet
13	Facts about petrol, paint and other inhalants	Alcohol and Other Drugs Knowledge Centre	Not-for-profit organisations	–	2-page fact sheet
14	Facts about heroin	Alcohol and Other Drugs Knowledge Centre	Not-for-profit organisations	–	2-page fact sheet
15	Cracks in the ice: Information for families and friends	Australian Government Department of Health, The University of Sydney Matilda Centre	Government/ Not-for-profit organisations	Families and friends	2-page brochure
16	Cracks in the ice: Information for the community	Australian Government Department of Health, The University of Sydney Matilda Centre	Government/ Not-for-profit organisations	Community	2-page brochure
17	Cracks in the ice: Information for staying safe	Australian Government Department of Health, The University of Sydney Matilda Centre	Government/ Not-for-profit organisations	Individuals who are using drugs	2-page brochure
18	Stay strong and healthy it’s worth it	NSW Health	Government	Women	2-page postcard
19	What is FASD?	NSW Health	Government	Young people	2-page postcard
20	Not Our Way: Are you standing on thin ice?	NSW Police Force	Government	–	2-page postcard
21	Breaking the ice in our community	NSW Health, Australian Drug Foundation	Government/ Not-for-profit organisations	–	4-page fact sheet
22	Grog and bladder or bowel problems	Continence Foundation of Australia, Australian Government Department of Health	Not-for-profit organisations/ government	–	5-page brochure
23	Yarning about alcohol and pregnancy	NSW Health	Government	–	8-page booklet
24	Stay strong and healthy it’s worth it	NSW Health	Government	–	9-page storybook
25	Stay strong and healthy it’s worth it	NSW Health	Government	–	9-page storybook
26	Your Guide to Dealing With Teenagers and Grog	NSW Health	Government	Parents and carers	11-page booklet
27	Not Our Way: Are you standing on thin ice?	NSW Police Force	Government	–	12-page booklet
28	Not Our Way: Drugs don't have to be illegal to be lethal-The misuse of pharmaceutical drugs	NSW Police Force	Government	–	12-page booklet
29	Not Our Way: Are you standing on thin ice?	NSW Police Force	Government	–	16-page storybook
30	Not Our Way: Drugs don't have to be illegal to be lethal	NSW Police Force	Government	–	16-page storybook

NSW, New South Wales.

Elements of graphical design and written communication

The majority of the included resources (n=27) used headings and subheadings (see table 4). Although none of the resource’s typography was rated as superior, most resources (n=25) were rated adequate, whereas the typography of a booklet and four storybooks were rated not suitable. Of the 30 resources, 9 contained 50% or more bulleted text, whereas 13 used no bulleted text.

Table 4

Elements of graphical design and written communication

No	Use of headings and subheadings	Typography*	% bulleted text/% visual to written text	Writing style*	Active voice/ use of professional jargons	Type of illustrations*	Relevance of the illustrations	Infographics/ use of colour supports
1	Y	Adequate	70/0	Superior	Y/Y	–	–	-/Y
2	Y	Adequate	20/20	Adequate	-/Y	Superior	Y	-/Y
3	Y	Adequate	0/50	Not suitable	-/-	Superior	Y	Y/Y
4	Y	Adequate	0/80	Not suitable	-/-	Superior	Y	Y/Y
5	Y	Adequate	40/70	Adequate	-/-	Superior	Y	Y/Y
6	Y	Adequate	0/80	Not suitable	-/Y	Superior	Y	Y/Y
7	Y	Adequate	0/85	Not suitable	-/Y	Adequate	Y	Y/Y
8	–	Adequate	0/60	Superior	Y/-	–	–	-/Y
9	–	Adequate	0/50	Superior	Y/-	–	–	-/Y
10	–	Adequate	0/65	Superior	Y/-	–	–	-/Y
11	Y	Adequate	75/20	Superior	Y/Y	–	–	Y/Y
12	Y	Adequate	0/25	Adequate	Y/Y	–	–	Y/Y
13	Y	Adequate	85/0	Not suitable	Y/Y	–	–	-/Y
14	Y	Adequate	70/0	Adequate	Y/Y	–	–	-/Y
15	Y	Adequate	35/30	Superior	Y/Y	Superior	Y	-/Y
16	Y	Adequate	60/10	Adequate	Y/Y	Adequate	Y	-/Y
17	Y	Adequate	10/25	Superior	Y/Y	Superior	Y	Y/Y
18	Y	Adequate	40/50	Superior	Y/-	Superior	Y	-/Y
19	Y	Adequate	50/50	Superior	Y/-	–	–	-/Y
20	Y	Adequate	0/30	Not suitable	Y/-	Superior	Y	-/Y
21	Y	Adequate	35/0	Superior	Y/Y	–	–	-/Y
22	Y	Adequate	90/35	Superior	Y/-	Superior	Y	-/Y
23	Y	Adequate	35/40	Superior	Y/-	Superior	Y	Y/Y
24	Y	Not suitable	0/80	Superior	Y/-	Superior	Y	-/Y
25	Y	Not suitable	0/70	Superior	Y/-	Superior	Y	-/Y
26	Y	Not suitable	35/40	Superior	Y/-	–	–	-/Y
27	Y	Adequate	50/30	Superior	Y/-	Adequate	Y	-/Y
28	Y	Adequate	55/20	Superior	Y/-	Adequate	Y	-/Y
29	Y	Not suitable	0/75	Superior	Y/-	Superior	Y	-/Y
30	Y	Not suitable	0/60	Superior	Y/Y	Superior	Y	-/Y

‘Y’ refers to a ‘yes’.

*Categories are as per Suitability Assessment Material evaluation criteria.

Fourteen resources contained 50% or more visual to written text, while 4 resources contained no visuals. The writing style of most included resources (n=19) were appraised as superior. Conversely, five and six resources were appraised as adequate and not suitable, respectively. The majority of resources (n=24) used active voice in the written text. Thirteen resources used at least 1 term classed as professional jargon in the written text, and 16 identified jargon terms were noted in online supplemental material 1.

Of the 30 included resources, 19 resources contained some type of illustrations that were relevant to the topic (example resource: https://cracksintheice.org.au/pdf/cracks-in-the-ice-indigenous-what-is-cracks-in-the-ice.pdf).⁴³ Fifteen resources displayed superior illustrations, while four resources were considered to be adequate. Less than one-third of included resources (n=9) provided infographics, and all resources used some kind of colour support. Overall, a brochure titled ‘Cracks in the ice: Information for staying safe’ performed the best in the graphical design and written communication aspect. It demonstrated an adequate typography, a superior writing style, superior illustrations and covered all elements of interest.

Thoroughness and content

Thoroughness and content of included alcohol resources and other drugs resources are presented in tables 5 and 6, respectively. Twelve resources had alcohol as the main content, whereas 18 resources had other drugs as the main content.

Table 5

Thoroughness and content of alcohol resources (n=12)

No	Background information	Impact of alcohol				Safe limit	Contact information for getting help	Supporting evidence
		Physical health	Mental health	Social impacts	During pregnancy and breast feeding
1	Y	–	–	–	Y	Y	–	Y
2	Y	Y	Y	Y	–	–	–	Y
3	–	–	–	–	Y	Y	–	–
4	–	–	–	–	Y	Y	–	–
5	–	–	–	–	Y	–	–	–
6	–	–	–	–	Y	Y	–	–
7	–	–	–	–	Y	Y	–	–
8	Y	Y	–	–	–	–	Y	–
9	Y	Y	Y	Y	Y	Y	Y	–
10	–	Y	Y	Y	Y	Y	Y	–
11	–	Y	Y	Y	Y	Y	Y	–
12	Y	Y	–	Y	–	–	Y	–

‘Y’refers to a ‘yes’.

Table 6

Thoroughness and content of other drugs resources (n=18)

No	Background information	Impact of drugs	Overdose and withdrawal	Treatment options	Contact information for getting help	Associated laws	Information for family and carers	Supporting evidence
		Physical health/mental health/social impacts
1	Y	Y/-/-	Y	Y	Y	–	–	–
2	Y	Y/-/Y	–	–	–	Y	–	Y
3	Y	Y/Y/Y	–	–	–	–	–	Y
4	Y	Y/Y/-	–	Y	–	–	–	Y
5	Y	Y/-/-	–	–	–	–	–	Y
6	–	-/-/-	–	Y	Y	–	–	–
7	–	-/-/-	–	Y	Y	–	–	–
8	Y	Y/Y/-	–	–	Y	–	Y	Y
9	Y	Y/Y/Y	Y	Y	Y	Y	Y	Y
10	Y	Y/Y/Y	–	–	Y	–	Y	–
11	Y	-/-/Y	–	–	Y	Y	–	–
12	Y	Y/Y/Y	Y	–	Y	–	–	–
13	–	Y/Y/-	–	–	Y	–	Y	–
14	Y	Y/Y/Y	Y	Y	Y	Y	Y	–
15	Y	Y/Y/Y	Y	–	Y	Y	Y	–
16	Y	Y/Y/Y	Y	–	Y	Y	Y	–
17	–	Y/Y/Y	–	–	Y	Y	–	–
18	–	Y/-/Y	Y	–	Y	Y	Y	–

‘Y’ refers to a ‘yes’.

Of all alcohol resources, the 3 most covered elements were the impact of alcohol during pregnancy and breast feeding (75.0%), safe limit (66.7%) and physical health impact of alcohol (50.0%). On the other hand, only 16.7% of resources provided supporting evidence. A booklet titled ‘Yarning about alcohol and pregnancy’ provided the most comprehensive information. The eight-page booklet published by NSW Health covered almost all elements of interest but did not provide any supporting evidence. While a poster titled ‘What is FASD?’ only covered the element of impact of alcohol during pregnancy and breast feeding.

The most covered element among other drugs resources was the physical health impact of drugs (83.3%). Four other elements: background information, mental health impact of drugs, social impact of drugs and contact information for getting help were covered in more than half the resources. On the contrary, treatment options and supporting evidence were covered the least (33.3%). Of the 10 elements, 4 resources (22.2%) covered 7 or more elements of interest. A two-page fact sheet titled ‘Facts about heroin’ provided the most comprehensive information, covering all elements of interest. On the contrary, the two fact sheets titled ‘Detox, rehab and COVID-19’ only covered two elements: treatment options and contact information for getting help.

Readability

Figure 1 illustrates the reading grade level of each AOD resource based on four readability indices: FKGL, Fog, SMOG and Flesch Reading Ease. Generally, assessment of the same AOD resource using four readability indices resulted in four different grade levels. For instance, a one-page poster titled ‘Grog is no good for our babies’ attained a FKGL score of 2.6, an Fog score of 4.7, an SMOG score of 3.8 and a Flesch Reading Ease grade level of 6.

Figure 1. Readability indices scores of the included alcohol and other drugs resources.

Overall, FKGL scores were consistently at the same grade level or lower as the Fog and Flesch Reading Ease. A one-page fact sheet titled ‘fetal alcohol spectrum disorder (FASD) among Aboriginal and Torres Strait Islander people’ attained the highest reading grade levels across all four indices, with an FKGL score of 10.6, an Fog score of 11.2, an SMOG score of 9.7 and an average Flesch Reading Ease grade level of 14.5. Conversely, a two-page postcard titled ‘What is FASD?’ attained the lowest FKGL, SMOG and Flesch Reading grade level scores of 1.7, 3.5 and 5, respectively. This resource also achieved the second lowest Fog scores of 4.1. Both resources covered the topic of FASD and included 50% or fewer elements of interest, but the later resource had a higher percentage of bulleted text and visual to written text, achieved a superior writing style, used active voice and contained no professional jargons. A resource titled ‘Facts about heroin’ which contained the most comprehensive information received an FKGL score of 6.3, an Fog score of 8.7, an SMOG score of 6.5 and an average Flesch Reading Ease grade level of 8.5.

Cultural appropriateness

The cultural appropriateness aspect of all included resources is provided in table 7. All resources avoided stereotypes and were culturally match in logic, language and experience. Twenty-seven out of 30 resources included culturally relevant artwork or images. Similarly, majority of the included images (n=22) represented diversity in communities, in terms of age, appearance, gender and lifestyle. A brochure produced by the Australian Government Department of Health and The University of Sydney Matilda Centre was the only resource that acknowledged intergenerational trauma as one of the underlying causes of drugs misuse. On the other hand, another resource stated that alcohol use may contribute to separation from culture.

Table 7

Cultural appropriateness of Aboriginal and Torres Strait Islander Peoples’ specific alcohol and other drugs resources (n=30)

No	Avoid stereotypes	Address underlying cause	Uses local terms	Include an aboriginal voice	Target local community/ developed in collaboration with local community	Directing people to culturally appropriate services	Cultural match in Logic, language and experience	Diversity of imagery/ culturally relevant artwork or images
1	Y	–	–	–	-/-	–	Y	-/Y
2	Y	–	–	–	-/Y	Y	Y	Y/Y
3	Y	–	–	–	Y/Y	–	Y	Y/Y
4	Y	–	–	–	-/Y	–	Y	Y/Y
5	Y	–	–	–	-/-	–	Y	Y/-
6	Y	–	–	–	-/-	–	Y	-/-
7	Y	–	–	–	-/-	–	Y	Y/-
8	Y	–	–	–	-/-	Y	Y	Y/Y
9	Y	–	–	–	-/-	Y	Y	Y/Y
10	Y	–	–	–	-/-	Y	Y	Y/Y
11	Y	–	–	–	-/-	Y	Y	-/Y
12	Y	–	–	–	-/-	Y	Y	-/Y
13	Y	–	–	–	-/-	Y	Y	-/Y
14	Y	–	–	–	-/-	–	Y	-/Y
15	Y	–	–	–	-/-	Y	Y	Y/Y
16	Y	Y	–	–	-/-	Y	Y	-/Y
17	Y	–	–	–	-/-	Y	Y	Y/Y
18	Y	–	–	–	-/-	Y	Y	Y/Y
19	Y	–	–	–	-/-	–	Y	Y/Y
20	Y	–	Y	–	-/-	Y	Y	Y/Y
21	Y	–	–	–	-/Y	Y	Y	-/Y
22	Y	–	–	–	-/Y	Y	Y	Y/Y
23	Y	–	–	–	-/-	Y	Y	Y/Y
24	Y	–	Y	–	Y/-	Y	Y	Y/Y
25	Y	–	Y	–	-/-	Y	Y	Y/Y
26	Y	–	Y	–	Y/-	–	Y	Y/Y
27	Y	–	Y	Y	-/Y	Y	Y	Y/Y
28	Y	–	–	Y	-/Y	Y	Y	Y/Y
29	Y	–	–	–	-/Y	Y	Y	Y/Y
30	Y	–	–	–	-/Y	Y	Y	Y/Y

‘Y’ refers to a ‘yes’.

Three resources specified a local Indigenous community, namely Arnhem Land, Bandjalang and Koori. Moreover, five resources used local expressions, such as binjal, womba and yarndi in the written text. Of all included resources, 21 directed people to culturally appropriate services. Nine resources were developed in collaboration with local community, two of which included the voices of Aboriginal and Torres Strait Islander Peoples.

Discussion

This study appraised online AOD resources available in NSW for Aboriginal and Torres Strait Islander Peoples on the basis of resource characteristics, elements of graphical design and written communication, thoroughness and content, readability and cultural appropriateness. It was noted that resources written in the format of fact sheets have higher literacy demand as they are more likely to contain more information, more likely to use professional jargon, less likely to use active voice and contain fewer illustrations. However, some resources demonstrated that it is possible to have low literacy demand while containing comprehensive information and fulfilled most elements of graphical design and communication. Although most resources fulfilled half or more elements of interest in cultural appropriateness, they may not contain culturally targeted messages that meet the needs of all Aboriginal and Torres Strait Islander Peoples and communities.

While many included resources demonstrated good usability, there is a room for improvement in the areas of typography, using more illustrations, increasing the use of bulleted text, and reducing professional jargon. Monton et al reported that the typography subcategory assessed by the SAM score was rated as one of the highest, however, none of the resources in this study achieved a ‘superior’ typography.⁴⁴ Typography is an important element to ensure that the text is reader-friendly. For instance, a written resource is difficult to read when it is written in small print, hard-to-read fonts or all capital letters.²⁹ On the other hand, it is easier to read when the key points are emphasised (ie, bold, colour) and when the text is interspersed with blank space.^{29 45} Other studies evaluating health resources with SAM score emphasised the scarcity of ‘Superior’ resources, particularly regarding the graphics.^{44 46 47} Similarly, only half of the resources in this study demonstrated ‘superior’ illustrations as per SAM score, and one-third of the included resources did not contain any illustrations. This was concerning as including visual explanations have been shown to enhance the readers’ understanding and retention of information.⁴⁸ Readability is further enhanced when presented with bullet points and by avoiding the use of professional jargon. Thus, elements of graphical design and written communications are essential to aid readers’ readability and comprehension.

A way to predict the level of reading difficulties of written resources is by using readability tools, which assess the physical characteristics of the text such as word length, sentence length and syllable count.^{29 37} Previous research, on the readability of written health resources intended for the general population, discovered that most resources are too difficult to read for most patients, requiring grade level 9 and higher.^{26 28 44 49} On the contrary, written health resources for Aboriginal and Torres Strait Islander Peoples are generally written at a lower average readability of grade level 6.⁵⁰ However, these findings were not entirely consistent with this study. Although most included AOD resources for Aboriginal and Torres Strait Islander Peoples meet the NSW Health recommendation of grades 6–8 level, some resources were written above the recommended grade level.⁵¹ To ensure effective communication of written resources on AOD, providers should tailor the educational information to match the health literacy and reading skills of the target population and aim for the recommended reading levels.^{16 45}

Nevertheless, relying on readability tools alone to improve written resources may make the included information too simple and insufficient, as it does not consider the content of the text or difficult vocabularies.⁴⁵ This was observed in this audit, in which a resource with the lowest readability titled ‘What is FASD?’ contained only one key content. At the same time, a resource titled ‘yarning about alcohol and pregnancy’ demonstrated that it is possible to provide a comprehensive information and still achieve low readability scores. Pretesting materials for their cultural suitability and readability allows the publisher to make appropriate adjustments based on the feedback and discover possible cultural challenges that are otherwise overlooked. Therefore, pretesting prior to dissemination is an essential step to maximising the accessibility and suitability of information provided.^{16 45}

Culture is one of the contributing factors to the social and emotional well-being of Aboriginal and Torres Strait Islander Peoples.¹⁰ Previous studies signify the importance of involving Aboriginal and Torres Strait Islander organisations during the development process.^{40 42} Even so, only nine included resources clearly mentioned some form of collaboration with Aboriginal and Torres Strait Islander organisations. Similarly, a report by the CIRCA found that almost half of the recommended quality resources did not clearly disclose the community consultation and were only apparent after in-depth interviews.⁴² This is concerning as the community consultation is necessary to not only ensure culturally appropriate messages, but also ensure that the specific needs of the target community are addressed.^{40 42}

Cultural diversity among Aboriginal and Torres Strait Islander communities constructs a consequential challenge in developing culturally sensitive educational resources that caters to all communities.⁴⁰ This could be the foremost reason on the scarcity of health resources that target a specific local community, make use of local terms, address the underlying cause of AOD use or include an Aboriginal voice. Gould et al found that organisations often compromised by keeping the information general, avoiding stereotypes and using diverse images, even though some communities may end up being under-represented.⁴⁰ For example, including more real stories or Aboriginal voice may make the resource more engaging, but choosing the role models can be challenging as some communities do not allow showing names or pictures of the deceased.^{40 42 50} Hence, organisations should focus on developing resources that are specific to a local community and avoid generalising the messages.

Limitations of this study

To the best of our knowledge, this study is the first to appraise the usability, readability, content and cultural appropriateness of AOD resources for Aboriginal and Torres Strait Islander Peoples in NSW. However, five main limitations were identified. First, this study was limited to online AOD resources, which exclude hard copy resources, especially AOD resources that are developed by local Aboriginal Medical Services and only available in their local clinic, and resources that are disseminated through the social media. Second, the findings of this study do not represent resources that are produced in other states or territories in Australia, as the search was limited to resources available in NSW. Third, only resources written in English language were included in this study, which may exclude resources that are written in local language and developed specifically for a local community. Fourth, elements of the thoroughness and content was chosen based on the commonly discussed elements in their respective groups, which may exclude some elements that may potentially influence the intended readers’ opinion. Lastly, this study did not assess end-users view, such as through a qualitative investigation with Aboriginal and Torres Strait Islander Peoples or organisations.

Implications and recommendations

All assessed categories in this study (ie, usability, readability, content and cultural appropriateness) are essential and should be considered when developing AOD resources. It will likely be important to develop a wide range of resources on a variety of topics on AOD while providing comprehensive information to different audiences, including individuals with or at risk of AOD issues, individuals seeking AOD treatment, individuals undergoing AOD treatment, and families and carers. Furthermore, collaboration and partnerships with Aboriginal and Torres Strait Islander organisations and communities are vital to identify and address specific needs of their communities. The findings of this study also highlight the importance of a multifaceted evaluation process during the resource development phase. Despite the abundance of recommendations from many peer-reviewed journals, there is no specific national protocol for developing targeted resources that are evidence based, comprehensible, user-friendly and culturally acceptable across diverse communities.⁵⁰ Lastly, this study noted the need for health literacy data of Aboriginal and Torres Strait Islander Peoples, as it was not reported in the 2006 Australian Bureau of Statistics survey or the 2018 Australian National Health Survey.^{20 21}

Conclusion

Overall, resources developed by government organisations performed better across all categories. Even so, some AOD resources designed for Aboriginal and Torres Strait Islander Peoples in NSW are not optimal in terms of their usability, content thoroughness and readability, and cultural appropriateness. Involvement of Aboriginal and Torres Strait Islander Peoples during the resource development and evaluation process may improve their usability, the comprehensiveness of information, readability and culturally appropriateness. In addition, development of a national or standardised protocol for developing resources for Aboriginal and Torres Strait Islander Peoples may benefit government and not-for-profit organisations in developing future health resources that will contribute to reducing disparities in health outcomes.

Data availability statement

Data sharing not applicable as no datasets generated and/or analysed for this study.

Ethics statements

Patient consent for publication

Not applicable.

Footnote

Twitter @M_C_Tracy

Contributors AA, KR and PP conceived and designed the study. RA, KR and AA conducted the data collection. RA conducted the initial data analysis with help from KR, AA, PS, MT, NB and PP on the evaluation methods. RA drafted the initial manuscript. AA is the guatantor. All authors contributed to the revision of the manuscript and approve the final manuscript.

Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.

Competing interests None declared.

Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.

Provenance and peer review Not commissioned; externally peer reviewed.

Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.

References

¹ Australian Institute of Health and Welfare. Australian burden of disease study 2018: key findings. Canberra: Australian Institute of Health and Welfare, 2021. Available: https://www.aihw.gov.au/reports/burden-of-disease/burden-of-disease-study-2018-key-findings/contents/key-findings

² World Health Organisation. Alcohol (Fayetteville, NY). Geneva: World Health Organisation, 2022. Available: https://www.who.int/health-topics/alcohol#tab=tab_1

³ United Nations Office on Drugs and Crime. World drug report 2021 booklet 1: executive summary / policy implications. Vienna: United Nations Office on Drugs and Crime, 2021. Available: https://www.unodc.org/res/wdr2021/field/WDR21_Booklet_1.pdf

⁴ Australian Institute of Health and Welfare. Alcohol, tobacco & other drugs in Australia. Canberra: Australian Institute of Health and Welfare, 2021. Available: https://www.aihw.gov.au/reports/alcohol/alcohol-tobacco-other-drugs-australia/contents/about

⁵ Department of Health. National drug strategy 2017-2026. Canberra: Department of Health, 2017. Available: https://www.health.gov.au/sites/default/files/national-drug-strategy-2017-2026.pdf

⁶ Australian institute of Health and Welfare, Canberra. Australian burden of disease study 2018: key findings for Aboriginal and Torres Strait Islander people. Australian Institute of Health and Welfare, 2021. Available: https://www.aihw.gov.au/reports/burden-of-disease/australian-bod-study-2018-key-findings-indigenous/contents/key-findings

⁷ Australian Bureau of Statistics. Estimates of aboriginal and torres strait Islander Australians. Canberra: Australian Bureau of Statistics, 2018. Available: https://www.abs.gov.au/statistics/people/aboriginal-and-torres-strait-islander-peoples/estimates-aboriginal-and-torres-strait-islander-australians/latest-release

⁸ Australian Institute of Health and Welfare. Australia’s health 2022: data insights. Canberra: Australian Institute of Health and Welfare, 2022. Available: https://www.aihw.gov.au/getmedia/c91a05ef-307f-4c18-8ed3-dfe33d0c603d/aihw-aus-240.pdf.aspx?inline=true

⁹ Snijder M, Lees B, Stearne A, et al. An ecological model of drug and alcohol use and related harms among Aboriginal and Torres Strait Islander Australians: a systematic review of the literature. Prev Med Rep 2021; 21: 101277. doi:10.1016/j.pmedr.2020.101277

¹⁰ Murrup-Stewart C, Whyman T, Jobson L, et al. connection to culture is like a massive lifeline: yarning with Aboriginal young people about culture and social and emotional wellbeing. Qual Health Res 2021; 31: 1833–46. doi:10.1177/10497323211009475

¹¹ Greenfield SF, Sugarman DE, Nargiso J, et al. Readability of patient handout materials in a nationwide sample of alcohol and drug abuse treatment programs. Am J Addict 2005; 14: 339–45. doi:10.1080/10550490591003666

¹² Kessels RPC. Patients’ memory for medical information. J R Soc Med 2003; 96: 219–22. doi:10.1177/014107680309600504

¹³ Nutbeam D. Health promotion glossary. Health Promot 1986; 1: 113–27. doi:10.1093/heapro/1.1.113

¹⁴ Australian Institute of Health and Welfare. Health literacy. Canberra: Australian Institute of Health and Welfare, 2022. Available: https://www.aihw.gov.au/reports/australias-health/health-literacy

¹⁵ Degan TJ, Kelly PJ, Robinson LD, et al. Health literacy in substance use disorder treatment: a latent profile analysis. J Subst Abuse Treat 2019; 96: 46–52. doi:10.1016/j.jsat.2018.10.009

¹⁶ Badarudeen S, Sabharwal S. Assessing readability of patient education materials: current role in orthopaedics. Clin Orthop Relat Res 2010; 468: 2572–80. doi:10.1007/s11999-010-1380-y

¹⁷ Schaefer I, Heneka N, Luckett T, et al. Quality of online self-management resources for adults living with primary brain cancer, and their carers: a systematic environmental scan. BMC Palliat Care 2021; 20: 22. doi:10.1186/s12904-021-00715-4

¹⁸ Andrus MR, Roth MT. Health literacy: a review. Pharmacotherapy 2002; 22: 282–302. doi:10.1592/phco.22.5.282.33191

¹⁹ Beauchamp A, Buchbinder R, Dodson S, et al. Distribution of health literacy strengths and weaknesses across socio-demographic groups: a cross-sectional survey using the health literacy questionnaire (HLQ). BMC Public Health 2015; 15: 678. doi:10.1186/s12889-015-2056-z

²⁰ Australian Bureau of Statistics. Health literacy. Canberra, 2006. Available: https://www.abs.gov.au/ausstats/[email protected]/latestproducts/4233.0main%20features22006

²¹ Australian Bureau of Statistics. National health survey: health literacy. Canberra, AU: Australian Bureau of Statistics, 2018. Available: https://www.abs.gov.au/statistics/health/health-conditions-and-risks/national-health-survey-health-literacy/latest-release

²² Australian Commission on Safety and Quality in Health Care. Health literacy: taking action to improve safety and quality. Sydney: ACSQHC, 2014.

²³ Sørensen K, Van den Broucke S, Fullam J, et al. Health literacy and public health: a systematic review and integration of definitions and models. BMC Public Health 2012; 12: 80. doi:10.1186/1471-2458-12-80

²⁴ van Beusekom MM, Grootens-Wiegers P, Bos MJW, et al. Low literacy and written drug information: information-seeking, leaflet evaluation and preferences, and roles for images. Int J Clin Pharm 2016; 38: 1372–9. doi:10.1007/s11096-016-0376-4

²⁵ Whittingham JRD, Ruiter RAC, Castermans D, et al. Designing effective health education materials: experimental pre-testing of a theory-based brochure to increase knowledge. Health Educ Res 2008; 23: 414–26. doi:10.1093/her/cym018

²⁶ Lin S, Ayre J, Muscat DM. How readable are Australian multilingual diabetes patient education materials? an evaluation of national english-language source texts. Public Health Res Pract 2020; 30: 3012002. doi:10.17061/phrp3012002

²⁷ Haas K, Brillante C, Sharp L, et al. Lung cancer screening: assessment of health literacy and readability of online educational resources. BMC Public Health 2018; 18: 1356. doi:10.1186/s12889-018-6278-8

²⁸ Ferguson C, Merga M, Winn S. Communications in the time of a pandemic: the readability of documents for public consumption. Aust N Z J Public Health 2021; 45: 116–21. doi:10.1111/1753-6405.13066

²⁹ Doak CC, Doak LG, Root JH. Teaching patients with low literacy skills. Am J Nurs 1996; 96: 16M. doi:10.1097/00000446-199612000-00022

³⁰ Kools M, Ruiter RAC, van de Wiel MWJ, et al. Increasing readers’ comprehension of health education brochures: a qualitative study into how professional writers make texts coherent. Health Educ Behav 2004; 31: 720–40. doi:10.1177/1090198104263340

³¹ Zheng J, Yu H. Readability formulas and user perceptions of electronic health records difficulty: a corpus study. J Med Internet Res 2017; 19: e59. doi:10.2196/jmir.6962

³² National Centre for Education and Training on Addiction (NCETA) Consortium. Alcohol and other drugs: a handbook for health professional. Adelaide, AU: Australian Government Department of Health and Ageing, 2004.

³³ Kincaid JP, Fishburne RP, Rogers RL. Derivation of new readability formulas (automated readability index, FOG count and flesch reading ease formula) for navy enlisted personnel. 1975.

³⁴ Gunning R. The technique of clear writing. New York, US: McGraw-Hill, 1952.

³⁵ McLaughlin GH. Smog grading-a new readability formula. J Read 1969; 12: 639–46.

³⁶ FLESCH R. A new readability yardstick. J Appl Psychol 1948; 32: 221–33. doi:10.1037/h0057532

³⁷ Ley P, Florio T. The use of readability formulas in health care. Psychol Health Med 1996; 1: 7–28. doi:10.1080/13548509608400003

³⁸ Readability Formulas. Readability formulas. 2022. Available: https://readabilityformulas.com/free-readability-formula-tests.php

³⁹ El-Haddad N, Spooner C, Faruqi N, et al. Readability and content analysis of lifestyle education resources for weight management in Australian general practice. BMC Obes 2016; 3: 16. doi:10.1186/s40608-016-0097-1

⁴⁰ Gould GS, Stevenson LC, Cadet-James Y, et al. Cultural challenges when developing anti-tobacco messages for Indigenous Australians. Health Promot Int 2018; 33: 345–55. doi:10.1093/heapro/daw072

⁴¹ Gould GS, McEwen A, Watters T, et al. Should anti-tobacco media messages be culturally targeted for Indigenous populations? A systematic review and narrative synthesis. Tob Control 2013; 22: e7. doi:10.1136/tobaccocontrol-2012-050436

⁴² Cultural and Indigenous Research Centre Australia. Nationwide stocktake of Aboriginal and Torrest Strait Islander health resources. Sydney: ACSQHC, 2020.

⁴³ Department of Health and Aged Care. Cracks in the ice: Information for families and friends. Canberra, AU: Australian Government, 2022.

⁴⁴ Monton O, Lambert S, Belzile E, et al. An evaluation of the suitability, readability, quality, and usefulness of online resources for family caregivers of patients with cancer. Patient Educ Couns 2019; 102: 1892–7. doi:10.1016/j.pec.2019.05.010

⁴⁵ Weiss BD. Health literacy and patient safety: help patients understand, 2nd edn. Chicago, US: American Medical Association Foundation, 2007.

⁴⁶ Tian C, Champlin S, Mackert M, et al. Readability, suitability, and health content assessment of web-based patient education materials on colorectal cancer screening. Gastrointest Endosc 2014; 80: 284–90. doi:10.1016/j.gie.2014.01.034

⁴⁷ Rathod P, Ould Brahim L, Belzile E, et al. An evaluation of the suitability, readability, and quality of online resources for the self-management of depression. Patient Educ Couns 2019; 102: 952–60. doi:10.1016/j.pec.2018.12.021

⁴⁸ Park J, Zuniga J, Lee A. Effectiveness of using picture-based health education for people with low health literacy: an integrative review. Cogent Medicine 2016; 3: 1264679. doi:10.1080/2331205X.2016.1264679

⁴⁹ Oliffe M, Thompson E, Johnston J, et al. Assessing the readability and patient comprehension of rheumatology medicine information sheets: a cross-sectional health literacy study. BMJ Open 2019; 9: e024582. doi:10.1136/bmjopen-2018-024582

⁵⁰ Bar-Zeev Y, Bovill M, Bonevski B, et al. Assessing and validating an educational resource package for health professionals to improve smoking cessation care in Aboriginal and Torres Strait Islander pregnant women. Int J Environ Res Public Health 2017; 14: 1148. doi:10.3390/ijerph14101148

⁵¹ Clinical Excellence Commission. NSW health literacy framework: a guide to action. Sydney: Clinical Excellence Commission, 2019.