- Open Access
Oral/dental items in the resident assessment instrument – minimum Data Set 2.0 lack validity: results of a retrospective, longitudinal validation study
Population Health Metrics volume 14, Article number: 36 (2016)
Oral health in nursing home residents is poor. Robust, mandated assessment tools such as the Resident Assessment Instrument – Minimum Data Set (RAI-MDS) 2.0 are key to monitoring and improving quality of oral health care in nursing homes. However, psychometric properties of RAI-MDS 2.0 oral/dental items have been challenged and criterion validity of these items has never been assessed.
We used 73,829 RAI-MDS 2.0 records (13,118 residents), collected in a stratified random sample of 30 urban nursing homes in Western Canada (2007–2012). We derived a subsample of all residents (n = 2,711) with an admission and two or more subsequent annual assessments. Using Generalized Estimating Equations, adjusted for known covariates of nursing home residents’ oral health, we assessed the association of oral/dental problems with time, dentate status, dementia, debris, and daily cleaning.
Prevalence of oral/dental problems fluctuated (4.8 %–5.6 %) with no significant differences across time. This range of prevalence is substantially smaller than the ones reported by studies using clinical assessments by dental professionals. Denture wearers were less likely than dentate residents to have oral/dental problems (adjusted odds ratio [OR] = 0.458, 95 % confidence interval [CI]: 0.308, 0.680). Residents lacking teeth and not wearing dentures had higher odds than dentate residents of oral/dental problems (adjusted OR = 2.718, 95 % CI: 1.845, 4.003). Oral/dental problems were more prevalent in persons with debris (OR = 2.187, 95 % CI: 1.565, 3.057). Of the other variables assessed, only age at assessment was significantly associated with oral/dental problems.
Robust, reliable RAI-MDS 2.0 oral health indicators are vital to monitoring and improving oral health related quality and safety in nursing homes. However, severe underdetection of oral/dental problems and lack of association of well-known oral health predictors with oral/dental problems suggest validity problems. Lacking teeth and not wearing dentures should be considered an indicator for urgent oral/dental treatment needs.
Oral health issues are among the 50 most prevalent pathologic conditions worldwide: dental caries of permanent teeth (rank 1, 35 %, n > 2.4 billion people), chronic periodontitis (rank 6, 11 %, > 740 million people), and edentulism (rank 36, 2 %, > 150 million people) . In many instances oral health conditions are chronic conditions, sharing common, modifiable risk factors (e.g., tobacco use, excessive consumption of alcohol, unhealthy diet, stress, less than optimal personal hygiene) with other chronic diseases . Periodontitis increases the risk for cardiovascular diseases, diabetes, or respiratory diseases , and periodontal diseases can be clinical manifestations of systemic diseases such as HIV, diabetes, respiratory diseases, and cancer . Oral conditions can cause disability, and their global burden is comparable to conditions such as hypertensive heart disease, schizophrenia, anemias, and various forms of cancer .
Older people are at particular risk for poor oral health. As people age, their oral health deteriorates, partly through physical changes but primarily through reduced/limited access to dental services  and through chronic diseases that increase frailty and limit ability for self-care . Therefore international organizations, such as the World Health Organization , the FDI World Dental Federation [8, 9], the English National Health System , the US Institutes of Medicine [11, 12], and the Canadian Academy of Health Sciences , have made strong policy statements calling for action to improve oral health care for frail older adults by applying an “oral-health-in-all-policies approach” , and proclaiming life-long oral health as a “fundamental human right” .
Oral health care is poor in nursing homes, internationally [13, 14] and in Canada [15, 16]. Nursing home residents are a highly vulnerable population at particularly high risk for poor oral health . In agreement with international studies [17–19], one of the few Canadian studies on oral health of nursing home residents  found that 41 % of residents had no natural teeth, 41 % had mucosal abnormities, 5 % reported toothache, and 1 % reported severe tooth/mouth pain at night. Of residents with natural teeth, 51 % had untreated coronal caries and 44 % had untreated root caries.
Poor oral health raises health care costs and affects residents’ quality of life and safety through unnecessary pain, suffering, and elevated risk of malnutrition, aspiration pneumonia, respiratory diseases, diabetes, cardiovascular diseases, and premature death [3, 20–25]. Bad breath, changed dental aesthetics, and altered speech can affect self-image and self-esteem, with serious psychological and social consequences [26, 27].
By 2021, baby boomers will enter nursing homes in greater numbers with more of their natural teeth, more complex prostheses and bridges than previous generations, and significantly increased and different care needs . Challenges in oral health care are further elevated by the rapidly growing number of residents with dementia, who need extra assistance and who may exhibit responsive behaviors that complicate care . However, 80 % or more of direct care in Canadian nursing homes is provided by unregulated care aides with limited formal training who face challenging workloads [30, 31].
Improving oral health assessments in nursing homes is a priority to promote health and quality of life [32, 33] but the lack of mandated, robust tools is a major problem [5, 34, 35]. Most Canadian provinces mandate use of the Resident Assessment Instrument – Minimum Data Set (RAI-MDS) 2.0  for nursing home residents on admission and subsequent quarterly intervals . It is a valid, reliable standardized tool to assess residents’ clinical and functional characteristics [38, 39], which can effectively monitor and improve quality of care. The RAI-MDS 2.0 or the related interRAI Long-Term Care Facilities (LTCF) version is currently used in Europe (Belgium, England, Finland, France, Germany, Iceland, Italy, Netherlands, Norway, Spain, Sweden, Switzerland), Asia (Hong Kong, Korea, Japan), and the Pacific Rim (Australia, New Zealand) . The United States is the only country using another related version, the RAI-MDS 3.0 . The oral/dental items of the RAI-MDS 2.0 are completed only on the full assessment version, done on admission, and then annually or when significant change occurs. The quarterly version of the RAI-MDS 2.0 done at other three-month intervals omits the oral/dental items.
Reliability and validity literature for the RAI-MDS in general, and its oral health items in particular, go back to the instrument’s development more than 25 years ago [41, 42]. Reliability studies using interrater methods have generally reported strong findings for RAI-MDS 2.0 items [43–45] and quality indicators (QIs), defined as rates of clinically relevant outcomes, such as falls, pressure ulcers or pain, aggregated on unit or facility level  (kappa values > 0.7 for almost all items and QIs). Internal consistency reliability was found to be high for the Cognitive Performance Scale (CPS), depression and pain items . Mor et al. , Poss et al. , and Shin and Scherer  provide summaries of the RAI-MDS 2.0 reliability and validity literature. The CPS and the Activities of Daily Living (ADL) scales especially were consistently found to have strong criterion and construct validity, while depression, behavior, pain and nutrition/weight loss items are less valid. Hirdes et al.  report internal consistency reliability and criterion validity results for the Canadian RAI-MDS 2.0 version. Cronbach’s alpha values for the ADL long form scale, the Depression Rating Scale (DRS) and the Aggressive Behaviour Scale (ABS) consistently exceeded 0.7, and criterion validity assessments demonstrated that cognition, ADL, continence, and behavior were related in the expected directions, with stable associations over time.
While the RAI-MDS 2.0 has been effectively used to monitor and improve safety and quality of care of nursing home residents [41, 47–49], early US studies indicate low reliability  and validity [20, 50–53] of the oral health components, partly due to items themselves and partly due to poorly trained assessors (care home staff) who underestimate mouth problems. Hawes et al.  report an average inter-rater reliability of 0.46 for the oral/dental items, which is at the low range of acceptable , and notably lower compared to other MDS items. Psychometric properties of the RAI oral health items are rarely reported, possibly because of their inclusion only on the full assessment version and their absence from important applications such as case mix or quality reporting. The American Dental Association (ADA) and Special Care Dentistry (SCD) challenged the content validity (i.e., completeness and appropriateness of wording) of the RAI-MDS 2.0 oral/dental items [20, 52]. Only 9 % of 236 surveyed nursing directors in US nursing homes thought the RAI oral/dental items were often useful to identify dental needs of residents . Arvidson-Bufano , Folse , and Cohen-Mansfield  report severe underdetection of oral/dental problems by nursing home care staff using the RAI oral/dental items, compared to clinical assessments by dental professionals. However, Arvidson-Bufano  demonstrated that the assessment quality could be improved by a 30-min training of nursing home care staff.
Mandatory use of the RAI-MDS 2.0 in most Canadian nursing homes supports monitoring of multiple facilities and a large population of residents. However, the oral/dental items are rarely used for this purpose, in part because they are not reported on the quarterly assessment version, and to our knowledge publications assessing their criterion validity (relationship with other outcomes) are not available. Criterion validity is defined as the association of a measure with outcomes of interest, either concurrently or predictively, as expected based on theory, evidence, and common reasoning [56, 57]. The Standards for Educational and Psychological Testing  – recognized as best practice in developing and validating scientific assessment tools  – consider these analyses to be an important source of validity evidence. Our Translating Research in Elder Care (TREC) study  offers a large longitudinal data set that we exploited to assess the criterion validity of the RAI-MDS 2.0 oral/dental items. While comparisons of RAI-MDS 2.0 regular staff assessments with assessments done by dental professionals have been conducted, to the best of our knowledge our study is the first to systematically assess criterion validity of the RAI-MDS 2.0 oral health items, using a modeling-based approach to assess if oral/dental problems as measured by the RAI-MDS 2.0 are associated with predictors of oral health as expected.
Setting and sample
Our data were collected within the Translating Research in Elder Care (TREC) study (see  for details). The database included 73,829 RAI-MDS 2.0 records collected from 10,975 residents (2007–2012) from a stratified random sample of 30 urban nursing homes in Alberta, Saskatchewan, and Manitoba. We listed all nursing homes in the three provinces and stratified them by province, owner-operator model (private for-profit, public not-for-profit, voluntary not-for-profit), and size (small: <80 beds, medium: 80–120 beds, large: >120 beds). The 30 nursing homes randomly drawn from this population reflect the population proportionality of the three stratification criteria (for details, see ).
From our RAI-MDS 2.0 database, we selected all residents with an admission assessment plus one or more consecutive annual full assessments. This resulted in a study sample of 7,368 RAI-MDS 2.0 records from 2,711 residents (n = 1,471 residents with two full assessments, n = 703 residents with three full assessments, n = 376 residents with four full assessments, n = 153 residents with five full assessments, and n = 8 residents with six full assessments). Figure 1 details the number of residents and corresponding assessments excluded from our analyses, and the reasons for exclusion.
This retrospective, longitudinal, and secondary data analysis assessed a) change in RAI-MDS 2.0 oral health variables over time and b) association of oral health variables with residents’ dentate status and with other variables known to influence residents’ oral health.
Oral health variables
The RAI-MDS 2.0 full assessment  covers the following dichotomous oral/dental items:
Chewing problems (K1a)
Swallowing problems (K1b)
Mouth pain (K1c)
Debris (easily removable substances) in mouth at bedtime (L1a)
Dentures/removable bridge (L1b)
Some/all natural teeth lost, no dentures/partial plates available/used (L1c)
Broken, loose, or carious teeth (L1d)
Inflamed, swollen or bleeding gums; oral abscesses, ulcers, or rashes (L1e)
Daily oral health care by resident or staff (L1f)
We combined the variables mouth pain (K1c), dental problems (L1d), and periodontal problems (L1e) into a dichotomous variable reflecting oral/dental problems overall (coded as 1 = one or more problems present or 0 = no problems present).
Our primary independent variable was dentate status, which is closely associated with oral health conditions (see Additional file 1 for a detailed description of the supporting evidence). We used RAI-MDS 2.0 variables L1b (dentures) and L1c (some/all teeth lost, no dentures used) to generate a categorical variable reflecting dentate status: dentate (“no” to L1b and L1c), dentures (“yes” to L1b, “no” to L1c), or no dentures (“yes” to L1c, regardless of the coding of L1b).
Our model included RAI-MDS 2.0 variables (known association with oral/dental problems) of dementia diagnosis and oral hygiene (debris, daily cleaning). We adjusted our final model using covariates known to be associated with nursing home residents’ oral health (see Additional file 1 for details on the supporting evidence): age at RAI-MDS 2.0 assessment (continuous); sex; physical functioning (Activities of Daily Living – Hierarchical score  >3); cognition (Cognitive Performance Scale score  >3); resists care at least weekly (yes/no); depressive symptoms (Depression Rating Scale score  >2); and dementia diagnosis. We included quarter of assessment (e.g., Q4/2007) as a continuous covariate to adjust for potential systematic differences in assessment practices and awareness of oral/dental problems at different time points. All independent variables and covariates were used from the same assessment as the dependent variable.
We used IBM® SPSS® version 23.0 for all analyses. With descriptive analyses, we compared prevalence of oral/dental problems and other resident characteristics over time. We ran General Estimating Equation (GEE) models to account for assessments nested within residents and to simultaneously assess effects of time, dentate status, and covariates on residents’ oral/dental problems. Model 1 included only dentate status as independent variable. Models 2–4 included dentate status plus dementia diagnosis, debris, or daily cleaning. Our final GEE model (model 5) included all those variables plus all additional covariates. We used a binomial logit link function with the dichotomous dependent variable and an autoregressive working correlation matrix (AR1) in all models. Due to the way RAI data are collected and cleaned in Canada, our data set did not include any missing variables. The completeness and integrity of RAI-MDS data items are extremely high in Canada due to universal use of electronic entry which constrains responses to available options, and only allows an assessment submission to occur when all items are completed. In addition, the Canadian Institute for Health Information, the national agency to which all facilities in TREC submit to, provides additional data checks on submitted records . Therefore, the assumption underlying GEEs that data are missing completely at random (MCAR) was met on an individual item level. In addition, we checked if entire assessments (i.e., residents only having two as opposed to three to six assessments) were missing completely at random. We generated variables, indicating if the third, fourth, fifth, and sixth assessment were either missing = 0 or present = 1 at a specific time point. Using these variables, we performed Little’s MCAR test , which failed to reject the null hypothesis that data are MCAR (χ2 = 9.327, DF = 6, P = 0.156). In addition, using non-parametric exact tests for categorical outcomes, and t-tests for continuous outcomes, we compared study outcomes at baseline between residents with exactly two assessments and residents with three or more assessments (Additional file 2). Compared to residents with exactly two assessments, residents with three or more assessments were significantly less likely to resist care, to be highly physically impaired and depressive, more likely to be female, and their baseline assessments were conducted in later quarters. All other variables did not differ significantly.
We performed multicollinearity assessment of all covariates included in the final model. The results (Additional file 3) indicate no substantial multicollinearity with tolerance values clearly above the recognized threshold value of 0.1 . We added variables in a stepwise approach. None of the regression coefficients or their related 95 % confidence interval (CI) changed substantially when adding or removing variables.
To check for possible model overfitting as well as for differences between residents included in the analyses and excluded residents, we conducted two types of sensitivity analysis. First, we ran four separate binary logistic regression models (corresponding to the final GEE model 5) with all first, second, third, and fourth assessments, respectively. Second, we compared admission information of our 2,711 included residents to admission information of all excluded residents with an admission assessment available (n = 5,132, Fig. 1), and we ran another binary logistic regression model (corresponding to the final GEE model 5), using the admission assessments of the excluded residents (Additional file 4).
As we used independent variables and covariates from the same assessment as the dependent variable, no conclusions are possible whether the independent variables/covariates (e.g., debris) predict oral/dental issues or if oral/dental issues predict the independent outcomes (reverse causality). Therefore, we ran the final model again, using independent outcomes/covariates from a) the assessment conducted previously to the one including the dependent variable, and b) the admission assessment of each resident (Additional file 5).
Resident characteristics and prevalence of model variables changed over time of assessment (Table 1). Overall, residents became more physically dependent, cognitively impaired, and depressed, and resisted care more frequently. Rates of diabetes, atherosclerosis, and pneumonia remained roughly constant.
Numbers of fully dentate residents decreased substantially over time (Table 1). Residents lacking teeth (edentulous) but not wearing dentures, the smallest group at first assessment, outnumbered dentate residents by the second assessment.
Tooth problems, periodontal problems, and mouth pain fluctuated across assessments (Table 1). Almost all residents (87.5 %–95.2 %) were assessed with no oral health problems. In all GEE models, dentate status was strongly associated with oral/dental problems (Table 2, Fig. 2). The final adjusted model, including all covariates, indicated that the odds to have oral/dental problems were less than half as high for denture wearers than for dentate residents. Edentulous residents not wearing dentures had almost two times higher odds of oral/dental problems than dentate residents. Oral debris and age at assessment were associated at statistically significant levels with oral/dental problems. Dementia and cognition care resistant behavior, physical functioning, depression, daily cleaning, and female sex were not significant predictors of oral health overall. Oral/dental problems of individual residents did not change significantly over time.
The cohort of excluded residents with an admission assessment did not differ from our included residents with respect to age, dementia diagnosis, atherosclerotic heart disease, and oral/dental problems (Table 3). However, excluded residents were more likely to be male, more physically dependent, cognitively impaired, depressed, resisted care more frequently, and more often had diabetes or pneumonia than our study cohort. Also, excluded residents wore dentures less often, had more chewing and swallowing problems, debris, and received daily cleaning more frequently than our study cohort. Despite these substantial differences, the binary logistic regression model conducted with the excluded residents, as well as the other regression models support the findings of our GEE models (Additional file 4). The additional GEE models, using independent outcomes/covariates from a) the assessment conducted previously to the one including the dependent variable, and b) the admission assessment of each resident, also confirm our conclusions (Additional file 5). In addition, they support the assumption that the independent outcomes indeed precede the dependent variable (oral/dental issues).
We measured variation of nursing home residents’ oral/dental problems as assessed by the RAI-MDS 2.0 oral/dental items over time, and association of well-known covariates of nursing home residents’ oral health with those oral health problems. Our findings indicate validity problems for the Canadian RAI-MDS 2.0 oral/dental items.
Prevalence of oral/dental problems
Our prevalence findings are similar to other studies using RAI-MDS 2.0 oral/dental items [53, 64–66]: tooth problems (our study, 3.5 %–4.5 %; other studies, 3.0 %–7.7 %); periodontal problems (our study, 1.0 %–1.9 %; other studies, 0 %–5.4 %); mouth pain (our study, 0.7 %–1.2 %; other studies, 0 %–9.5 %); and overall oral/dental problems (our study, 4.8 %–5.6 %; one other study, 7.6 % ).
However, comparing our findings with studies using clinical assessments by dental professionals indicates that – like the US version of the RAI-MDS 2.0 [53, 55, 68] – the RAI-MDS 2.0 in our study significantly underdetects oral/dental problems. Clinical assessments find caries rates of 44.4 %–76.2 % among dentate residents [16–19, 69–71]. Among all residents, 31.7 %–48.7 % need periodontal treatment [16, 70, 72], 65.6 %–74 % have gingivitis [16, 19], 33.3 %–41.2 % have mucosal abnormalities [16, 71], 5.2 %–8.2 % have dental pain [16, 72], 1.2 % have severe tooth/mouth pain at night , and 3.4 % report gum pain or discomfort .
Association of oral/dental problems with other variables
According to the literature, each variable in our model is robustly associated with nursing home residents’ oral health (Additional file 1). However, we saw significant association of only a few included variables with oral/dental problems, further underscoring potential validity problems with the Canadian RAI-MDS 2.0 oral/dental items. We did find three significant predictors of oral/dental problems in our models: dentate status, debris, and age at assessment.
Our measured dentate status rates are similar to other studies. In this study, 54.7 %–63.9 % of residents wore dentures, comparable to 45.9 %–57.5 % in other studies based on RAI-MDS 2.0 data [53, 64, 65]. Studies using different tools or methods [73, 74] found that 24.4 %–43.6 % of residents wore complete dentures and 25 %–26.9 % wore removable partial dentures. Matthews et al.  reported 68.1 % of residents with maxillary (upper) dentures and 35 % with mandibular (lower) dentures. In our study, 18.5 %–23.3 % of residents lacking some or all natural teeth did not wear dentures, compared to 14.8 %–35 % in studies using RAI-MDS 2.0 data [64–66] and 36 % in Matthews et al. .
A third to a half of residents in North American nursing homes lack teeth [16, 19, 71, 75, 76], the source of potential dental problems and most oral/dental problems. This fits our finding that denture wearers have fewer oral/dental problems. In contrast, denture wearers show higher risk of periodontal problems than dentate older adults [22, 77], a finding not confirmed by our study. Nursing home staff, who complete the RAI-MDS 2.0, may not detect periodontal problems; staff are not well-trained in oral health assessments and oral health care is a neglected topic in nursing homes [35, 78–82].
Our study confirms that tooth loss is associated with oral health problems in older adults who do not wear dentures [22, 77]. Residents may be discouraged from wearing dentures by painful remaining teeth, gum diseases such as gingivitis or periodontitis, or irritated gum tissues from ill-fitting dentures. Matthews et al.  found that almost 60 % of residents’ dentures were ill-fitting.
Poor oral hygiene is a major risk factor for tooth loss as well as periodontal diseases, and maintaining good oral hygiene helps retaining teeth and preventing oral inflammatory processes [83, 84]. Matthews et al.  found that a high debris score (≥2) increased the risk for coronal decay (adjusted OR = 2.12, 95 % CI: 1.02, 4.34). This matches the results of our final model in which oral debris was significantly associated with oral/dental problems (OR = 2.187, 95 % CI: 1.565, 3.057).
Age at assessment
One of the strongest risk factors for poor oral health is old age – partly due to physical changes, but primarily due to frailty, multiple chronic diseases increasing self-care deficiencies, and higher barriers to accessing professional dental services [85, 86]. While we found that age at assessment was significantly associated with oral/dental problems in our final model, our findings suggest that younger residents are more likely to have oral/dental problems than older residents. Specifically, the odds of oral/dental problems decrease by 1.7 % with each additional year of age (OR = 0.983 in our model) or by 8.2 % with each additional 5 years of age (OR = 0.9835 = 0.918). This is in disagreement with the available evidence, and it may be another indicator of validity problems related to the RAI-MDS 2.0 oral/dental items. However, the available evidence mostly focuses on older adults in general. Specific evidence on the effect of age on nursing home residents’ oral health is limited (see  as one of the few examples suggesting that older nursing home residents have poorer oral health than younger nursing home residents). Therefore, another possible reason for our finding may be that people whose health and personal situations require a move into a nursing home at a younger age may have generally poorer oral health than people who are older when moving into the facility. There is some evidence (although weak and inconsistent) that availability of strong social networks, especially of a spouse, partner, or friend who is able to assist with the necessary care can prevent a nursing home admission . Although the supporting evidence is inconclusive , older people are thought to be more vulnerable to loneliness and social isolation than younger people, due to the loss of spouses, partners, and other confiding relationships . Therefore, strong informal supports that protect from nursing home admission may be absent at older age resulting in relatively better health as well as functional and oral health status at nursing home admission, compared to admission at younger age. Furthermore, younger people who move into a nursing home may have different disease processes which might be associated with different risks for oral/dental problems than the typical frail senior, but further studies would be needed to establish the validity of these assumptions.
Change over time
Although oral/dental problems did not significantly change for individual residents across the six assessments, they increased overall in our sample. In quarters 3/2007, 4/2007, and 2/2008, 0 % of records indicated oral/dental problems; in quarter 2/2012, 12.7 % of records indicated problems. With our data, we cannot determine whether this increase reflects true trends for Canadian nursing home residents over time or increased detection of problems by care providers completing assessments. The age-standardized prevalence of untreated caries  and severe periodontitis  remained constant in North America between 1990 and 2010; prevalence of severe tooth loss decreased . However, evidence is scarce, and often of poor quality, on trends over time for older adults’ oral health [93, 94]. Even less is known about nursing home residents’ oral health over time. In 2007 the RAI-MDS 2.0 had been implemented only recently in Alberta, Saskatchewan, and Manitoba, with substantial later efforts to improve data quality [37, 95–97]. Most likely, care providers completing assessments became better at detecting oral/dental problems.
The interRAI LTCF is poised to be the successor of the RAI-MDS 2.0 in Canada. It drops or modifies some of the oral/dental items and does not use an abbreviated quarterly version omitting these items, and thus may be better suited for monitoring quality in this area, compared to the RAI-MDS 2.0.
Strengths and limitations
Our study is the first to assess criterion validity of oral/dental items in the RAI-MDS 2.0 and to assess them over time (five consecutive years). It is unique in assessing differences in rates of oral/dental problems by resident dentate status. We used GEE models that are robust against non-normality, do not require continuous data, account for dependency of multiple assessments per resident, and simultaneously assess effects of various factors. From a representative sample of residents in Western Canadian nursing homes, we included all residents with an admission assessment plus two or more consecutive annual full assessments.
We excluded all residents with no admission assessment available and all residents with an admission assessment but no consecutive annual assessment available. Our cohort may therefore not represent the population of short-stay nursing home residents with a length of stay of less than 12 months . They might be healthier and have lower prevalence of oral/dental problems. When comparing our sample to excluded residents with admission assessments available, we noticed that the excluded residents are sicker and frailer than our study sample, but do not differ with respect to oral/dental problems, and a binary logistic regression using admission assessments of excluded residents confirmed results of our GEE model. As we were interested in general associations between variables rather than in causal relationships and generalizable effect sizes, we decided not to include sampling weights (i.e., proportional representation of residents according to the strata of the stratified random sample) into our models. Including these sampling weights may have altered our findings . Also, the assumption relevant to GEEs that data are missing completely at random may not have been met entirely in our data set. We did not have any missing items, and Little’s MCAR test was non-significant for missed follow-up assessments. But residents with exactly two assessments differed in some baseline outcomes from residents with three or more assessments. However, prevalence of oral/dental problems in our study was similar to other studies using RAI-MDS 2.0 data, indicating that such data generally underdetect oral/dental problems. RAI-MDS variables do not allow detailed assessment of kinds of tooth or periodontal problems or of dentate status. Information was missing on how many residents are completely edentulous or have some teeth left, type of dentures used (complete, partial, upper, lower), and why residents do not wear dentures. Further studies based on detailed clinical assessments could determine if specific oral health problems are associated with specific dental and prostheses status.
Oral health in nursing home residents is poor. Compared to non-institutionalized older adults, nursing home residents are particularly frail and barriers to accessing dental services are even higher. Care providers should pay special attention to oral health of edentulous residents not wearing dentures.
Oral health of nursing home residents must be assessed regularly using a valid, reliable, and practical tool. RAI-MDS 2.0 oral/dental items likely significantly underdetect oral/dental problems and are not associated with well-proven predictors for oral health, indicating poor validity . Research is needed on whether these validity problems are due to inappropriate application of the tool or if the tool is flawed more fundamentally (i.e., vague and poorly defined constructs). In the former case, major investments into additional training of assessors and modified prioritizing are required, in the latter case, RAI-MDS 2.0 oral/dental items need to be modified or supplemented by more robust tools. The potential effect of the interRAI LTCF with its modified oral/dental items and more frequent collection is unknown. Residents’ dentate status is a promising indicator of poor oral health and as such, a promising avenue for improving quality of care and quality of life.
Widespread use and the longitudinal nature of the RAI-MDS 2.0 offer great potential to close a severe knowledge gap in nursing home residents’ oral health: trajectories of oral health problems for residents, health regions, and jurisdictions. By incorporating robust items with acceptable reliability and validity, the RAI-MDS 2.0 can also be used to monitor and improve quality and safety of oral health care in nursing homes.
Vos T, Flaxman AD, Naghavi M, Lozano R, Michaud C, Ezzati M, Shibuya K, Salomon JA, Abdalla S, Aboyans V, et al. Years lived with disability (YLDs) for 1160 sequelae of 289 diseases and injuries 1990–2010: a systematic analysis for the Global Burden of Disease Study 2010. Lancet. 2012;380:2163–96.
Petersen PE, Ogawa H. The global burden of periodontal disease: towards integration with chronic disease prevention and control. Periodontol 2000. 2012;60:15–39.
Azarpazhooh A, Tenenbaum HC. Separating fact from fiction: Use of high-level evidence from research syntheses to identify diseases and disorders associated with periodontal disease. J Can Dent Assoc. 2012;78:c25.
Marcenes W, Kassebaum NJ, Bernabe E, Flaxman A, Naghavi M, Lopez A, Murray CJ. Global burden of oral conditions in 1990-2010: a systematic analysis. J Dent Res. 2013;92:592–7.
Canadian Academy of Health Sciences. Improving access to oral health care for vulnerable people living in Canada. Ottawa: CAHS; 2014.
MacEntee MI. Muted dental voices on interprofessional healthcare teams. J Dent. 2011;39 Suppl 2:S34–40.
World Health Organization: Tokyo declaration on dental care and oral health for healthy longevity – World Congress 2015: Dental care and oral health for healthy longevity in an aging society. http://www.who.int/oral_health/events/worldcongress2015/en/ (2015). Accessed 07 Feb 2016.
FDI World Dental Federation. The challenge of oral disease: A call for global action – The Oral Health Atlas. Brighton: Myriad Editions; 2015.
FDI World Dental Federation. FDI policy statement on oral health for healthy ageing. Int Dent J. 2016;66:7–8.
National Health System England: Improving dental care and oral health – a call to action. https://www.england.nhs.uk/ourwork/qual-clin-lead/calltoaction/dental-call-to-action/ (2014). Accessed 07 Feb 2016.
Institute of Medicine and National Research Council of the National Academies. Improving access to oral health care for vulnerable and underserved populations. Washington: The National Academies Press; 2011.
Institute of Medicine of the National Academies. Advancing oral health in America. Washington: The National Academies Press; 2011.
Weening-Verbree L, Huisman-de Waal G, van Dusseldorp L, van Achterberg T, Schoonhoven L. Oral health care in older people in long term care facilities: A systematic review of implementation strategies. Int J Nurs Stud. 2013;50:569–82.
Coker E, Ploeg J, Kaasalainen S. The effect of programs to improve oral hygiene outcomes for older residents in long-term care: a systematic review. Res Gerontol Nurs. 2014;7:87–100.
MacEntee MI, Kazanjian A, Kozak JF, Hornby K, Thorne S, Kettratad-Pruksapong M. A scoping review and research synthesis on financing and regulating oral care in long-term care facilities. Gerodontology. 2012;29:e41–52.
Matthews DC, Clovis JB, Brillant MGS, Filiaggi MJ, McNally ME, Kotzer RD, Lawrence HP. Oral health status of long-term care residents: a vulnerable population. J Can Dent Assoc. 2012;78:c3.
Shimazaki Y, Soh I, Koga T, Miyazaki H, Takehara T. Relationship between dental care and oral health in institutionalized elderly people in Japan. J Oral Rehabil. 2004;31:837–42.
Chalmers JM, Carter KD, Fuss JM, Spencer AJ, Hodge CP. Caries experience in existing and new nursing home residents in Adelaide, Australia. Gerodontology. 2002;19:30–40.
Patrick DL, Murray TP, Bigby JA, Auerbach J, Mullen J, Johnson DE, Bethel LA. The Commonwealth’s high-risk senior population: results and recommendations from 2009 statewide oral health assessment. Boston: Massachusetts Department of Public Health, Office of Oral Health; 2010.
Haumschild MS, Haumschild RJ. The importance of oral health in long-term care. J Am Med Dir Assoc. 2009;10:667–71.
US Department of Health & Human Services: Healthy People 2010, Volume II (second edition). http://www.healthypeople.gov/2010/Document/tableofcontents.htm#Volume2 (2000). Accessed May 29, 2016.
Emami E, de Souza RF, Kabawat M, Feine JS. The impact of edentulism on oral and general health. Int J Dent. 2013;2013:498305.
Awano S, Ansai T, Takata Y, Soh I, Akifusa S, Hamasaki T, Yoshida A, Sonoki K, Fujisawa K, Takehara T. Oral health and mortality risk from pneumonia in the elderly. J Dent Res. 2008;87:334–9.
Taylor GW, Loesche WJ, Terpenning MS. Impact of oral diseases on systemic health in the elderly: diabetes mellitus and aspiration pneumonia. J Public Health Dent. 2000;60:313–20.
Frenkel H, Matthews DC, Nitschke I. Prevention of oral diseases for a dependent population. In: MacEntee MI, Müller F, Wyatt CCL, editors. Oral healthcare and the frail elder. Ames: Wiley-Blackwell; 2010. p. 187–209.
Locker D, Slade G. Oral health and the quality of life among older adults: the oral health impact profile. J Can Dent Assoc. 1993;59:830–3. 7-8, 44.
Slade GD, Spencer AJ, Locker D, Hunt RJ, Strauss RP, Beck JD. Variations in the social impact of oral conditions among older adults in South Australia, Ontario, and North Carolina. J Dent Res. 1996;75:1439–50.
McNally ME, Matthews DC, Clovis JB, Brillant M, Filiaggi MJ. The oral health of ageing baby boomers: a comparison of adults aged 45-64 and those 65 years and older. Gerodontology. 2014;31:123–35.
Jablonski RA, Kolanowski AM, Litaker M. Profile of nursing home residents with dementia who require assistance with mouth care. Geriatr Nurs. 2011;32:439–46.
Berta W, Laporte A, Deber R, Baumann A, Gamble B. The evolving role of health care aides in the long-term care and home and community care sectors in Canada. Hum Resour Health. 2013;11:25.
Estabrooks CA, Squires JE, Carleton HL, Cummings GG, Norton PG. Who is looking after Mom and Dad? Unregulated workers in Canadian long-term care homes. Can J Aging. 2015;34:47–59.
Canadian Dental Association. Report on seniors' oral health. May 2008 edn. Ottawa: CDA; 2008.
Federal Provincial and Territorial Dental Working Group: Reducing dental disease: a Canadian oral health framework. Ottawa: Federal Provincial and Territorial Dental Working Group; 2013.
Compton SM, Yoon MN, Clovis JB. Assessing and taking action on oral health for older adults in Canada. Can J Dent Hygiene. 2013;47:182–88.
Chalmers JM, Pearson A. A systematic review of oral health assessment by nurses and carers for residents with dementia in residential care facilities. Spec Care Dentist. 2005;25:227–33.
Canadian Institute for Health Information. Resident Assessment Instrument (RAI) RAI-MDS 2.0 User's Manual, Canadian Version. Ottawa: CIHI; 2012.
Canadian Institute for Health Information. Data quality documentation, continuing care reporting system, 2013-2014. Ottawa: CIHI; 2015.
Poss JW, Jutan NM, Hirdes JP, Fries BE, Morris JN, Teare GF, Reidel K. A review of evidence on the reliability and validity of Minimum Data Set data. Healthc Manage Forum. 2008;21:33–9.
Hutchinson AM, Milke DL, Maisey S, Johnson C, Squires JE, Teare G, Estabrooks CA. The Resident Assessment Instrument-Minimum Data Set 2.0 quality indicators: a systematic review. BMC Health Serv Res. 2010;10:166.
interRAI: Instruments: Long-Term Care Facilities (LTCF). http://www.interrai.org/long-term-care-facilities.html (2016). Accessed 06 Feb 2016.
Mor V. A comprehensive clinical assessment tool to inform policy and practice: applications of the Minimum Data Set. Med Care. 2004;42:50–9.
Shin JH, Scherer Y. Advantages and disadvantages of using MDS data in nursing research. J Gerontol Nurs. 2009;35:7–17.
Hawes C, Morris JN, Phillips CD, Mor V, Fries BE, Nonemaker S. Reliability estimates for the Minimum Data Set for nursing home resident assessment and care screening (MDS). Gerontologist. 1995;35:172–8.
Sgadari A. Efforts to establish the reliability of the Resident Assessment Instrument. Age Aging. 1997;26:27–30.
Mor V, Angelelli J, Jones R, Roy J, Moore T, Morris J. Inter-rater reliability of nursing home quality indicators in the U.S. BMC Health Serv Res. 2003;3:20.
Hirdes JP, Poss JW, Caldarelli H, Fries BE, Morris JN, Teare GF, Reidel K, Jutan N. An evaluation of data quality in Canada's Continuing Care Reporting System (CCRS): secondary analyses of Ontario data submitted between 1996 and 2011. BMC Med Inform Decis Mak. 2013;13:27.
Hirdes JP, Mitchell L, Maxwell CJ, White N. Beyond the 'iron lungs of gerontology': Using evidence to shape the future of nursing homes in Canada. Can J Aging. 2011;30:371–90.
Estabrooks CA, Poss JW, Squires JE, Teare GF, Morgan DG, Stewart N, Doupe MB, Cummings GG, Norton PG. A profile of residents in prairie nursing homes. Can J Aging. 2013;32:223–31.
Achterberg WP, van Campen C, Pot AM, Kerkstra A, Ribbe MW. Effects of the Resident Assessment Instrument on the care process and health outcomes in nursing homes. A review of the literature. Scand J Rehabil Med. 1999;31:131–7.
Arvidson-Bufano UB, Blank LW, Yellowitz JA. Nurses' oral health assessments of nursing home residents pre- and post-training: A pilot study. Spec Care Dentist. 1996;16:58–64.
Ettinger RL, O'Toole C, Warren J, Levy S, Hand JS. Nursing directors' perceptions of the dental components of the Minimum Data Set (MDS) in nursing homes. Spec Care Dentist. 2000;20:23–7.
Guay AH. The oral health status of nursing home residents: What do we need to know? J Dent Educ. 2005;69:1015–7.
Cohen-Mansfield J, Lipson S. The underdetection of pain of dental etiology in persons with dementia. Am J Alzheimers Dis Other Demen. 2002;17:249–53.
Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977;33:159–74.
Folse GJ. National MDS and dental deficiency data reported by the US Health Care Financing Administration (HCFA). Spec Care Dentist. 2001;21:37–8.
American Educational Research Association (AERA), American Psychological Association (APA), National Council on Measurement in Education (NCME): Standards for Educational and Psychological Testing. 6th ed. edn. Washington; AERA, APA, NCME; 2014.
Streiner DL, Norman GR, Cairney J. Health measurement scales: a practical guide to their development and use. 5th ed. Oxford: Oxford University Press; 2015.
Estabrooks CA, Squires JE, Cummings GG, Teare GF, Norton PG. Study protocol for the translating research in elder care (TREC): building context – an organizational monitoring program in long-term care project (project one). Implement Sci. 2009;4:52.
Morris JN, Fries BE, Morris SA. Scaling ADLs within the MDS. J Gerontol A Biol Sci Med Sci. 1999;54:M546–M53.
Morris JN, Fries BE, Mehr DR, Hawes C, Phillips C, Mor V, Lipsitz LA. MDS Cognitive Performance Scale. J Gerontol A Biol Sci Med Sci. 1994;49:M174–M82.
Burrows AB, Morris JN, Simon SE, Hirdes JP, Phillips C. Development of a Minimum Data Set-based depression rating scale for use in nursing homes. Age Ageing. 2000;29:165–72.
Roderick JAL. A test of missing completely at random for multivariate data with missing values. J Am Stat Assoc. 1988;83:1198–202.
O’Brien RM. A caution regarding rules of thumb for Variance Inflation Factors. Quality Quantity. 2007;41:673–90.
Duffy RE, Mattson BJ, Zack M. Comorbidities among Ohio's nursing home residents with diabetes. J Am Med Dir Assoc. 2005;6:383–9.
Thai PH, Shuman SK, Davidson GB. Nurses' dental assessments and subsequent care in Minnesota nursing homes. Spec Care Dentist. 1997;17:13–8.
Wang TF, Chen IJ, Li IC. Associations between chewing and swallowing problems and physical and psychosocial health status of long-term care residents in Taiwan: A pilot study. Geriatr Nur (Lond). 2012;33:184–93.
Blaum CS, O'Neill EF, Clements KM, Fries BE, Fiatarone MA. Validity of the minimum data set for assessing nutritional status in nursing home residents. Am J Clin Nutr. 1997;66:787–94.
Nordenram G, Ljunggren G. Oral status, cognitive and functional capacity versus oral treatment need in nursing home residents: a comparison between assessments by dental and ward staff. Oral Dis. 2002;8:296–302.
Wyatt CC. Elderly Canadians residing in long-term care hospitals: Part II. Dental caries status. J Can Dent Assoc. 2002;68:359–63.
Arpin S, Brodeur JM, Corbeil P. Dental caries, problems perceived and use of services among institutionalized elderly in 3 regions of Quebec, Canada. J Can Dent Assoc. 2008;74:807.
Maupome G, Wyatt CC, Williams PM, Aickin M, Gullion CM. Oral disorders in institution-dwelling elderly adults: a graphic representation. Spec Care Dentist. 2002;22:194–200.
Adegbembo AO, Leake JL, Main PA, Lawrence HL, Chipman ML. The effect of dental insurance on the ranking of dental treatment needs in older residents of Durham Region's homes for the aged. J Can Dent Assoc. 2002;68:412–8.
Wyatt CC. Elderly Canadians residing in long-term care hospitals: Part I. Medical and dental status. J Can Dent Assoc. 2002;68:353–8.
Kim HY, Jang MS, Chung CP, Paik DI, Park YD, Patton LL, Ku Y. Chewing function impacts oral health-related quality of life among institutionalized and community-dwelling Korean elders. Community Dent Oral Epidemiol. 2009;37:468–76.
Chen X, Clark JJJ, Naorungroj S. Oral health in nursing home residents with different cognitive statuses. Gerodontology. 2013;30:49–60.
Bush HM, Dickens NE, Henry RG, Durham L, Sallee N, Skelton J, Stein PS, Cecil JC. Oral health status of older adults in Kentucky: results from the Kentucky Elder Oral Health Survey. Spec Care Dentist. 2010;30:185–92.
Preshaw PM, Walls AW, Jakubovics NS, Moynihan PJ, Jepson NJ, Loewy Z. Association of removable partial denture use with oral and systemic health. J Dent. 2011;39:711–9.
Dharamsi S, Jivani K, Dean C, Wyatt C. Oral care for frail elders: Knowledge, attitudes, and practices of long-term care staff. J Dent Educ. 2009;73:581–8.
Forsell M, Sjogren P, Kullberg E, Johansson O, Wedel P, Herbst B, Hoogstraate J. Attitudes and perceptions towards oral hygiene tasks among geriatric nursing home staff. Int J Dent Hyg. 2011;9:199–203.
Garrido Urrutia C, Romo Ormazabal F, Espinoza Santander I, Medics Salvo D. Oral health practices and beliefs among caregivers of the dependent elderly. Gerodontology. 2012;29:e742–e7.
Jablonski RA, Munro CL, Grap MJ, Schubert CM, Ligon M, Spigelmyer P. Mouth care in nursing homes: knowledge, beliefs, and practices of nursing assistants. Geriatr Nur (Lond). 2009;30:99–107.
Sonde L, Emami A, Kiljunen H, Nordenram G. Care providers' perceptions of the importance of oral care and its performance within everyday caregiving for nursing home residents with dementia. Scand J Caring Sci. 2011;25:92–9.
Ong G. Periodontal disease and tooth loss. Int Dent J. 1998;48:233–8.
Burt BA, Eklund SA. Dentistry, dental practice, and the community. 6th ed. St. Louis: Elsevier/Saunders; 2005.
Yao CS, MacEntee MI. Inequity in oral health care for elderly Canadians: part 1. Oral health status. J Can Dent Assoc. 2013;79:d114.
Friedman PK, Kaufman LB, Karpas SL. Oral health disparity in older adults: dental decay and tooth loss. Dent Clin North Am. 2014;58:757–70.
Luppa M, Luck T, Weyerer S, Konig HH, Brahler E, Riedel-Heller SG. Prediction of institutionalization in the elderly. A systematic review. Age Ageing. 2010;39:31–8.
Courtin E, Knapp M: Social isolation, loneliness and health in old age: a scoping review. Health Soc Care Community 2015. doi: 10.1111/hsc.12311.
Gardiner C, Geldenhuys G, Gott M: Interventions to reduce social isolation and loneliness among older people: an integrative review. Health Soc Care Community 2016. doi: 10.1111/hsc.12367.
Kassebaum NJ, Bernabe E, Dahiya M, Bhandari B, Murray CJ, Marcenes W. Global burden of untreated caries: a systematic review and metaregression. J Dent Res. 2015;94:650–8.
Kassebaum NJ, Bernabe E, Dahiya M, Bhandari B, Murray CJ, Marcenes W. Global burden of severe periodontitis in 1990-2010: a systematic review and meta-regression. J Dent Res. 2014;93:1045–53.
Kassebaum NJ, Bernabe E, Dahiya M, Bhandari B, Murray CJ, Marcenes W. Global Burden of Severe Tooth Loss: A systematic review and meta-analysis. J Dent Res. 2014;93:20s–8s.
Thomson WM. Epidemiology of oral health conditions in older people. Gerodontology. 2014;31:9–16.
Dye BA. Global periodontal disease epidemiology. Periodontol 2000. 2012;58:10–25.
Alberta Government. 2013/2014 Alberta long-term care resident profile. Edmonton: Alberta Government; 2015.
Government of Saskatchewan - Ministry of Health. Program guidelines for special-care homes. Regina: Government of Saskatchewan; 2013.
Winnipeg Region Health Authority (WRHA). Caring for people, caring for health: Winnipeg Region Health Authority annual report 2009/2010. Winnipeg: WHRA; 2010.
Hoben M, Chamberlain SA, Knopp-Sihota JA, Poss JW, Thompson GN, Estabrooks CA. Impact of symptoms and care practices on nursing home residents at the end of life: A rating by front-line care providers. J Am Med Dir Assoc. 2016;17:155–61.
Korn EL, Graubard BI. Epidemiologic studies utilizing surveys: accounting for the sampling design. Am J Public Health. 1991;81:1166–73.
This study was funded by a Canadian Institute of Health Research grant in aid (MOP 53107). MH has conducted this research within his postdoctoral fellowship, funded by Alberta Innovates―Health Solutions (AIHS) (201500543).
Availability of data and materials
The data supporting the conclusions of this article are housed in the secure and confidential Health Research Data Repository (HRDR) in the Faculty of Nursing at the University of Alberta (https://www.ualberta.ca/nursing/research/research-supports-and-services/hrdr), in accordance with the health privacy legislation of participating TREC jurisdictions. These health privacy legislations as well as the ethics approval covering TREC data does not allow the removal of completely disaggregated data (i.e., resident-level records) from the HRDR – even if de-identified. Aggregated summary data specific to this manuscript can be requested through the TREC Data Management Committee (firstname.lastname@example.org) on the condition that researchers meet and comply with the TREC and HRDR data confidentiality policies.
MH developed the research design, carried out the analyses, drafted the figure and tables and prepared the first draft of the manuscript. JWP contributed to the analyses. PGN contributed to the theoretical framing of the paper. CAE supervised the project. All authors reviewed and commented on several versions of the manuscript and approved the submitted version.
The authors declare that they have no competing interests.
Consent for publication
Not applicable; the manuscript does not include any individual person’s data.
Ethics approval and consent to participate
Ethical approvals for TREC were obtained from the University of Alberta Health Research Ethics Board (File B-051007), University of Calgary Conjoint Health Research Ethics Board (Ethics ID E-21379), University of Saskatchewan Behavioural Research Ethics Board (BEH08e165), and University of Manitoba Fort Garry Campus Research Ethics Board (Protocol E2008:010). Operational approvals were obtained from all relevant health care organizations.
Justification for included independent variables/covariates. (PDF 124 kb)
Comparison of baseline outcomes between residents with exactly two assessments and residents with three or more assessments. (PDF 16 kb)
Results of the multicollinearity assessment. (PDF 191 kb)
Results of the binary logistic regression models. (PDF 36 kb)
Additional General Estimating Equations (final, fully adjusted model), using variables of a) the assessment conducted previously to the assessment including the oral/dental issues variable and b) the admission assessment. (PDF 19 kb)
About this article
Cite this article
Hoben, M., Poss, J.W., Norton, P.G. et al. Oral/dental items in the resident assessment instrument – minimum Data Set 2.0 lack validity: results of a retrospective, longitudinal validation study. Popul Health Metrics 14, 36 (2016). https://doi.org/10.1186/s12963-016-0108-y
- Dental care for aged
- Quality improvement
- Nursing homes
- Outcome assessment (Health Care)
- Epidemiologic geriatric assessment