Abstract

The purpose of this study was to develop and validate an effective instrument to measure undergraduate students’ self-efficacy in engaging with information for online coursework. Recognizing the increasing importance of information literacy in online learning environments, the Online Information Literacy Self-Efficacy (OILS) instrument was designed to assess key competencies associated with students’ abilities to engage with information effectively. The validity and reliability of the OILS instrument were examined using exploratory factor analysis (EFA) and reliability analysis. The instrument initially comprised 30 items, grounded in two authoritative frameworks: the Association of College and Research Libraries (ACRL) Framework for Information Literacy and the Association of American Colleges and Universities (AAC&U) Information Literacy VALUE Rubric. Survey responses from 259 undergraduate students enrolled in online courses at a large Midwestern US R1 institution were analyzed using EFA and reliability analysis. EFA results revealed a four-factor structure explaining substantial variance in the item response patterns: scoping research topics, obtaining information, evaluating information quality and producing research documents, and crediting sources. These four factors demonstrated high reliability, with omega coefficients ranging from 0.891 to 0.960. One item was recommended for removal due to cross-loadings and skewness. Exploratory measurement invariance analyses suggested that the factor structure was comparable for students with and without prior online learning experience. The findings support the OILS instrument’s utility in measuring students’ confidence in applying information literacy skills in online learning contexts, offering valuable insights for educators and librarians to enhance instructional strategies and support student success.

Keywords: information literacy, self-efficacy, online learning, factor analysis, undergraduate education

Introduction

In today’s increasingly digitized academic landscape, equipping students in higher education with the skills necessary to engage critically and effectively with information in online coursework is essential. Information literacy—defined as the ability to use information critically to learn and solve problems—has therefore emerged as a key competency in supporting students’ academic success and lifelong learning (Bradley, 2013). As online learning platforms become the norm, students constantly face challenges in locating highly relevant information, discerning credible sources, synthesizing information, and applying it to their academic tasks.

An essential concept related to information literacy in online settings is self-efficacy, which significantly influences students’ motivation and engagement. Self-efficacy, rooted in Bandura’s (1997) social cognitive theory, refers to an individual’s belief in their capacity to execute behaviors necessary to achieve specific performance outcomes. Honicke and Broadbent’s (2016) systematic review demonstrated that higher self-efficacy empowers students to approach challenges with confidence, viewing obstacles as opportunities for learning rather than threats. This mindset can lead to greater persistence in problem-solving and more proactive engagement in learning activities, which are crucial in an online education context. Studies have shown that higher self-efficacy is linked to increased motivation, persistence, and overall academic performance (Adigue et al., 2023; Honicke & Broadbent, 2016). In the context of online learning, where students navigate information-rich but unstructured environments, self-efficacy for information literacy becomes a critical predictor of their ability to perform information-related tasks. Consequently, it is imperative to both assess and support the development of students’ self-efficacy in online information literacy throughout their online learning experience.

Technology-enhanced online education also provides various opportunities for developing information literacy skills for online courses. Online learning platforms can leverage interactive tools and resources to promote active learning and collaboration among students. More recently, generative artificial intelligence (AI) tools have become embedded in many of these platforms, further mediating how students search for, evaluate, synthesize, and produce information (Giannakos et al., 2024; Kassorla et al., 2024; UNESCO, 2023). For instance, structured online modules emphasizing the critical evaluation of information sources, including both human-authored and AI-generated content, can be implemented, thereby enhancing the learning experience (Hovious, 2024; Johnston, 2010; McClellan, 2016). Furthermore, instructors can incorporate assessments that evaluate not only content mastery but also students’ information literacy skills, which remain essential for exercising judgment, credibility assessment, and ethical information use in AI-augmented learning contexts, ensuring a comprehensive approach to education that prepares students for real-world challenges (Kassorla et al., 2024; McGrew et al., 2018; Weaver, 2024).

The purpose of this study is to develop an effective instrument to measure undergraduate students’ perceived confidence in engaging with information for online coursework. Specifically, this study employed exploratory factor analysis (EFA) to empirically test the instrument and examine its psychometric properties. The study has two specific research questions:

1. Which set of items, determined through psychometric analyses, should be included in the final instrument to assess self-efficacy in online learning information literacy?
2. What is the reliability and validity evidence of the developed instrument?

Conceptual Framework

The conceptual framework guiding this study is informed by two authoritative resources in the field of information literacy. The first is the Association of College and Research Libraries (ACRL) Framework for Information Literacy for Higher Education (ACRL, 2015). This framework outlines six core concepts that shape students’ understanding and practice of information literacy in academic contexts:

• Authority is constructed and contextual: Recognizes that information authority varies based on context and audience.
• Information creation as a process: Emphasizes the iterative nature of information production.
• Information has value: Highlights the ethical and economic considerations tied to information use.
• Research as inquiry: Frames research as an ongoing, iterative process of questioning.
• Scholarship as conversation: Promotes the view of academic work as a dialogue among scholars.
• Searching as strategic exploration: Encourages strategic and adaptable information-seeking behaviors.

Collectively, this framework establishes a holistic view of information literacy as a set of interconnected, critical-thinking practices essential for student success in higher education.

Complementing the ACRL framework is the Association of American Colleges and Universities’ (AAC&U) Information Literacy VALUE Rubric (AAC&U, 2009), which provides a measurable, skills-based approach to assessing information literacy competencies. The VALUE Rubric includes five key dimensions:

• Determine the extent of information needed: Focuses on defining the scope of research inquiries.
• Access the needed information: Ensures students can efficiently retrieve relevant sources.
• Evaluate information and its sources critically: Encourages discernment and reliability checks.
• Use information effectively to accomplish a specific purpose: Emphasizes the application of information toward academic goals.
• Access and use information ethically and legally: Underscores the importance of academic integrity and compliance with legal guidelines.

Together, these two foundational frameworks provide both the theoretical and practical bases for the development and validation of the Online Information Literacy Self-Efficacy (OILS) instrument factors used in this study.

Methods

Participants

The study sample consisted of undergraduate students enrolled in 40 online courses across 73 sections at a Midwestern US R1 university. A total of 1,832 undergraduate students were invited to participate. Institutional review board approval was obtained prior to data collection. All the online courses’ instructors were informed via email regarding the purpose of the study. Subsequently, all students received an email invitation to participate, followed by two reminder emails, which were distributed by the university’s institutional registrar. Twenty percent of the students who completed the survey were randomly drawn and each received an incentive of a $5 Amazon e-gift card.

A total of 354 students submitted survey responses. Of these, 259 students completed at least part of the information literacy scale and were included in the final analysis. Among these, 215 were domestic students and 43 were international students (one participant declined to respond about their national origin). In terms of the academic levels, 5 students had freshman status, 53 had sophomore status, 90 had junior status, and 111 students had senior status. Additionally, 67 students reported that they were enrolled in an online class for the first time, while 192 students had taken at least one online class previously.

Instrumentation

Based on the two framework sources, the project team initially defined the construct of information literacy for online learning to consist of five main ability components: (a) determining the extent of information needed to complete a research assignment, (b) accessing the information needed, (c) evaluating obtained information critically, (d) accessing and crediting sources, and (e) producing the assignment submission. Two librarian authors independently reviewed the draft item set for content alignment and clarity, settling on 30 items to include in the initial version of the information literacy instrument. The items directed participants to self-rate various aspects of their information literacy on a 5-point Likert scale: 1 = almost always true, 2 = often true, 3 = sometimes true, 4 = usually not true, and 5 = almost never true. The complete text of the items is provided in Appendix A. The survey instrument contained an initial section requesting basic demographic information, followed by two psychometric scales: the information literacy self-efficacy scale composed of 30 items, and validation scale consisting of 8 multiple-choice information literacy knowledge items (Appendix A). The survey was administered using an online platform (Qualtrics).

Analytic Methods

To analyze evidence about the meaning of total scores from the scale, we conducted traditional item analysis, EFA, and measurement invariance analysis. We first computed item analysis statistics: the mean, standard deviation, and skewness for each item’s rating scale responses. We used these statistics to identify any items that showed little variation in the population, had rating scale points that were too difficult or easy to endorse, or had a badly skewed response distribution (which would violate a key assumption of EFA, the next step).

After examining item descriptive statistics as a screening step, we applied EFA to the item response data. The goals of EFA were to identify and describe any latent factor(s) formed by patterns in the item response data and to compare the empirical latent factor structure to our proposed five-factor measurement model. Because any traits underlying responses to the information literacy self-efficacy items seemed likely to be interrelated, an oblique rotation, geomin, was used to find a simple-structure solution for each specified number of factors. Model parameters were estimated using maximum likelihood. We used criteria suggested by Little (2013) for the root mean square error of approximation (RMSEA) and the Tucker-Lewis index (TLI) to judge model fit. Then, we selected the most plausible measurement model for the observed data considering theory about information literacy, statistical model fit indices, and the principle of parsimony.

Once we had mapped items to underlying latent factors, we estimated the reliability of the total score on each factor using coefficient omega (McDonald, 1970). Coefficient omega is a suitable score reliability coefficient when factor loadings vary across items. To address concerns that the underlying factor structure might differ between students taking their first online course and students who had previously taken online courses, we examined the factor structure separately within these two groups. Because the scale did not yet have a well-established factor structure (i.e., replicated in an independent sample), we again used EFA modeling, now within these two subgroups of respondents, to produce a tentative assessment of the scale’s measurement invariance across these respondent groups. Finally, we computed the correlation between each latent factor underlying the information literacy self-efficacy scale, and the total score on the eight information literacy knowledge items, as a form of external evidence for the meaning of scores from the information literacy self-efficacy scale. All analyses were conducted using Mplus software.

Results

Item Analysis

Descriptive statistics for responses to the 30 information literacy self-efficacy rating scale items are reported in Table 1. The mean response of these questions ranges from 1.582 to 2.413. Given the rating scale labels, the more strongly respondents tended to agree with an item statement, the lower its mean value would be. The lowest mean value (1.582) was observed for the item about citing all reference sources. The highest mean (2.413) was for the item about searching using keywords and Boolean logic. The range of mean values suggests respondents generally reported being able to carry out the items’ information literacy behaviors on most, or at least some, occasions. Items had similar standard deviation values, and no item showed a low standard deviation that might raise questions about that item’s utility for measuring information literacy. Items from the “accessing information needed” ability domain, some of which referred to library resources, displayed slightly higher response variability than other items. One item (10) about seeking “information on the Web” displayed notably high skewness (1.53), meaning the respondents tended to select answers from one end, without much variability or discrimination. The rate of missing data did not exceed 7% on any item.

Table 1

Descriptive Statistics for Survey Items

Note. The text of the scale items is abbreviated for display purposes.

Exploratory Factor Analysis

Because the design specifications for the instrument consisted of five categories of information literacy skills, we examined results for EFA models containing between one and seven underlying latent factors. (For the most complex hypothesized model, with seven factors, the estimation process did not converge.) One factor in the six-factor model had no items that loaded highly (> .30) on it, so the loading pattern for the six-factor model was not substantively interpretable. The five-factor model contained four well-defined factors and a fifth factor on which only two items loaded highly. These two items, about ability to “seek information on the Web” (10) and to “use Internet search tools” (11), seemed to be measuring highly related behaviors. Because the response distribution of the item about seeking information on the Web also displayed high skewness, indicating that this item might be too easy in the target respondent population, we suggested eliminating that item from future versions of the instrument. Eliminating that item produced a more parsimonious four-factor EFA model with fit that we judged to be adequate but not excellent (see Table 2). Little (2013, pp. 109, 115) suggested that an RMSEA value of .08 indicates “mediocre” to “acceptable” fit and a TLI value of .89 indicates “mediocre” fit. As previously, models with five or six factors displayed factor loading patterns that were not interpretable. Models with fewer than four factors fit the data poorly. (All results are available from the corresponding author.) Table 3 displays the factor loadings for the four-factor model. The four factors in the preferred model are interpreted as (a) scoping the research topic and identifying needed information, (b) obtaining information via library Web pages and other sources, (c) evaluating information quality and producing research documents, and (d) crediting sources.

Table 2

Fit Statistics for Six Exploratory Factor Analysis Models

Note. RMSEA = root mean square error of approximation; TLI = Tucker-Lewis Index.
* Chi-square model fit test statistic is statistically significant (p < .001).

Table 3

Factor Loadings for the Four-Factor EFA Model

Note. N = 259. Boldface indicates the factor that a certain item is judged to be primarily measuring.
*Factor loading is significantly different from zero (p < .05).

Factor Total Score Reliability

We also estimated the omega reliability coefficient for observed total scores on each factor in the best EFA model, the four-factor model. The total score reliability coefficient for Factor 1 (Scoping the research topic) was 0.902, for Factor 2 (Obtaining information) was 0.917, for Factor 3 (Evaluating information quality and producing research documents) was 0.960, and for Factor 4 (Crediting sources) was 0.891. Omega reliability coefficients ≥ 0.80 may be treated as “sufficiently high” for score use in most social science research (Raykov & Marcoulides, 2011, p. 169). DeVellis (2017) suggested that when score reliability values exceed 0.90, reducing the number of items could be considered.

Measurement Invariance Analysis

We also attempted to test measurement invariance of the information literacy self-efficacy scale between students who were taking an online course for the first time (67 students) and students who had previously taken one or more online courses (192 students). The sample size of the smaller group was marginal for factor analysis, so we conceptualized this analysis as highly exploratory. We tested “configural” invariance, first, constraining a multiple-group confirmatory factor analysis model so that both groups had the same four-factor structure but possibly different factor loadings. The chi-square test of model fit indicated that the configural invariance model did not fit the data—χ² (742) = 1370.83, p < .0001—but fit indices suggested it was marginally adequate: TLI = 0.855; RMSEA = 0.081. We proceeded to test a “scalar” or “strong” invariance model with the same four-factor structure, constraining factor loadings, and item intercepts to be equal across groups. The fit of the scalar invariance model was no worse than that of the configural invariance model: χ² (50) = 63.76, p = .091. We interpret these results as suggesting that the information literacy self-efficacy items may be measuring the same underlying traits in the same metric, regardless of whether students have previously taken online courses or not, but we also recommend that this analysis should be repeated in a larger sample from the same population to provide stronger evidence.

Correlations Between Self-Reported Information Literacy Abilities and Information Literacy Knowledge

To analyze external evidence for the meaning of scores from our information literacy self-efficacy instrument, we computed correlations between factors in the self-efficacy and total scores on the eight-item information literacy knowledge test. Because lower scores on the information literacy self-efficacy scale indicated more consistent ability to carry out information literacy activities, we reverse-coded the information literacy knowledge test scores so that, similarly, lower values corresponded to higher information literacy knowledge. The correlation of the information literacy test scores with Factor 1 (Scoping the research topic) was 0.122; with Factor 2 (Obtaining information), 0.134; with Factor 3 (Evaluating information quality and producing research documents), 0.158; and with Factor 4 (Crediting sources), 0.185. Although they are in the expected positive direction, these correlations are fairly small, indicating perhaps modest evidence of students’ ability to accurately self-report on their own information literacy. We expect the magnitude of the correlation may have been attenuated to some extent due to limited reliability of the information literacy knowledge test scores, which consisted of only eight binary-scored items.

Future Research

Although the instrument demonstrated promising preliminary results, additional confirmatory validation is advised to further establish its reliability and verify the underlying dimensional structure. It is also recommended that future studies employ confirmatory factor analysis to assess the predictive validity of the OILS instrument, as the current study primarily used EFA and reliability analysis to examine the instrument’s psychometric properties.

In addition, future research should examine how emerging generative AI tools intersect with students’ information literacy self-efficacy. While the OILS instrument captures foundational competencies such as evaluating information quality, ethical use, and attribution, AI-mediated information practices introduce new dimensions of judgment, transparency, and responsibility that warrant explicit investigation. Subsequent studies could extend the instrument by incorporating items that assess students’ confidence in critically engaging with AI-generated content, disclosing AI use, and integrating AI tools responsibly into academic work.

Conclusions

The Online Information Literacy Self-Efficacy (OILS) instrument, as detailed in Appendix C, presents a robust, nuanced measure of students’ perceived abilities in information literacy within online learning environments. Through empirical testing using exploratory factor analysis and reliability analysis, the instrument’s four-factor structure—scoping research topics, obtaining information, evaluating information quality and producing research documents, and crediting sources—was confirmed, with strong reliability coefficients. These findings highlight the critical role of self-efficacy in online information literacy and provide educators and librarians with a practical tool to assess and support student readiness in navigating complex information landscapes. Future research is encouraged to examine other types of validity evidence, such as confirmatory factor analyses and further predictive score correlations, ensuring the instrument’s applicability across diverse student populations and learning contexts.

References

Adigue, A. P., Tumacder, J. R. O., & Urbano, J. M. (2023). Self-efficacy and learning strategies in the context of online learning. In Proceedings of the 3rd International Conference on Education and Technology (ICETECH 2022) (pp. 547-564). Atlantis Press. https://doi.org/10.2991/978-2-38476-056-5_55

Association of American Colleges and Universities. (2009). Information literacy VALUE rubric. https://www.aacu.org/value/rubrics/information-literacy

Association of College and Research Libraries. (2015). Framework for information literacy for higher education. American Library Association. https://www.ala.org/acrl/standards/ilframework

Bandura, A. (1997). Self-efficacy: The exercise of control. W. H. Freeman.

Bradley, C. (2013). Information literacy in the programmatic university accreditation standards of select professions in Canada, the United States, the United Kingdom, and Australia. Journal of Information Literacy, 7(1), 44-68. https://doi.org/10.11645/7.1.1785

DeVellis, R. F. (2017). Scale development: Theory and applications. SAGE Publications.

Giannakos, M., Azevedo, R., Brusilovsky, P., Cukurova, M., Dimitriadis, Y., Hernandez-Leo, D., Järvelä, S., Mavrikis, M., & Rienties, B. (2024). The promise and challenges of generative AI in education. Behaviour & Information Technology, 44(11), 2518-2544. https://doi.org/10.1080/0144929X.2024.2394886

Honicke, T., & Broadbent, J. (2016). The influence of academic self-efficacy on academic performance: A systematic review. Educational Research Review, 17, 63-84. https://doi.org/10.1016/j.edurev.2015.11.002

Hovious, A. (2024). Information creation as an AI prompt: Implications for the ACRL framework. Kansas Library Association College and University Libraries Section Proceedings, 14(1), Article 5. https://doi.org/10.4148/2160-942X.1094

Johnston, N. (2010). Is an online learning module an effective way to develop information literacy skills? Australian Academic & Research Libraries, 41(3), 207-218. https://doi.org/10.1080/00048623.2010.10721464

Kassorla, M., Georgieva, M., & Papini, A. (2024). AI literacy in teaching and learning: A durable framework for higher education. EDUCAUSE. https://www.educause.edu/content/2024/ai-literacy-in-teaching-and-learning/executive-summary

Little, T. D. (2013). Longitudinal structural equation modeling. Guilford Press.

McClellan, S. (2016). Teaching critical thinking skills through commonly used resources in course-embedded online modules. College & Undergraduate Libraries, 23(3), 313-327. https://doi.org/10.1080/10691316.2014.987416

McDonald, R. P. (1970). The theoretical foundations of principal factor analysis, canonical factor analysis, and alpha factor analysis. British Journal of Mathematical and Statistical Psychology, 23(1), 1-21. https://psycnet.apa.org/doi/10.1111/j.2044-8317.1970.tb00432.x

McGrew, S., Breakstone, J., Ortega, T., Smith, M., & Wineburg, S. (2018). Can students evaluate online sources? Learning from assessments of civic online reasoning. Theory & Research in Social Education, 46(2), 165-193. https://doi.org/10.1080/00933104.2017.1416320

Raykov, T., & Marcoulides, G. A. (2011). Introduction to psychometric theory. Routledge. https://doi.org/10.4324/9780203841624

UNESCO. (2023, September 7). Guidance for generative AI in education and research. https://www.unesco.org/en/articles/guidance-generative-ai-education-and-research

Weaver, K. D. (2024). The artificial intelligence disclosure (AID) framework: An introduction. College & Research Libraries News, 85(10), 407-411. https://doi.org/10.5860/crln.85.10.407

Appendix A

Initial Online Information Literacy Self-Efficacy (OILS) Instrument

Appendix B

Eight Multiple-Choice Information Literacy Knowledge Items

Correct answers are marked in italics.

Q1. When performing research, one important part of forming a search strategy is to:

1. Locate books using the library’s online catalog
2. Search a computer database for articles
3. Analyze your topic to identify alternative keywords/concepts
4. Check the Internet for background information on your topic

Q2. The best sites always come up first in a search result. (Yes; No)

Q3. Read the following statement and answer the question below:

Statement: “Describe the effects of automobile emissions on air quality.”

Which of the following keyword examples may yield the best results for the statement above in an electronic database search? (The * indicates multiple suffix possibilities.)

1. automobile* and effects
2. automobile* and emission* and air quality
3. effects and emission* and quality
4. emission* and air quality

Q4. To refine your Web search results, you should:

1. Use the advanced search feature
2. Use only one search engine
3. Limit your results to the first 10 pages
4. Use the built-in Help

Q5. Which of the following is a characteristic of a scholarly journal?

1. The presence of glossy photos and advertisements
2. Articles are written by people from a variety of fields
3. Footnotes and bibliographies are regularly used to cite sources
4. Topics covered are not confined to one major field of study

Q6. When evaluating resources, which one is the least important to consider?

1. Document format
2. Publishing dates
3. Publisher
4. Author

Q7. Read the following statement and answer the question below:

Statement: “Describe the effects of automobile emissions on air quality.”

Which source would most likely provide you with objective information for the main concepts in the statement?

1. A personal interview with an influential lobbyist
2. A website that advocates clean air
3. The latest annual report from a major automobile manufacturer
4. A study featured in a peer-reviewed periodical

Q8. When evaluating resources, which of the following statements are not correct?

1. Check the author’s authority to write on the topic
2. Use only the most current resources
3. Read the mission statement of a Web page
4. Find out if others have cited the resource
5. Use resources that include different viewpoints

Appendix C

Online Information Literacy Self-Efficacy (OILS) Instrument

An Exploratory Factor Analysis of the Online Information Literacy Self-Efficacy (OILS) Instrument by Wei Zakharov, Anne Traynor, and Clarence Maybee is licensed under a Creative Commons Attribution 4.0 International License.