Development and Validation of the Sense of Online Community Scale

This study focused on the development and validation of the Sense of Online Community Scale (SOCS), which includes 28 Likert-type scale items across six subscales: (a) program community, (b) program academic activities, (c) program social activities, (d) institutional academic activities, (e) institutional social activities, and (f) affiliation. The validation process included an implementation with 293 learners enrolled in online programs at a higher education institution in the eastern United States. The model was evaluated with and without outliers, and results show that the model aligned well with the SOCS. The means of all items except one exceeded 3.5 on a 5.0 scale, ranging from 1 (strongly disagree) to 5 (strongly agree). Time in an online program was not a statistically significant predictor of the validation model, though most of our participants were in the first or second year of their degree programs. Findings demonstrate that the SOCS is a reliable and valid instrument that other researchers may use to investigate community in online environments on both the program and the institutional level.


Literature Review
Although most research focuses on course-based communities in online degree programs (O' Shea et al., 2015;Rockinson-Szapkiw & Wendt, 2015;Speiser et al., 2022), the concept of extending community to other settings is not new.Prior to online learning, researchers discussed the need to extend community concepts beyond proximate, physical spaces to consider disbursed relationships (McMillan & Chavis, 1986;Wellman, 1979).These concepts heralded views of community in online settings (Rovai, 2001) and pointed to broader foci of community establishment.Hill (1996) indicated that communities look and function differently in classrooms and programs.Wenger et al. (2002) stated that community members are often associated with myriad communities, some accounting for larger social perspectives.In higher education settings, some researchers highlight the individual, the course, and the institution as separate community layers (Bolliger et al., 2019;Glazer et al, 2013;Palloff & Pratt, 2007;Shepherd & Bolliger, 2019).Jason et al. (2015) referred to these layers as micro and macro elements of community.In these instances, a sense of community was associated with individual, instructor, and institutional factors.Shepherd

216
(2023a) further included a professional dimension that begins within programs but extends beyond degree attainment.The following sections describe these layers.

Institutional Community
Higher education institutions exert time and energy to make their campuses welcoming and inviting.
Physical buildings and manicured grounds provide a sense of prestige (Nathan, 2005).Recreation facilities, updated housing, museums, and other amenities attract students.Yet, community often begins prior to entering a campus.Decades of academic tradition, collegiate sports, university colors, mascots, and logos brand institutions of higher education (Dennis et al., 2016).These resources provide a sense of affiliation and shared interest among individuals on and off campus.Indeed, many aspects of higher education promote community.Admission requirements formalize membership and instill a sense of belonging (Shepherd & Bolliger, 2023a;Tinto, 1999).Administrative policies and procedures articulate shared governance among students, faculty members, and professionals while encouraging continued and sustained interaction to promote learning among groups with similar interests and goals (Smith et al., 2017;Tinto, 2012;Wenger et al., 2002).These interactions over time foster trust, a sense of belonging and value, and interdependence, and they add to the shared history that influences the institution's reputation and brand (Dennis et al., 2016).The role of the institution for online students follows a similar trajectory.
Although physical grounds and buildings may be less proximate, student ties to a physical, storied location influence a sense of membership.Indeed, most online students live within an hour of campus (Seaman et al., 2018;Shepherd & Bolliger, 2023b;Xu & Jaggars, 2013) and are aware of its physical and social presence in the surrounding community.Existing policies and procedures also provide a guide for community entrance, interaction, and goal attainment among online students (Shepherd & Bolliger, 2023a;Wenger et al., 2002).
Institutions play a large role in helping students enter and feel that they belong in learning communities (Lee & Choi, 2011;Tinto, 1999).Academic and social programs help students enter and maintain success in the community.Academic supports such as writing centers, libraries, and career and counseling centers provide needed services for diverse students (Kang & Pak, 2023;Milman et al., 2015;Trespalacios et al., 2023).So, too, do the myriad student groups, clubs, and activities that help students locate others with similar interests and aid the transition to university life (Exter et al., 2009;Nathan, 2005;Shepherd & Bolliger, 2019;Tinto, 1999).These resources promote a sense of belonging and safety.They also promote trust that individual goals will be attainable at the institution.

Program Community
One focus of higher education is to help students narrow or refine their learning interests to those that match degree programs.Although university services focus on the broader community, program areas support more specific academic and professional goals.Within programs, learners find course sequences and action plans designed to help them enter or progress in a desired profession.Within programs, learners interact with others who have similar goals and interests; they devote their time and energy to extend learning, collaborate with others, and strengthen professional relationships.Program policies and procedures provide guidance about how to navigate the environment successfully, and how to deepen involvement with the learning community (Tinto, 1999).
A portion of community building occurs in courses.Within courses, students become better acquainted with each other and their instructors.They also gain additional knowledge regarding program and professional interests.Group projects, discussions, instructor feedback, and other activities are often cited as community promoters (Milman et al., 2015;O' Shea et al., 2015;Rockinson-Szapkiw & Wendt, 2015;Shepherd & Bolliger, 2019, 2023b).Although courses are useful for community promotion, they may be insufficient to develop sustained communities, particularly ones that extend beyond graduation and help students enter a larger profession (Shepherd & Bolliger, 2023a, 2023b).Courses are limited in duration, ranging from a few weeks to a few months.These short durations may provide insufficient interaction to yield the perceptions of trust, safety, belonging, and interdependence needed for sustained community formation (Bellah et al., 1985;Liu et al., 2007;Smith et al., 2017).Course-based communities put the onus of development and maintenance on individual faculty members.This onus is rarely supported in institutions of higher education and may not be sustainable (Bolliger et al., 2019;Fong et al., 2016;Larson & James, 2022).Additional services are required to sustain interaction, develop shared goals and histories, bolster a sense of belonging, and promote interdependence (Jason et al., 2015;Lee & Choi, 2011;Milman et al., 2015;Shepherd & Bolliger, 2019, 2023b).
Like the institutional level, program support consists of academic and social endeavors.Academic endeavors provide background knowledge to further enter the learning community, extend opportunities to collaborate and grow the knowledge base over time, and facilitate entrance or access to professional communities (Aldosemani et al., 2016;Wenger et al., 2002).These services (e.g., individual and group advising, research participation, reading groups, service learning opportunities, intern-and externships) extend interactions among students, faculty, and others over time; are not limited to course enrollments; and may broaden community foci (Shepherd & Bolliger, 2023b).Thus, programs represent a microcommunity within higher education institutions as well as within related professions.
Yet, social activities are also important for community formation and maintenance (Shepherd & Bolliger, 2019, 2023b).Social events promote shared experiences in non-threatening environments.These experiences provide easy access to faculty and other professionals that may otherwise seem beyond reach to online students (Exter et al., 2009).Additionally, social events may not be limited to online activities.
Whereas some research suggests that reminders of on-campus activities further alienate distance students because they are reminded of the fewer services available to them (O' Shea et al., 2015), other studies suggest that campus-specific social events are helpful for community formation because most online students live within close proximity of campus (Shepherd & Bolliger, 2023b).

Individual
Regardless of the services provided for community facilitation and maintenance, students play an integral role in community formation.Most online students are nontraditional learners (Kang & Pak, 2023;Seaman et al., 2018).They are more likely to be older students, have dependent care responsibilities, and have years of experience in an occupation (Milman et al., 2015;Seaman et al., 2018).These students come with myriad support communities.While the majority indicate they want a sense of community within their degree program, a smaller number state they have no desire for student and faculty interaction and the endeavors that promote it (Exter et al., 2009;O' Shea et al., 2015;Shepherd & Bolliger, 2023b).Indeed, when faculty members were surveyed about the importance of community in online degree programs, several indicated that limited student attendance made them second-guess their efforts (Bolliger et al., 2019).Expressed interests in community formation may not equate with actual effort.Community partners may need to better educate students regarding the purposes behind event creation and the role of community during and after graduation (Shepherd & Bolliger, 2023b).Additionally, it is likely that time in the program influences the desire for specific institutional and program activities.Those entering the program may desire social events as opportunities to get to know faculty members, community partners, and colleagues (Baker & Pifer, 2011;Pifer & Baker, 2016).More established students may desire opportunities to coconstruct knowledge and network with the larger profession.Strengthening professional networks and communities may be a larger focus of those nearing graduation.Yet, little research has examined the time factors of community formation (Smith et al., 2017).

Research Purpose and Questions
The purposes of this study were to develop the Sense of Online Community Scale (SOCS), based on the framework developed by Shepherd and Bolliger (2023a), who focused on academic and social activities offered on the program and institutional levels, and to establish the instrument's reliability and construct validity.More specifically, the objectives of the study were to (a) develop an instrument to measure students' sense of community in online academic programs, (b) validate the instrument by verifying its factor structure, and (c) examine the relationships of the latent variables to time in an online program.

Method
The instrument was developed in two phases.First, several items were developed in different categories by the researchers.Once a first draft was generated, the instrument was reviewed by a panel of experts.Second, the instrument was validated by conducting a confirmatory factor analysis.

Phase 1: Instrument Development
Scale items were developed after an extensive literature search.Thirty items rated on a 5-point Likert-type scale (1 = strongly disagree, 2 = disagree, 3 = neither agree nor disagree, 4 = agree, and 5 = strongly agree) were developed for all factors.A "not applicable" response category was also included for all items.To ensure face validity, a panel of four experts reviewed the instrument prior to data collection.These individuals had at least six years of online teaching experience in higher education and were active researchers in online learning environments.All experts had terminal degrees; three faculty members held the rank of associate professor, and one person held the rank of professor emerita.
Panel members received instructions and the instrument via email.They were informed about the aim of the instrument and were instructed to review all items and categories for clarity and fit, modify items, and add or delete statements.Based on the reviewers' feedback, several items were revised, two items were deleted, and items that addressed connectedness, trust, and affiliations were grouped into categories.The final version of the instrument included 28 Likert-type statements, one open-ended question, seven questions pertaining to participants' academic programs, and three demographic questions.Categories included the importance of program community, affiliation, and program and institutional elements.
Program and institutional elements were divided into two subcategories: academic and social elements.

Participants and Data Collection
After approvals from all relevant Institutional Review Boards were obtained, data were collected from undergraduate and graduate students at one midsize, urban university located in the southern United States during fall 2021 and spring 2022 semesters.All students in online degree programs were invited to participate in the study via a mailing list.The invitation included the purpose of the study, its voluntary nature, and benefits and limitations of participation.A link to an anonymous online survey was embedded in the email invitation, and completers were informed that they could register for the drawing of one of ten $10 gift cards by providing their names and email addresses in a Google form.To increase the response rate, three reminders and "thank you" emails were sent (Dillman et al., 2014).

Data Screening
A total of 319 students responded to the survey.However, 26 cases had one third or more data missing and were deleted.The dataset included 16 outliers (z = ±3.0);however, these cases were deleted for the majority of analyses but not for testing the initial fit of the model.This resulted in 293 valid cases and a 14.4% response rate.Frequencies, mean scores, and standard deviations were generated.
Data were examined a priori to confirm parametric and multivariate assumptions were met.Bivariate correlations were examined for multicollinearity and correlations.

Descriptive Statistics
The means and standard deviations based on the data with outliers removed (more on the removal of outliers discussed later) are reported in Table 1.All means exceeded 3. Affiliation, α = .88.According to Kline (2016) and Nunnally and Bernstein (1994), reliability coefficients greater than .80 are considered adequate.The correlation coefficients among the items ranged from .37 to .74.Note.Scale items ranged from 1 (strongly disagree) to 5 (strongly agree).

Model Fit Without Outliers
We also ran the model removing outliers.Outliers were determined to be responses with a z-score equal to or greater than 3.0.We evaluated structural model fit with several fit indices, such as the CFI (values above 0.95 indicate very good fit and those at or above 0.90 indicate reasonable fit; Bentler, 1990), Steiger's RMSEA (values below 0.05 indicate a very good fit and those at or below 0.10 indicate a reasonable fit; Steiger, 1990), and the SRMR (< .05;Hu & Bentler, 1999).Model fit estimates for the validation model were either good or acceptable fit to the data (x² = 812.78,df = 340; CFI = 0.86; RMSEA = 0.08; SRMR = 0.07).

Structural Path Estimates
Given that both models exhibited an acceptable/good fit to the data, we present the model with outliers removed (Figure 1).Estimates of the structural relationships among the model variables were interpreted.

224
Elements (0.999).The correlation coefficients between the six factors ranged between 0.37 to 0.74 (Table 2).This suggests shared variance among the factors.

Figure 1
Confirmatory Factor Analysis Model

Academic Versus Social Activities
Across institutional and program events, participants rated academic activities higher than social activities for helping them feel like part of a program community.This finding is not surprising given the purpose of degree attainment and the nontraditional makeup of most online students, who often juggle family, professional, and personal activities with their educational pursuits (Milman et al., 2015;Stephen et al., 2020).It is noteworthy that although students rated completion of program milestones (M = 4.21), professional resource sharing (M = 4.14), and program advising activities (M = 4.02) higher than opportunities to attend academic program events outside of courses (M = 3.97), the latter was most predictive of the Academic Program Activities subscale.Similarly, although five of the six items in the Academic Institutional Activities subscale had higher mean scores than voluntary interest groups, the latter was the best predictor.Although all of these endeavors are useful for community formation and student persistence (Lee & Choi, 2011;Milman et al., 2015;Trespalacios et al., 2023), voluntary interest groups and extracurricular events may better capture student initiative to participate in community events because advising sessions, orientations, writing centers, and possibly research participation may be expected or mandated.Alternatively, some activities like orientations, advising, program milestones, and resource sharing may be expected norms in online programs and therefore not as indicative of program community efforts.
Remember, however, that the focus of this instrument was to move online community-building conversations beyond course activities.Thus, items regarding the role courses played in program community development were minimized.Although extracurricular events are the most predictive of Academic Program Activities, it does not mean that course activities are not useful for community formation.Ample research suggests that course endeavors play a major role in community formation (O' Shea et al., 2015;Rockinson-Szapkiw & Wendt, 2015;Rovai, 2001;Speiser et al., 2022).This instrument focuses on additional activities that lend to community formation.
Similarly, although institutional and program social activities were rated lower than academic activities, participants still rated them moderately useful for the formation of program community (M = 3.76 and 3.69 respectively).Interestingly, student-initiated activities emerged as the best predictor of the Social Program Activities subscale, even though professor-initiated social activities and virtual-remote social activities received higher mean scores.Again, it is possible the best predictive item captured student willingness and initiative to promote community.Additionally, although research suggests that most online students live within 50 miles of their institution (Seaman et al., 2018;Xu & Jaggars, 2013), participants rated in-person social activities lowest of all items in the social program activities subscale.This finding aligns with those of O' Shea et al. (2015), who reported that campus activities isolated some distance-based students by reminding them of unavailable opportunities.Despite this concern, the best predictor of the Institutional Social Activities subscale was in-person or remote institution-wide sporting events-despite having the smallest mean score on the subscale (behind institution-wide celebrations, equity and inclusion initiatives, and fine and performing arts events).
This finding regarding sporting events may demonstrate the desire of some students to align their educational interests and pursuits with the perceived prestige and branding of the institution (Dennis et al., 2016;Nathan, 2005).Students may want to tie their degree to an established and storied institution, believing that it may provide greater value than an online institution.More research is needed regarding this interpretation.Our study took place among online degree programs at a traditional brick-and-mortar institution.Students self-selected to attend this institution, and their views may not represent those of students who enroll in solely online institutions.Similarly, the institution competes in the top tier of collegiate athletics.In fully online institutions, university sports may not be a consideration.Yet, the predictability of institution-wide sporting events raises other questions.The United States has a long history of broadcasting collegiate sporting events to viewers across the nation via radio, television, and the Internet.
Indeed, the means to provide these social activities at a distance predates most distance programs.The same may not be said of other social activities.Performing arts events, gallery exhibitions, student association events, and other endeavors do not receive comparable treatment.Thus, it is possible sporting events best predict Institutional Social Activities because they are the only social events consistently provided at a distance.Additional research is needed at fully online institutions as well as those without collegiate sporting programs to see how perceptions of institutional social activities compare at those locations.

Time in Program
Pifer and Baker (2016) alluded to time being a factor that impacts learner perceptions of community.As students become more familiar with norms and values, and as they progress in their academic programs by completing coursework and milestones, their desire may change for academic and social opportunities to connect with others in their program (Baker & Pifer, 2011;Pifer & Baker, 2016).However, results of this study did not show that learners' time in program was a good predictor.Because most of our participants were students in their first or second year of studies, the dataset was limited.There is a need for future research with a more representative sample in terms of time in program that includes undergraduate and graduate students.

Instrument Validity and Reliability
Evidence from the validation model in this study supports inferences from the SOCS.Results show that the instrument has good reliability and a good factor structure that supports the theoretical framework developed by Shepherd and Bolliger (2023a).Correlations among most of the subscales were moderate or low.However, a strong correlation was found between two of the subscales: Academic Institutional Activities and Social Institutional Activities (r = 0.74).Caution should be taken when using these subscales individually.Therefore, we recommend administering the entire scale to a sample.
In conclusion, the SOCS instrument is a reliable and valid instrument that other researchers may use to investigate community in online environments on the program and institutional level.Practitioners and administrators may use the instrument to develop appropriate academic and social activities for and make resources available to online learners that support the formation of community and assist in sustaining it.
5 on a 5.0 scale (1 = strongly disagree, 2 = disagree, 3 = neither agree nor disagree, 4 = agree, and 5 = strongly agree) except for one item.Within the Program Community subscale, the item with the highest mean was item 5 (M = 4.42, SD = 0.68), which pertained to the feeling of belonging to the academic program.Completing program milestones (item 8) was the highest rated item for the subscale Academic Program Activities (M = 4.21, SD = 0.85).Virtual or remote social activities (item 14) was the highest rated item for the Social Program Activities subscale (M = 3.81, SD = 1.02).Academic support services provided by the institution (item 21) had the highest mean for the Academic Institutional Elements subscale (M = 4.14, SD = 0.86).Institution-wide initiatives regarding equity and inclusion (item 23) were most important for students in the Social Institutional Elements subscale (M = 3.87, SD = 0.93).With regard to Institutional Affiliation, students agreed the most with having a strong sense of affiliation with their current university (M = 3.76, SD = 1.06).Mean scores of all subscales ranged from 3.69 to 4.11.The Program Community subscale had the highest mean (M = 4.11, SD = 0.61), whereas the Social Program Activities had the lowest mean (M = 3.69, SD = 0.88).Internal reliability coefficients, using Cronbach's alpha, were adequate for all subscales: (a) Program Community, α = .80;(b) Academic Program Activities, α = .80;(c) Social Program Activities, α = .89;(d) Academic Institutional Activities, α = .84;(e) Social Institutional Activities, α = .87;and (f) Institutional

Table 1
Descriptive Information for Survey Items and Subscales (N = 277) The following elements help me feel like I am part of a program community:

Development and Validation of the Sense of Online Community Scale Shepherd, Bolliger, and McKim
28 (0.888), "I have a strong sense of affiliation with my current university," was the best indicator of Institutional Affiliation.Academic Program Activities was a better predictor of Program Elements (0.920) than Social Program Activities, and Academic Institutional Activities was a strong predictor of Institutional

Table 2
(Dennis et al., 2016)s Coefficients Among the Six Factors , reducing the need for brand affiliation(Dennis et al., 2016).Regardless, program community is desired by most online students and is supported by institutional and program activities.