Bruce A. Chadwick, Brent L. Top, and Richard J. McClendon, “Appendix A: Research Methodology,” in Shield of Faith: The Power of Religion in the Lives of LDS Youth and Young Adults (Provo, UT: Religious Studies Center, Brigham Young University; Salt Lake City: Deseret Book, 2010), 325–44.
The research findings reported in this book came from a number of different studies. Each was conducted with great attention to the principles of scientific research.We endeavored to collect data utilizing research techniques that yield findings that are not only statistically valid but also generalizable to populations of LDS youth. The procedures we followed in each study are reported below.
High School Surveys
Data were collected from the various samples of high school students via mail questionnaire surveys. The same data collection procedures, with some slight local variations, were employed in each of the survey samples. In addition, the same 12-page questionnaire was administered in each of studies. The questionnaire included questions measuring several different dimensions of religiosity, personality traits such as self-esteem and risk taking, measures of family life, attitudes about school and academic performance, delinquent activities including drug use and premarital sex, and peer pressures to participate in various activities. Most of the questions were close-ended, meaning the youth selected the appropriate answer such as “strongly agree” or “strongly disagree.” A few open-ended questions were included so the youth could speak about the influence of the gospel in their own words.
The scales measuring family life that were used in the East Coast study were replaced by the family characteristics of connection, regulation, and psychological autonomy in the later surveys. New research indicated that these three traits provided greater insight into the influence of family in the lives of teens. We added them to the questionnaire for the subsequent studies. This is why the data from the students living along the East Coast were excluded from some of the models which included these three family characteristics.
East Coast sample. The mail questionnaire survey was conducted in the spring of 1990 with a random sample of 2,143 LDS high school students between the ages of 14 and 19 in Delaware, New York, North Carolina, Pennsylvania, Virginia, Washington DC, and West Virginia. The sampling frame was the list of ninth- through twelfth-graders who were eligible to attend early morning seminary. This list of potential seminary students is drawn from membership records of the Church and includes all youth of the appropriate age, regardless of their level of church activity.
A packet was sent to the parents of the young people in the sample with a letter explaining the study and asking their permission for their son or daughter to participate. All of the mailings were sent first-class postage so that undeliverable packets were returned to us. Parents were informed that the questionnaire asked about sensitive topics such as drug use and sexual activity. If parents did not want their child to be in the study, they were asked to return the mailing labels in the business reply envelope. Those families that returned their mailing label were not bothered with any follow-up requests. Surprisingly, we did not receive a single refusal from parents by this means.
Parents were instructed to give the explanation letter, the questionnaire, and business reply envelope to their child. This letter was addressed to the teenagers and requested their honest replies to the questions. The confidentiality of the responses of the youth was strongly emphasized. We also stressed to both parents and teens that if meaningful data were to be collected, the teens had to answer the questionnaire in complete privacy. In order to encourage the youth to participate and to honestly answer the sensitive questions, it was promised that parents and Church leaders would never have access to their responses. Finally, the youth were asked to return the completed questionnaire in the enclosed business reply envelope. Using a business reply envelope allowed the research project to pay the postage for the return of the completed questionnaire.
The cover letters to both parents and youth listed a toll-free phone number and those with questions about the study were encouraged to call. A few parents did so with questions about who was sponsoring the study and how the findings would be utilized. The cover letters also pointed out that a number was printed on the business reply envelope for tracking purposes. It was explained that this number would marked out with black pen before the questionnaire was removed from the envelope. This maintained the anonymity of the information provided by each teenager. This number allowed us to track those who had responded and to limit follow-up mailings to those who had not.
A postcard reminder was mailed to the families who had not responded approximately three weeks later. One month after the postcard, a new packet was sent to those who had not returned the questionnaire. A final appeal for participation was made one month later with another full packet to hold-outs. The postcard and letters were modified for each subsequent contact.
One hundred twenty packets were undeliverable. Most of these packets were undeliverable because the family had moved and had not left a forwarding address with the post office. The youth in these families were removed from the sample, which reduced its size to 2,123. The four appeals obtained 1,398 completed questionnaires for a 69% response rate. An 80% response rate is the gold standard extolled in textbooks on research methods. Unfortunately, social scientists rarely achieve this high of a response rate. Most studies end up with a response rate in the 60s. We wish we had obtained a higher level of participation from the youth, but the 69% response rate is very good for this kind of study.
The data were entered into an SPSS data file for statistical analysis. A software program was utilized that required the data to be entered twice so that errors in entry were identified. This reduced data entry errors to very near zero. Finally, the responses to the open-ended questions were transcribed so they could be used as illustrations of the youths’ feelings and also for content analysis.
Pacific Northwest survey. The Pacific Northwest survey was conducted in the spring of 1995. This region was selected because sociologists have determined that it is the most secular part of the country. LDS youth received little social pressure to live the commandments in these communities, so it was likely that adherence to gospel principles would result from internalized religious values. We obtained permission from the Church and from the respective Area President to survey youth living in the Seattle and Portland areas. The potential seminary student lists were again utilized as sampling frames in those areas.
The data collection procedures used in the East Coast study were employed. Parents were sent a packet via first-class mail which included a letter addressed to them, a letter to their teen, a questionnaire, and a business reply envelope. Only three parents returned their mailing labels in order to remove their youth from the study. As in the earlier study, both parents and youth were informed that it was imperative that the youth be allowed to complete the questionnaire in complete privacy and that parents should not have access to the youths’ responses.
The sample was reduced by 81 youth because the packets were undeliverable by the postal service. We obtained 658 completed questionnaires from the 997 potential respondents for an acceptable response rate of 66%.
Utah County survey. The Utah County survey was conducted in the spring of 1995 concurrently with the Pacific Northwest survey. The potential seminary student lists from towns in Utah County, ranging from Lehi in the north to Provo in the south, were used as a sampling frame. We selected a random sample of 1,849 teenagers from these lists. An interesting note is that the youth in Utah County attended release-time seminary during the school day as compared to the youth in the other surveys who primarily attended early morning seminary.
We employed the same procedure of sending via first-class mail a packet addressed to the parents of the teen selected in the sample and asking permission for their child to participate. Only four parents overtly refused by returning the mailing label.
Only 39 potential youth were removed from the sample because of incorrect addresses. This number is low because staff members who live in Utah County were able to track down some families who had moved. Over 1,000 young people (1,122) completed questionnaires from the 1,810 potential respondents, which produced a 62% response rate.
Great Britain survey. We collected data from LDS youth living in Great Britain in 1999 because it is an English speaking country with a very low religious ecology. For example, in 1987 only 12% of the adult population in the United Kingdom reported they were affiliated with a religion or church. This compares to over 60% of adults affiliated with a church in the United States. Less than 8% of the British adult population indicated that they attended church on any given Sunday during 1999. In the United States, over 40% of adults attend some type of religious service each week. LDS youth trying to keep the commandments of the gospel in Great Britain find themselves going against the grain. Not only do they fail to receive social support for their beliefs and practices, but they are often ridiculed.
Data collection procedures used in the surveys in the United States were modified somewhat to fit the British Isles. First, potential seminary lists were not used as a sampling frame since most youth in Great Britain do not attend seminary. Instead we selected a representative sample of ten stakes scattered across Great Britain. Because of problems with postage, the Irish stake was dropped from the study. The membership rosters of families in the wards in the remaining nine stakes were used as sampling frames. In families with more than one teenager, we selected the oldest youth for the sample. This turned out to be a mistake, as will be explained later. These procedures produced a sample of 1,490 LDS teenagers.
The data collection was supervised by Renata Forste and Cory Harmon, who were teaching at the BYU London Center at this time. They hired and supervised research assistants who prepared the packets and recorded the returned questionnaires.
The questionnaire was modified slightly to conform to British society. For example, questions dealing with money were changed from dollars to pounds. Also, questions about school attendance were altered to match public schools in England.
As in the United States, a packet containing a cover letter asking parents’ permission for their son or daughter to participate was sent via the British equivalent of first-class mail. The packet also included a letter relevant to LDS youth in Great Britain, a questionnaire, and a business reply envelope. The parents were requested to permit their teen to complete the questionnaire in privacy.
The cover letter to both parents and youth called attention to the number printed on the business reply envelope. It was explained that this number allowed the researchers to identify those who had responded and to spare them follow-up contacts. It was emphasized that the number was marked out before the questionnaire was removed from the envelope to preserve anonymity. In addition, a toll-free number to the BYU campus in Provo, Utah, was listed. Parents and teens were encouraged to call with questions. A few called asking about who was conducting the study and what its purpose was. A senior member of the research team answered these calls.
Two weeks after the initial mailing we sent a postcard reminding those who had not returned the questionnaire to please do so. About five weeks after the first contact another packet with cover letters, questionnaire, and return envelope was sent those that had not responded. Finally, two months after the initial mailing, we sent a third packet of the research material to those who had failed to return the questionnaire.
The sample of 1,490 was reduced to 1,286 because 214 packets were returned by the postal service due to inadequate addresses. In addition, 31 parents either wrote or called refusing to allow their teen to participate. This parental resistance is stronger than we encountered in the United States. The parents who thus refused appeared to not be of the LDS faith and did not wish for their son or daughter to be involved in any way with the Church, including participating in a Church-related survey.
Despite four contacts with the family, we received only 475 completed questionnaires from the 1,286 potential youth. This response rate of 38% is considerably lower than what we obtained in the United States survey samples. Some families or individual youth had joined the Church at an earlier time but had subsequently fallen away and no longer considered themselves members. They refused to participate because they wanted to distance themselves from the Church and also because they felt the survey did not pertain to them.
A second reason was the selection into the sample of the oldest child in a family. We did not realize that after age 16 youth in the British Isles can leave their equivalent of high school. Some go on to additional education, while others enter the labor force. Parents were reluctant to forward the questionnaire to “adult” children who had left the home. And when parents did forward the materials, they often came back to us undeliverable. Finally, the British are more hesitant about participating in social research, especially a mail survey conducted by a university in the United States.
To identify who had refused to participate in the study, we telephoned a sample of 50 bishops and asked them about the level of church activity of the 179 youth in their wards. Bishops were given the name of the teens in the sample residing in their ward and were asked to describe them as “active,” “semiactive,” or “inactive.” “Active” denoted a high degree of participation in Sunday worship and church activities. “Semiactive” referred to youth who occasionally came to church and church-related activities. “Inactive” identified teens who rarely or never attended church functions. Youth whose names were not recognized by the bishop were assumed to be inactive.
The information from the bishops revealed considerable bias in participation in the study. For example, 64% of those identified as “active” returned their questionnaire compared to only 18% of the “inactive” who did so. This 64% response is comparable to what we obtained in the United States among active LDS youth. Obviously, most of the bias was a consequence of those having little, if anything, to do with the Church not returning the questionnaire.
This bias does prevent us from generalizing about the activities of all LDS youth in Great Britain because inactive teens are underrepresented. But the sample is appropriate for testing the model predicting the behavior of LDS youth. The fact that the sample contains youth with a broad range of religious activity demonstrates religiosity’s relationship to delinquency and other behaviors.
Mexico survey. The survey of Mexican LDS teenagers offered us the opportunity to test the influence of the gospel in the lives of youth from a different culture who also speak a different language. Data were collected from the students attending the LDS-sponsored boarding school Benemerito de las Americas, located a short distance from Mexico City. In addition, a sample of youth in several stakes in the Mexico City area was selected for participation.
The questionnaire was translated into Spanish using the back-translation method. A small team of bilingual individuals translated the English questions into Spanish. A different team translated the Spanish questions back into English. The two English versions were compared and any differences were corrected in the Spanish version.
The questionnaires were administered by Church Educational System personnel, primarily teachers at Benemerito and seminary teachers in Mexico City. Teachers in the school simply passed out the packets, which included a cover letter, a Spanish version of the questionnaire, and an envelope in which to place the completed questionnaire. Parental consent laws are not as rigorous in Mexico as in the United States, and since the youth were attending a boarding school, parental permission to participate in a school-sponsored survey was implicit. For the teens not attending Benemerito, parental consent was obtained. Seminary teachers in the Mexico City stakes passed the questionnaires out in class and asked the students to take them home. The parents were asked in a cover letter which the students took home to allow their son or daughter to participate in the study.
We marked a set of questionnaires for use in Benemerito so we could differentiate them from those filled out by youth living at home. Unfortunately, those collecting the questionnaires mixed the questionnaires together before administration so that it was impossible to identify from which population a given questionnaire came.
We obtained 1,303 completed questionnaires from the Mexican students. CES employees assure us that nearly all the students attending Benemerito participated and that the response rate is probably around 95%. The response rate for the students living at home was estimated by CES employees to exceed 80%.
Castle Dale survey. The study in Castle Dale, Utah, was conducted at the request of educational and Church leaders in the community. We were pleased to accommodate their request, because it provided us an opportunity to sample another religious ecology. Castle Dale and surrounding communities have a large LDS population that is nonetheless different from Utah Valley’s which had been surveyed earlier. In contrast with the youth surveyed in the earlier Utah study, high school students in Emery county generally had many non-Latter-day Saint peers. As a result, the study in Castle Dale provided us with a different snapshot of LDS youth in Utah. The area is not only different as a religious ecology, but also considerably different in socioeconomic ways. This sample is predominantly rural, whereas the previous Utah sample was primarily suburban.
The data were collected by the seminary staff. The research packets were passed out in class and the students were asked to take them home to their parents. If parents were agreeable to their son or daughter participating, then they gave them the questionnaire and business reply envelope. The completed questionnaires were returned to the seminary teachers, who collected the sealed envelopes and delivered them to us. With this process, the study was completed much quicker than usual as we did not have to wait for four time-consuming mailings.
As was done in the other studies, a cover letter listed a toll-free number at BYU and parents and students were invited to call if they had questions. Very few did so. The seminary teachers made several appeals to the students but we did not do any systematic follow-ups. Completed questionnaires were obtained from 354 youth, or about 70%. The data were analyzed in detail and a lengthy presentation was made to Church leaders and parents.
Utah County Unwed Mothers Survey
We were surprised to discover in our surveys of LDS high school students that a larger percentage of young women were sexually active than young men. This was an anomaly in adolescent research because young men in American society had a much higher rate of early sexual experience than girls. We were puzzled as to the difference between LDS youth and youth in society at large. To shed some light on the sexual initiation of LDS young women, we conducted an exploratory interview study of young women in Utah County who were 19 or younger and had a baby born out of marriage. The study was conducted during the summer of 1998.
One of the major methodological problems in the study of premarital sexual activity is the validity of self-reported sexual behavior. Some teens are afraid of the consequences of such behavior and deny their involvement. On the other hand, others brag about exploits that never happened. We conducted this survey among women with a baby born outside of marriage because they obviously had been sexually active.
Data were collected through a lengthy, in-depth, face-to-face interview. The interviewer asked a leading question about a particular aspect of the teens’ life and then let them talk. An example was “Tell me about school and how you did.” If insufficient detail was given, the interviewer asked a probing question to stimulate discussion. An example was “Which was your favorite class? Which was your worst?” The basic areas asked about where family life, school activity, friends, dating, sexual activity, religion, how they dealt with their pregnancy, and their plans and hopes for the future. The interview generally lasted an hour and a half, but some ran well over two hours. The interview was tape-recorded with the girls’ permission.
The interviews were conducted by a young woman graduate student. She was only three or four years older than the girls she interviewed. This similarity in age helped her establish good rapport with these teenagers. Also, she dressed up a little to match the role of interviewer, but not so formally as to inhibit the girls from telling their stories.
This exploratory study did not select a sample because this is a very difficult population to identify. Rather, we located young unwed mothers from a variety of sources. A poster was placed in a doctor’s office whose practice included unwed mothers. An advertisement seeking unwed mothers to be interviewed was placed in the local newspaper. We also placed a poster in the alternative high school were many unwed mothers finish their high school careers. The girls were paid a $100 honorarium for their time. Once the study commenced, a few girls who heard about the study by word of mouth called to volunteer so they could earn the honorarium.
Fifty girls were interviewed. Unfortunately, the tape recorder malfunctioned and the interviews were lost. The interviewer transcribed the interviews. Since she had conducted the interview, she usually could remember any nuances in the girls’ answers as she listened to the tapes. The interviews averaged 12 double-spaced pages and contained a wealth of qualitative information.
A team of three research assistants independently conducted content analysis of the written comments. They first worked together to develop a series of quantitative categories for important topics in the interview such as the reason(s) for the girls’ initial sexual experience or the quality of their relationship with their father. Once the categories were established, each research assistant independently coded the comments into them. The three independent codings were then compared and disagreements resolved. These procedures produced quantitative values from these subjective comments with at least 85% agreement among three research assistants. These data were then entered into a SPSS file, which allowed statistical analysis.
BYU–Provo survey. A 2001 study of 1,000 young women attending four-year colleges and universities across the United States found that “dating” has all but disappeared from American college campuses. Only half of the women reported they had been asked on six or more dates during their entire college career. In fact, one-third of the women had two or fewer dates during the same four years. Instead of dating, college students now “hang out” in mixed groups. From these associations young people pair off and “hook up” with a member of the opposite sex.
To ascertain whether this shift in dating culture had spread to the BYU campus we conducted a mail survey of BYU students during Winter Semester 2002. We developed a 14-page questionnaire that asked the students about their hanging out and dating activities, their attitudes about marriage and family, personality traits including self-esteem and a scale of depression, traits they were seeking in a mate, traits they felt they would bring to marriage, reasons for delaying marriage, school performance, and religiosity. The questionnaire was pretested, then revised and pretested a second time. The pretests were conducted with seniors who graduated before we started data collection so they did not contaminate the study.
We selected a 5% sample of the 35,000-plus students listed in the electronic student directory. The sample of 1,893 students was sent packets with a cover letter explaining the study, a questionnaire, and a business reply envelope. The cover letter explained the intent of the study and asked the students to participate by completing the questionnaire. We asked those that had married during the previous semester, to answer about their behavior during the semester before they did so.
Attention was called to a number printed on the business reply envelope. It was explained this number would be marked out before the envelope was opened so that the student’s responses remained anonymous. In addition, a toll-free number was listed and students were encouraged to call if they had questions about the study.
Two weeks later a postcard reminder was sent asking the students to return the questionnaire sent earlier. If it had been misplaced, they were invited to call a toll-free number and request another. Five weeks after the initial mailing, another complete packet was mailed to those who had not responded. Two months after the first contact, another complete packet was sent pleading for their participation in this important study.
One hundred fifty-five packets had incorrect addresses. We attempted to find forwarding addresses from school records but we were not very successful. We were surprised that 176 potential respondents either telephoned or wrote to explain that they were older married students who were taking correspondence courses. Because their dating experience had occurred many years earlier, they removed themselves from the study. We obtained completed questionnaires from 1,124 students for a 72% response rate. This is a rather high response rate for a mail survey but was probably influenced by the topic and by the study being conducted by BYU faculty. We are confident that if we could have removed all of the Independent Study students who were not living on campus that the response rate would have perhaps exceeded 80%.
A little over 70% of the respondents were single, and the responses from these 784 students are reported in this book.
BYU–Idaho survey. In 2003 we replicated the study among a random sample of students attending BYU–Idaho. The same procedures were followed and 1,187 completed questionnaires were obtained. The response rate was 73%, which is similar to that obtained at BYU–Provo. The findings of the BYU–Idaho students were nearly identical to those of BYU–Provo. In order to not slow down the production of this book, and because the results were so similar we chose not to redo the analysis for the dating chapter with the BYU–Idaho data.
BYU–Hawaii survey. The following year, 2004, we repeated the study among students attending BYU–Hawaii. Given that only approximately 2,000 students attend this campus, we surveyed the entire student body rather than a random sample. We were surprised with the high percentage of students attending the Hawaii who hail from Taiwan, Japan, China, Polynesia, and other Pacific Rim countries. If students returned home before completing the survey, it was difficult to follow-up with them. This influenced the response rate which was 67%, which is only slightly lower than in Provo and Rexburg.
We anticipated that the students attending the Hawaii campus would be somewhat different than those in Provo or Rexburg because of the variety of cultures and the social environment of the Island. We expected them to be a little more liberal in attitudes about drinking and sex and to engage in more deviant behavior. Such was not the case, and their responses were very similar to the other two campuses. Again, we opted not to redo the analysis for the dating chapter with the Hawaii sample.
Returned Missionary Study
When Bruce Chadwick served as a bishop on the BYU campus, he was frequently asked how the returned missionaries in his ward were doing in general and in the Church in particular. He regretted that he did not have a reasonable answer to these questions. This was the major motivation behind this survey of returned missionaries.
At the time of this study, Richard McClendon was a graduate student in the BYU Department of Sociology. He was also a seminary teacher on leave from the Church Educational System. This study was part of his doctoral dissertation.
During the winter of 1999, four random samples were generated of 1,000 men and 500 women who had been back from their missions approximately 2, 5, 10, and 17 years. Originally, the older cohort included men who had been home 15 years, but we discovered that these missionaries had served during the brief era of 18-month missions. The shorter mission would have made analysis much more complex, so we opted instead for missionaries who had been home for 17 years.
The random samples included returned missionaries who came from both the United States and Canada. As we prepared to print the business reply envelopes, we encountered some serious problems with business reply mail from Canada. Thus we decided to delete the returned missionaries from Canada. The loss was modest with only 71 men and three women being dropped from the samples.
A rather exhaustive questionnaire was developed and pretested on returned missionaries who had been home for a length of time that so they were excluded from being selected in the samples. The questionnaire went through at least six revisions. Questions were asked about the missionary’s high school experience and what he or she did during the time after graduation and before their mission call. Another set of questions focused on their mission experience. A third set of questions asked about what the young man or woman did during the first year they were home from the mission field. The final set of questions focused on their current life. We asked about religious beliefs and behaviors before, during, and immediately after their mission, and their current activity. Questions focused on school and academic performance. Others asked about dating, marriage, and family life. Finally, a number of demographic items were assessed such as marital status, number of children, educational attainment, employment, income, and church positions held.
During the first two weeks of March 1999, a packet was sent via first-class mail to the sample of men. The packet included a cover letter explaining the study and asking for their participation, a questionnaire, and a business reply envelope. The women’s sample was surveyed two weeks later during early April.
A toll-free number was listed in the cover letters and postcard, and those with questions about the study were invited to call. A number of men and women telephoned and asked who was sponsoring it, why was it being done, how were the results going to be used, and how would the results be made available to the membership of the Church. Senior members of the research team fielded these phone calls.
The cover letter explained that the number on the business reply envelope would be blacked out before the envelope was opened. This protected their anonymity while also allowing us to identify those who had not returned their questionnaire. A postcard was sent to those who had not responded approximately three weeks after the initial mailing encouraging them to do so.
Approximately six weeks after the first mailing, another complete packet including a modified cover letter, questionnaire, and business reply envelope was sent to nonrespondents. A fourth and final packet was sent to the holdouts approximately three months after the initial contact asking one last time for their participation in the study.
The postal service was unable to deliver 332 of the packets sent to men which reduced the sample to 3,081. Competed questionnaires were obtained from 2,078 men for a 67% response rate.
The nondeliverable packets numbered 194 for the women, which reduced their sample to 1,802 potential respondents. Among the women, 1,535 participated in the study, producing an 85% response rate. We are not really sure why the women were more willing than the men to participate in the study.
The responses to the open-ended questions were transcribed so they could be used as illustrations of the returned missionaries’ feelings. In addition, important content analyses was conducted on questions such as “What could Church leaders have done to ease your transition after your mission?”
This study was prompted by the discovery that only 30% of the young men of the Church who live in the United States currently serve full-time missions. This observation elicited questions about why 70% of the young men in the Church choose not to serve a mission. Also we were interested in what happens with these men in later years. This study was conducted in 2000 and was a follow-up to the previous study of returned missionaries. This study was the doctoral dissertation of Darrell Janson, a graduate student in the BYU Department of Marriage, Family, and Human Development and a seminary teacher.
We attempted to match the men who did not serve missions with the samples from the returned missionaries. Since missionary service is not emphasized for young women, we excluded women from this study. We drew samples of four cohorts of men who were 23, 26, 31, and 38 years old who had not served a mission. The sampling procedures produced a sample of 3,796 men who had not served.
In September 2000, packets containing a cover letter, a questionnaire, and a business reply envelope were sent through first-class mail to the men in the samples. The cover letter explained that the study was being conducted by the Family Studies Center at BYU and was interested in LDS young adults who had not served missions for the Church. They were encouraged to participate regardless of their present Church activity. The number on the business reply envelope was explained as a means to note who had responded so as to spare them any follow-up contacts. They were assured that the number would be blacked out before the envelope was opened so their responses would be anonymous. Those who did not wish to participate were encouraged to return the empty envelope. Those that did so were removed from the samples.
About two months later, postcards were sent to encourage the nonrespondents to participate. Approximately four months after the initial mailing, a new packet containing a revised cover letter, questionnaire, and business reply envelope was sent. Because of the very negative response of the men in the sample to the study, the usual fourth mailing was not sent.
The cover letters and postcard listed a toll-free telephone number, and those with questions about the study were invited to call. Some days people called to complain. They were upset at being contacted by BYU, an affiliate of the LDS Church. For the most part these were men who had dropped out of the Church years earlier and wanted nothing to do with it. We attempted to soothe their concerns but promptly removed them from the samples.
The sample was reduced by 28 men who had actually served missions. Another 466 were dropped because we could not find a correct address. The packet sent to them via first-class mail came back undeliverable. Thus the sample was reduced to 3,302 men including 697 23-year-olds, 737 26-year-olds, 926 31-year-olds, and 942 38-year-olds. Only 380 completed questionnaires were returned for a 12% response rate. We had anticipated a low response rate given the population; we certainly were not prepared for 12%.
In an effort to acquire some information about the church activity of the nonrespondents, we selected a sample of 230 of the nonrespondents. The current bishops of these 230 men were telephoned. To maximize contact with the bishops, calls were made at different times of the day and during different days of the week, including weekends. One hundred seventy-five of the 230 bishops were contacted. Twenty-seven (15%) of the bishops had never heard of the person. They were confident that the potential respondent was not on their ward roll. One hundred fifteen of the bishops, 66%, reported that the man we asked about was totally inactive. The bishops were confident there had been no contact with the man by home teachers. The other 33 bishops, 19%, reported that the man asked about occasionally came to church. Very few of these relatively active men attended more than once a month.
It is very obvious that the 12% who responded was heavily biased with an overrepresentation of somewhat active men. Those who had physically and emotionally withdrawn from the Church are vastly underrepresented. Perhaps the most significant finding to emerge from this study was the loss of so many young men. It was extremely disconcerting to see how many men had completely dissociated themselves from the Church. We had originally anticipated that many of the men who had not served missions later met a young woman who wanted to be married in the temple. We had hoped they many had become worthy to enter the temple and then continued to live the gospel. Such was not the case. Once these young men dropped out, many while still in Primary, they became alienated from the Church with little prospect of return.